Building AyuSwar: Voice-First Health AI for Indian Languages

Picture a busy hospital OPD in India and the first five minutes of a consultation. The doctor types. The patient waits. Now make that patient a 70-year-old who is most comfortable in Telugu, speaking the half-Telugu, half-English mix that real consultations sound like. A text-first app can be a wall for people like that.

In an emergency, typing may not be an option at all. So I built AyuSwar (Ayu means life, Swar means voice) to explore one question: how far can voice alone take a health app? Speak to it in your language, and let it listen, understand and answer back.

Honesty up front: AyuSwar is a prototype built on synthetic data. It is not a medical device, it is not clinically validated, and it does not give diagnoses. In a real emergency, dial 112 or 108.

Everything in AyuSwar hangs off one 4-stage pipeline. Each stage carries a status chip, and I want to be clear about what those chips are: they are my own labels for where each stage stands, not an independent audit. Here is what they currently say:

The live /technology page draws this pipeline in real time, with LIVE, DEMO and PLANNED chips on each stage.

Many patients do not say "I am experiencing palpitations". In my native Andhra, someone is more likely to say "గుండెల్లో దద" (gundello dada). So one core piece of the app is a vernacular mapping table: regional idioms in Telugu, Hindi, Tamil, Malayalam, Kannada, Marathi and Bengali, linked to standard clinical concept codes. This is a real excerpt from the repo:

// src/data/indicVernacularOntology.js (real excerpt, trimmed) { id: ' ont_01 ', vernacular: ' గుండెల్లో దద ', // Telugu, Rayalaseema / Coastal Andhra transliteration: ' Gundello Dada ', clinicalTerm: ' Palpitations & Cardiac Flutter ', icd10: ' R00.2 ', // WHO ICD-10 snomed: ' 80313002 ' // SNOMED CT } Enter fullscreen mode Exit fullscreen mode Two things to be clear about. First, this is a developer-authored mapping. I am a student builder, not a clinician, and the table is unvalidated: it is a structure waiting for clinician review, not an authority on symptoms. Second, that is exactly why it is plain data anyone can read and challenge on GitHub, instead of a black box.

Code-mixing is the default, not the edge case. A real sentence sounds like "doctor garu, naa chest lo burning sensation undi". Anything that only understands clean English or clean Telugu will fail people who speak like this, so the language handling assumes mixed input.

The Audio-Rx studio turns a prescription into spoken instructions in the patient's own language, with word-by-word karaoke highlighting over colour-coded pill cards. It can then format the schedule as text and open WhatsApp with it ready to send; the send itself is always a human tap. The OPD Ambient Scribe drafts a structured SOAP note from a consultation conversation and is designed around a rule the site states openly: a doctor reviews before anything is exported.

The Acoustic Signal Lab measures physical signal properties in the browser (F0, spectral centroid, energy flux) and labels itself on the page as non-diagnostic. The Sanjivani-108 emergency console triages a spoken distress call and prepares a hospital handover, with dispatch clearly marked as simulation and a 110 BPM CPR metronome on screen.

The Audio-Rx studio with sample medication cards - synthetic demo data, not real prescriptions and not medical advice. On-screen wording like "60 FPS" and "direct WhatsApp dispatch" is the site's own UI copy: aspirational, not measured performance, and the WhatsApp send is always a human tap.

The most deliberate part of the codebase is the test suite, because much of it checks what the app must NOT do. Alongside the repo's Node test files, the /benchmarks page defines a contract-verification suite that runs in the browser against the app's own services; the assertions are written and wired, and I am still verifying a full green run end to end. They include: the assistant must refuse to prescribe drug doses, an emergency phrase must trigger a 112/108 warning, the acoustic analyzer must return isDiagnostic: false with a disclaimer, and no secrets may leak into the client bundle.

The /benchmarks runner is built to execute these assertions in the browser and export a software test report; a full verified green run is still in progress.

Most of my tests check what the app must never do.

What is unfinished: the OPD scribe's full voice-capture pipeline is still under end-to-end test; the Gnani Prisma (speech-to-text) and Timbre (text-to-speech) adapters are wired with a browser-speech fallback while the deployed provider configuration is verified; and all patient data on the site is synthetic. The vernacular mapping needs review by someone with clinical training before I would trust it beyond a demo. Next steps are technical: finish the scribe pipeline, complete and verify the provider integration, expand the contract tests, and keep hardening the mobile layout.

Built for the Great Indian AI Internship Challenge 2026 by Gnani AI. All patient data shown is synthetic. #GreatIndianAIInternshipChallenge #GnaniAI