How MediKiosk works
The loop
A patient answers at the kiosk by speaking or tapping, in their own language, and scans the paper they already carry. The system assembles a structured history and puts it on the doctor's screen before the patient sits down. The patient hears their own history read back and confirms it. The doctor verifies, amends anything wrong, is warned about drug interactions as they prescribe, and what they write becomes the baseline the next follow-up starts from.
Against the problem statement SIH26047 §3.2–3.4
Every line the statement asks for, what this build does about it, and where to see it. Built means working in this deployment; Partial means working with a stated limit; Mocked means the code path exists but the outside system is not connected.
| Module | Asked for | Status | Where to see it |
|---|---|---|---|
| A | Adaptive questioning branching on the chief complaint | Built | Kiosk → choose any complaint: the model writes 4–5 follow-up questions for it (SOCRATES elements, tap-able options, in the patient's language with English beneath, "asked because you mentioned…"). Generated once per complaint and language, then shared; a hand-written block is the fallback. Works in all seven languages. |
| A | Dual-mode input — every question by speaking or tapping | Built | Mic on every open question; spoken "next / back / repeat" commands. |
| A | AYUSH history mode — Dashavidha Pariksha, Agni, Koshtha, Ahara-Vihara | Built | Choose Ayurvedic or Both at the system screen; all ten parameters, Vikriti kept apart from Prakriti. |
| A | Red-flag detection with immediate alert to triage staff | Built | Red button, or a red-flag answer: token turns P-, case tops the queue, a tone and banner on the console, one SMS to TRIAGE_PHONE. |
| A | Indian-language ASR (Bhashini / AI4Bharat) | Partial | Browser speech recognition today, all seven languages; Bhashini needs an API key and is the planned swap. |
| A | Text-to-speech for audio prompts | Built | Every screen reads aloud — heading, hints, every option — in all seven languages, on any machine. |
| B | OCR of printed and handwritten prescriptions, lab reports, discharge summaries | Built | Kiosk → scan; a handwritten Ayurvedic prescription is read line by line, doses included. |
| B | Intelligent extraction — diagnoses, medicines with dose, values with reference ranges | Built | Console → open a document. |
| B | Chronological organisation into a timeline | Built | Ordered by the date printed on the paper; undated papers are said to be undated, not guessed into place. |
| B | Abnormal-value highlighting | Built | Out-of-range values in red on the console and in the patient's own record. |
| B | Potential drug interactions | Built | Console: an amber panel — scanned papers + reported medicines + today's prescription as you type it. Curated rules, allopathic and Ayurvedic pairs. |
| C | Standard clinical format: CC → HPI → past → drugs/allergy → family → personal → ROS → investigations | Built | Console → structured history. |
| C | Editable and verifiable — a draft the physician accepts, amends or rejects | Built | Accept as written, or amend and accept; attributed to the clinician. |
| C | Bilingual output — patient-facing audio confirmation in local language; physician summary in English | Built | Kiosk "Is this right?" screen: "In your language" block with a Listen button. |
| D | ABHA authentication | Mocked | ABHA numbers are validated by format and generated locally; issuing them needs ABDM sandbox credentials. |
| D | Consent-first: granular, revocable, with audio explanation | Built | Consent screen read aloud; four separate permissions; "Withdraw consent" on the patient dashboard locks every clinician out of that visit. |
| D | Secure processing within the platform | Partial | Voice never leaves the kiosk as audio; document and summary text go to a model API over TLS. A deployment would point the same code at a hospital-hosted model. |
| D | Session termination — temporary data cleared after submission | Built | Finish clears the screen; a patient who walks away is asked at 90 s and the screen wipes itself 30 s later. |
| D | FHIR push to HIS/EMR, linked to the ABHA PHR | Mocked | A real FHIR R4 document Bundle — Composition, Patient, Practitioner, Encounter, Condition (ICD-10 + NAMASTE), Observation, DocumentReference, Consent — downloadable from the console. Only the transport is mocked, pending sandbox access. |
| Access. | Audio-guided mode; large text / high contrast; multilingual UI | Built | Accessibility screen: three switches. Sign-language avatar (stretch goal) not attempted. |
| Dash. | Personal login ID and password; patients keep and see their records | Built | Login ID sent by SMS; "My records" shows every visit, document, diagnosis and prescription, read-only. |
| Journey | Identify → Converse → Scan → Summarise & route → Consult | Built | Token slip with QR; department routing to a per-department queue; clinician console with queue, case, home. |
A three-minute demo
- Kiosk, Marathi. Tap मराठी. Tap the speaker: the screen is read aloud. Mobile number → the code is shown on screen (no gateway in demo mode) → name, age → account created, login ID on screen.
- Consent is read out. Leave all four on. First visit → Both systems.
- Answer three questions by tapping; say one answer into the mic. Pick "cough" as the complaint and show the follow-up questions change.
- Scan the sample handwritten prescription (
tools/bench-samples/). Watch it come back labelled and dated. - "Is this right?" — tap Listen: the patient hears their history in Marathi. Confirm. Token slip with QR.
- Console (second screen): the case is already on the queue, department assigned. Open it: structured history, the scanned paper read line by line, abnormal values, the amber interaction panel. Type "Diclofenac" in the prescription — the panel updates. Save reading for height and weight. Mark assessed.
- Download FHIR R4 bundle. Open the JSON. Then, on the kiosk, sign in with the login ID → My records → Withdraw consent → the console can no longer open that visit.
- Emergency: from any screen press the red button — the console sounds and shows the banner, the token turns P-.
Real in this build
Three ways to identify — mobile, ABHA number, or Aadhaar with a real Verhoeff checksum · a personal login ID and password so a patient can return to their own dashboard of past visits, documents and summaries · adaptive question flow with first-visit, follow-up, proxy and emergency branching, in all seven languages, and complaint-specific follow-up questions written by the model for each complaint · voice input, spoken prompts for every screen and spoken navigation commands in every language, with a server voice for any machine that has none of its own · audio-guided, large-text and high-contrast modes · a literacy-free body map, drawn again with the touched zone lit on the review screen, in the patient's record and on the physician's console · all ten Dashavidha Pariksha parameters with Vikriti separated from Prakriti, plus Agni, Koshtha and Ahara-Vihara · document capture with AI extraction of diagnoses, medicines and lab values · a chronological document timeline · AI clinical summary with assessment, differentials and suggested investigations, and a plain recap in the patient's language they can hear · ICD-10 and NAMASTE coding suggestions · rule-based red-flag triage with an SMS and a console alarm · curated drug-interaction check, live as the prescription is typed · height, weight and BMI taken at the desk · real SMS delivery when a gateway is configured · clinician registration with HPR ID and approval gating · a clinician home with today's numbers, next up, my patients and record search · live queue · automatic department routing with per-department queues, audited override and reassignment · single-action history verification with amend · prescription write-back · granular and revocable DPDP consent · idle wipe · FHIR R4 document bundles · rate limits on every endpoint that spends a quota · full audit trail.
The models behind the summary and the document reader run on free tiers — Groq first, Mistral as fallback — and the app degrades to an assembled summary and stored-unread papers, saying so on screen, if neither answers.
Mocked, and why
ABHA numbers are generated locally in the correct format, and the FHIR bundle is built for real but not transmitted, because ABDM sandbox credentials require an application and an approval period. Speech recognition runs on the browser's own Web Speech API, which is Chrome-only; a deployment would use Bhashini, and that needs a key. The drug-interaction check is a short curated table, not a formulary. The translations into Marathi, Gujarati, Punjabi, Tamil and Telugu are machine-assisted and under native-speaker review. The complaint-specific question sets follow standard history-taking structure and need a vaidya's review before use on real patients. Everything else — including SMS — is real once configured.
Seven languages, the whole interview
Hindi, English, Marathi, Gujarati, Punjabi, Tamil and Telugu — not the buttons, the whole interview: every question, every option, every consent line, every notice. A string nobody has translated shows English rather than Hindi, because a Tamil reader cannot tell a Hindi sentence from a bug. Sanskrit terms — Prakriti, Vikriti, Agni, Koshtha — are transliterated into the reader's script rather than translated, because a physician uses them by name. Red-flag detection, department routing and the complaint classifier follow the languages too. A review sheet per language lets a native speaker correct any line and send the corrections back as text.
Terms of use prototype
MediKiosk is a Smart India Hackathon prototype for problem statement SIH26047. It is not a medical device and not a live clinical service. Nothing it shows is a diagnosis: the summary, the department suggestion, the coding and the interaction notices are drafts for a registered clinician to confirm or reject. By continuing, you agree that the information you enter is used only to prepare a history for the clinician who sees you at this visit, under the permissions you choose on the consent screen, and that this prototype may be reset at any time.
Privacy DPDP Act 2023
What is collected. Your name, age, sex, mobile number, the answers you give, the papers you scan, and the height and weight recorded. What is not. Your voice: speech is turned into text on the kiosk and the audio is never stored. Who sees it. Only the clinician seeing you at this visit, and only while you allow it — every time a record is opened or searched, the clinician's name is written to an audit trail. Where it goes. The text of your answers and your scanned papers is sent to a model service to be read and summarised; the spoken prompts you hear are fetched from a speech service, but never your own words. Your control. Every permission on the consent screen can be turned off and you are still seen. From your dashboard you can withdraw consent for any past visit; no clinician can open it after that, and you can allow it again. Retention. A kiosk screen left alone clears itself within two minutes. This prototype's records may be deleted at any time.
Why this is not what already exists
Hospital registration systems capture demographics only. Health apps assume a smartphone, literacy and enrolment before the visit. Nurse-led history desks do not scale to five thousand patients a day. None of them digitise the paper the patient is already holding. This does both — deep structured history and document digitisation, in the patient's language, read back to them — before the encounter begins.