Back to Nayanaviram's profile
12views

AarogyaRekha: Multimodal Rule-Based Triage Assistant

Clinical triage, made versatile and effective.

Nayanaviram Panigrahi
  • Video of it working
  • Problem, approach and result explained
  • Public source code
  • Answered the full brief· 13 of 13
  • Expert reviewed· 6 reviews
AarogyaRekha: Multimodal Rule-Based Triage Assistant

95%

Triage rule mapping with source and subsequent question handling

95%

Patient Data anonymization and secure sharing via FHIR

70%

RAG generated questions and their accuracy to patient symptoms and mapped triage rules (connection successful; low accuracy)

Walkthrough

Overview

The problem

Crowded outpatient clinics in India have too few doctors, so a nurse decides who is seen first, usually by instinct and in Hindi or Odia. We built a triage assistant that helps the nurse ask the right few questions and see a clear priority, while the doctor stays in charge.

The approach

A deterministic, explainable rules engine decides the tier, and every decision is logged. A local embedding model only picks which questions to ask. We left out diagnosis, treatment advice and emergency-room decisions, and AI alone never changes the tier.

The result

Desktop app: an offline-first Windows app (Electron) with React 19, Vite and TypeScript.

Backend and data: a local Fastify backend and bundled PostgreSQL, with sign-in and two-factor authentication.

Engine and selection: a 350-rule engine with , tiers 1 to 4, most urgent wins, logging rule ID, version and hash. An embedding model matches the complaint to the rules and picks four questions. Pgvector; storing embeddings for a clinical knowledge graph that links symptoms to parent concepts and SNOMED/ICD codes.

Language and records: English, Hindi and Odia, voice and read-aloud, PDF and scanned-report reading, FHIR R4 referrals.

The tricky part: picking the right four questions from 350

Reflection

We would get a clinician to review the rules, then build a labelled set of real complaints to measure and train on.

We would improve the accuracy of the embeddings and a clinical knowledge graph in Postgres with pgvector, so symptoms link to parent concepts and standard codes, and relevant questions are generated.

We'd run a pilot at one clinic and use the doctors’ tier changes as feedback.

Submitted to The Multimodal Triage Assistant

Your idea in one line

AarogyaRekha is a triage desk for clinics, health camps and campus health centres. It turns a patient’s words, vitals and uploaded reports into a clear four-level priority with the reasons shown, so a nurse or doctor sees who needs attention first and always signs off the decision. The application is strictly non-diagnostic.

What will be working by the finale?

A complete, deployed triage desk that a clinic can run a patient through from arrival to handover: - Register from a report: upload a lab report (PDF or photo) and the patient’s details and test results are read automatically, then checked by a person. - Priority with reasons: written rules give four levels (Immediate to Routine). They can only raise a priority, and a named nurse or doctor signs off. The rules will have been reviewed and approved by a clinician. - AI help: it picks the danger-sign questions that fit the patient, and answers can be tapped or spoken. Odia voice input will be more accurate, tested with real Odia speakers. - Every page in the patient’s language: the whole interface switches between English, Hindi and Odia, with wording checked by native speakers. - Clinic tools: a queue that shows why each patient is where they are, referrals to other facilities, a one-page handover PDF, and results over time. - Fast and live: the server sits close to users, so priorities appear in about a second, and patients get status texts in their own language. - Admin and privacy: consent checks, an audit log that shows tampering, and staff and facility management.

Tech stack and hardware you plan to use

- Frontend: React 19, Vite, TypeScript (Vercel). - Backend: Fastify 5, TypeScript (Render). - Data: Supabase (Postgres with row-level security, MFA sign-in, private storage). - Reports: pdf.js, Tesseract, Gemini vision. - AI: Google Gemini for translation and speech to text. - Output: pdf-lib and FHIR R4. - Hardware: none special. Any PC, tablet or phone with a browser, a camera for report photos and a microphone for voice. No GPU. The server runs on free-tier cloud hosting.

What have you built or tested so far?

What have you built so far? A deployed, working web app: - Register from a report: upload a lab report (PDF or photo) and the patient’s details and test results are read automatically, then checked by a person. - Priority with reasons: written rules give four levels (Immediate to Routine). They can only raise a priority, and a named nurse or doctor signs off. - AI help: it picks the danger-sign questions that fit the patient, and answers can be tapped or spoken. - Clinic tools: a queue that shows why each patient is where they are, referrals to other facilities, a one-page handover PDF, and results over time. - Admin and privacy: consent checks, an audit log that shows tampering, and staff and facility management.- - Tested: about 2,000 automated tests, including accessibility.

Pitch deck (PDF)
Pitch deck (PDF)PDF
What's working now?

Nurse workflow (desktop app): find or register a patient, intake by typing or voice (symptoms, BP, weight), triage queue, “Assess now”, yes/no questions, and doctor hand-off. It runs from a Windows installer, and the test suites pass. Triage engine: deterministic and rules-based, with 244 rules and clinical knowledge graph (set 1.1.0). It assigns tiers 1 to 4, rejects unapproved rule sets, and logs the rule ID, version and hash on every output. Embedding classification: a local embedding model matches the complaint to the rules and picks the four questions. We measured recall-at-four at about 40% on held-out complaints. Languages and voice: the UI works in English, Hindi and Odia. Translation, speech-to-text and read-aloud for questions are in place. Report reading: PDF text layer plus Tesseract for scanned pages, copied verbatim with no clinical judgment. Standards: FHIR R4 referrals and ABDM consultation documents. Onboarding: an in-app, replayable nurse tutorial in three languages.

Deck or report (PDF)
Deck or report (PDF)PDF
A photo or screenshot
A photo or screenshot: AarogyaRekha: Multimodal Rule-Based Triage Assistant

AI tools used

Claude

Claude helped us generate over 300 triage scenarios to train our models.

Links & files

Artifacts

2

Gallery

5