---
title: "Getting a Clinical RAG System Approved Internally"
section: "Case studies"
canonical_url: "https://leverge.ai/case-studies/rag-clinical-documentation-citations"
topic: "clinical RAG case study"
published: "2026-06-05"
updated: "2026-07-29"
publisher: "Ailoitte Technologies Private Limited"
---

# Getting a Clinical RAG System Approved Internally

A healthcare client had a retrieval system that scored well on internal accuracy tests and that clinicians would not use. The blocker was not accuracy — it was that answers arrived as unsourced prose, and a clinician carrying liability for acting on an answer will not accept one they cannot verify. We rebuilt the retrieval layer for passage-level citation and added calibrated refusal, and the tool went from stalled to approved by the clinical review board in six weeks.

## Key takeaways

- Accuracy was never the blocker — the tool scored 89% and adoption was still near zero because answers could not be verified.
- Passage-level citations that open the exact source turned a tool clinicians distrusted into one they use daily.
- Calibrated refusal mattered as much as citation — the review board needed the system to admit gaps rather than fill them.
- Chunking on document structure rather than token count was what made passage-level citation possible at all.
- Record-level access logging and in-tenancy inference resolved the compliance questions before they were raised.

## Accuracy was not the problem

This engagement is the clearest example we have of a system failing for a reason that
does not appear in its metrics. The tool was 89% accurate. Adoption was near zero.

The gap is explained entirely by who carries the risk. A clinician who acts on an answer
owns the consequence. An unsourced statement asks them to accept that on trust, which
means the rational response is to verify it independently — and verifying it independently
is slower than not using the tool at all.

So the system's accuracy was, from the user's point of view, irrelevant. What mattered was
the cost of confirming an answer.

## Document citations are not citations

The original system attached a document name to each answer. That sounds like citation and
functionally is not: confirming a claim still meant opening a forty-page guideline and
searching it.

Passage-level citation — the specific paragraph or table row, linked so it opens at that
location — makes verification a single click. That is the entire difference, and it is what
moved daily usage from a handful of people to the whole team.

Making it possible required re-chunking. Fixed-token chunks cut across sections and
tables, so there was no stable passage to point at. Chunking on document structure, with
parent context retained, gave us addressable units that were also meaningful on their own.

## Refusal was the review board's real concern

The board's second objection was sharper than the first: a system that always answers will
answer questions the corpus does not cover.

We built refusal into the evaluation set as a scored behaviour, including questions
deliberately outside the corpus, and enforced a grounding check that rejects unsupported
claims before they reach the user. Refusal accuracy reached 91%.

That number, more than the accuracy figure, is what the board approved on. A tool that
admits its gaps is auditable. One that is fluent everywhere is not.

## The accuracy gain was incidental

Answer accuracy moved from 89% to 92% during the rebuild, and we would not present that as
the achievement. It was a side effect of better chunking — passages that arrive with their
heading intact are simply easier to reason over.

The result that mattered was approval, and approval came from verifiability rather than
from three points of accuracy.

## Frequently asked questions

### If the system was 89% accurate, why would clinicians not use it?

Because a clinician acting on an answer carries the professional liability for it, and an unsourced statement asks them to accept that risk on trust. From their perspective an 89% accurate system that cannot show its work is not 89% useful — it is a claim they have to independently verify every time, which is slower than looking it up themselves. Citations changed the economics of using it.

### What exactly changed about the citations?

The original system cited documents. The rebuilt one cites passages — the specific paragraph or table row that supports each claim, linked so it opens at that location. Document-level citation still requires the clinician to search inside a forty-page guideline. Passage-level citation makes verification a click, and that difference is what moved adoption.

### How was HIPAA handled?

Model inference ran through the client's own cloud tenancy with zero retention configured, so protected health information stayed inside infrastructure already covered by their existing agreements. Retrieval indexes, embeddings and logs lived in storage they controlled. Access was enforced at retrieval time against their identity provider, and every retrieval was logged at record level.

---

Source: https://leverge.ai/case-studies/rag-clinical-documentation-citations — Leverge