The citation is the product
In most industries a well-written AI summary is enough. In healthcare it is close to
worthless on its own, because the clinician reading it carries professional
liability for whatever they do next. An uncited statement asks them to accept that
risk on the say-so of a system they cannot inspect.
So the retrieval layer, not the generation layer, is where the effort goes. Every
asserted fact links to the note, guideline or result behind it. Verification is a
click. When the corpus does not contain an answer, the system says so rather than
producing a fluent guess.
This single design decision predicts adoption better than any other in our
experience. Tools that cite get used. Tools that assert get opened once.
Human sign-off is architecture, not a disclaimer
“A clinician reviews the output” is often written into a project description and not
into the system. The difference shows up in the details: whether a draft can be
filed without a signature, whether the interface makes editing easier than
accepting, whether the audit log records who approved what.
We build the review step as a hard gate. The system prepares, evidences and drafts.
A licensed professional decides. That is what makes the tool a documentation aid
rather than a regulated clinical decision system, which is a distinction with
substantial consequences.
Design for security review in week one
The most common avoidable failure in healthcare AI is architecting first and
discovering the residency, retention and audit requirements at security review.
Retrofitting them usually means rebuilding.
We establish them at the start: where processing may happen, what the retention terms
are, which agreements are in place, how access is scoped, what the audit log must
contain. It constrains the design, and it is far cheaper than discovering the
constraints after the build.