Physiological grounding makes clinical language-model recommendations safer in controlled tests

What they did

The authors built a training-time framework around a 7B clinical language model and a heterogeneous graph neural network (HGNN) operating over an 847,000-node biomedical knowledge graph. Candidate responses were evaluated for physiological stability, plausible multi-step biomedical relationships and drug interactions; those rankings then guided iterative, on-policy odds-ratio preference optimization (ORPO).

They evaluated the model on the Clinical Safety Benchmark, comprising 2,500 scenarios designed to expose physiological constraint violations in generated clinical reasoning. Additional tests used blinded physician review, an independent rule-based safety scorer, synthetic electronic-health-record noise, ablations and calibration against 200 clinician labels.

Key findings

The method improved the benchmark's Clinical Safety Score (CSS) from 69.5% with standard ORPO to 90.8%, a gain of 21.3 percentage points.
On the blinded physician-reviewed subset, the reported harm rate fell from 14.1% to 5.1%; drug-interaction detection rose from 72.8% to 91.6%.
An HGNN-independent rule engine produced an 86.4% safety score, 21.2 points above ORPO, and closely agreed with CSS (r = 0.97). PhysioScore calibration against 200 clinician labels yielded an expected calibration error of 0.038 and κ = 0.91.
The model retained 84.2% CSS under synthetic EHR-style noise. Removing HGNN scoring reduced CSS by 16.2 points, while removing iterative training cost 11.5 points; the full method also exceeded five-shot GPT-4 on the reported safety metrics and beat an inference-time self-correction baseline by 11.4 CSS points.

Why it matters

The results suggest that clinical-model alignment can benefit from explicit physiological structure rather than relying only on textual preferences or post-generation self-correction. If validated beyond the benchmark, this approach could provide a more inspectable way to discourage recommendations that sound medically plausible but violate homeostatic or drug-interaction constraints.

Caveats

The evidence comes from a controlled benchmark and synthetic EHR-style corruption, not prospective use or external validation on real clinical records. Several safety measures depend on the paper's own physiological scoring framework, although physician review and an independent rule engine provide partial corroboration; comparisons with GPT-4 also do not establish equivalent clinical capability or deployment safety.