Patient risk stratification: from score to clinical action

State the intended use in one sentence: in which population, at what point, to anticipate which event, over what time horizon, to trigger which action? The risk of readmission within seven days means something very different from ten-year cardiovascular risk.

One response per category

Pair each risk level with a possible response: a nurse call, an earlier appointment, a medication review, a test, a social referral or usual follow-up. If the team’s capacity is limited, the threshold must reflect it: flagging more patients as “high risk” than the team can call amounts to prioritizing no one.

Prediction is not explanation

A model relies on associations; it does not show that acting on a variable will change the outcome. An indicator of social deprivation may improve the prediction of hospital admission without being a clinical target. The action a score triggers should rest on a plausible mechanism and, wherever possible, on an intervention whose effectiveness has been evaluated.

A score developed in another country, specialty or period may lose performance in your setting. Compare its inclusion criteria, exclusions, point of entry and event rate with those of your patients. The TRIPOD+AI statement, which sets out what a prediction model study should report, asks authors to describe the target population and the model’s intended purpose in the care pathway, including its intended users. If those details are missing, ask for them.

Then compare the tool with current practice: clinical judgment alone, an existing rule or no stratification at all.

Discrimination measures how well a model ranks the patients who will experience the event above those who will not; calibration compares predicted probabilities with observed frequencies. A model can rank patients well while consistently overestimating their risk. For a team setting thresholds, that is a serious flaw, since a threshold is expressed as a probability.

Then comes clinical utility: at a given threshold, how many true positives, false positives, interventions and missed events? The net benefit approach described by Vickers and colleagues in The BMJ puts benefits and harms on the same scale and plots them across a range of thresholds as a decision curve. It shows whether deciding with the model would do more good than harm; choosing the threshold remains a clinical and organizational call.

Ask for performance figures with uncertainty intervals, measured on external data. Then plan a local re-evaluation after any change in population, practice or information system.

A missing measurement is not always missing at random. It may reflect a problem with access, a habit of the department or a different clinical state. Replacing every missing value with an average can hide that mechanism. TRIPOD+AI accordingly asks studies to describe how missing data were handled.

Document the share of missing data by variable and by group, and test performance in the patients with the least complete records. Completeness is only one dimension of health data quality, and it should be checked before a score runs in routine care.

Measure sensitivity, specificity, calibration and intervention rates by age, sex, disability, area of residence and other relevant groups. An average performance can hide systematic under-detection. TRIPOD+AI asks for performance estimates with confidence intervals for key subgroups, including sociodemographic ones.

Sensitive attributes should therefore not be stripped out of the evaluation by reflex: they may be needed to detect bias. Using them in the calculation, in the audit and in the decision raises three different questions, to be worked through with clinical, legal and ethics teams.

Choosing the outcome to predict is itself a clinical and social choice. A study published in Science in 2019 examined a widely used commercial algorithm that identified patients with complex health needs for additional help. It predicted health care costs rather than illness. Because access to care was unequal, less money was spent caring for Black patients: at the same risk score, they were considerably sicker than White patients.

Clinicians should see the score, its time horizon, the main inputs behind it and its limits. They must be able to override or challenge the recommendation, with a reason. The interface should tell apart a measured value, a value reported by the patient, such as patient-reported outcome measures (PROMs), and an estimate.

Define who receives the alert, within what time, at what priority and with what audit trail: the alert chain set out in our guide to remote patient monitoring. An alert that nobody handles creates a false sense of safety. Track the workload created and the share of alerts that led to no action.

That is how MyTwin for clinicians works. Built from the medical record, exams, treatments and biomarkers, then fed by real-world data, the patient digital twin is analyzed continuously and alerts the physician only when necessary. It recommends; the clinician decides whether to propose an appointment, adjust a treatment or recommend an exam.

1. Silent phase

The score is calculated without changing care. The team checks data availability, local calibration, bias and alert volume. This phase demonstrates no clinical benefit, but it exposes integration flaws before they reach a patient.

2. Supervised pilot

The tool is switched on for a limited scope, with a trained team. Decisions, overrides and incidents are documented, and patients are informed as the applicable rules require.

3. Comparative evaluation

The assisted pathway is compared with usual practice, using indicators defined in advance: time to intervention, events, workload, patient experience, equity and adverse effects. The effect of the model must not be confused with that of the extra resources added for the pilot.

4. Ongoing monitoring

The team tracks data drift, calibration, false positives and performance by subgroup. Every model update is versioned and evaluated, and there is a way to switch the tool off.

A model card sets out the purpose, developers, data, exclusions, performance, thresholds, limits and version history. The clinical protocol defines roles and response times. Where an organization develops or retrains a model on its own patients’ data, data protection rules apply from the design stage. In the European Union, France’s data protection authority, the CNIL, has published recommendations on developing AI systems under the GDPR: a purpose defined up front, data minimization and clear information for the people concerned.

For organizations operating in the EU, the AI Act scales its requirements to risk. An AI system is classed as high-risk when, among other cases, it is itself a product, or the safety component of a product, covered by legislation listed in Annex I, which includes the Medical Device Regulation, and required to undergo third-party conformity assessment. Annex III adds, among others, emergency healthcare patient triage systems. For high-risk systems, the regulation requires effective human oversight: understanding the system’s capacities and limits, staying aware of automation bias, being able to disregard or override its output, and being able to stop it. The deploying organization must assign that oversight to people with the necessary competence, training and authority, and monitor how the system operates.

Regulatory status depends on the claimed purpose and the context of use: it calls for review by the people accountable for it, and the label “decision support” does not exempt a tool from that review. The medical device side is covered in our article on AI in medical imaging.