AI in medical imaging: five conditions for trust

The specifications should state the exam, the condition, the population, the user, the point in the care pathway and the output produced. Does the tool draw attention to an area? Does it provide a measurement? Does it sort exams? Does it generate text?

Then describe what the professional does. Do they confirm every result? Can they dismiss the suggestion? How does the interface show uncertainty? Who acts when the AI and the human reading disagree?

This clarification determines the potential regulatory status. In the European Union, software intended for a medical purpose may fall under the Medical Device Regulation. The EU Artificial Intelligence Act adds obligations depending on the category and level of risk.

Ask where, when and how the development and validation data were obtained. Was performance measured on external data from other centers? Do the scanner vendors, protocols, prevalence levels and demographics resemble your own setting?

A retrospective validation enriched with positive cases can be useful to test a technical capability, but it does not necessarily reflect the real frequency of abnormalities or the interruptions of a clinical department. Predictive values change with prevalence.

Require results with confidence intervals and, where relevant, subgroup analyses. A subgroup that is too small cannot establish equivalence: the absence of a statistical difference is not proof of the absence of bias.

An accurate tool can still make work harder if it slows down the workstation, adds clicks or displays an ambiguous signal. Test it with real users inside the PACS and RIS, across different shifts and exam types.

Observe:

• the time added or saved;
• how well symbols and scores are understood;
• how often work is interrupted;
• automation bias, meaning the tendency to follow the suggestion without sufficient checking;
• the opposite risk, when repeated alerts end up being ignored;
• how the result is documented in the report.

Training should cover known errors and out-of-scope cases, not just successful examples.

“Human in the loop” only means something if that human has the time, the information and the authority required. Define who checks, at what point and following which procedure. The professional must be able to access the original image and understand the nature of the output.

The World Health Organization (WHO) emphasizes autonomy, transparency, accountability, inclusiveness and sustainability in the use of AI for health. These principles translate into operations: appropriate information, logging, a way to contest results, incident reporting and governance of updates.

That is how MyTwin for clinicians integrates imaging. Alongside the medical record, test results and treatments, imaging is part of the data MyTwin uses to build the patient’s digital twin. Its imaging analyses inform the radiologist, who keeps the reading and the diagnosis. Preoperative 3D models, produced through specialized partner applications, follow the same logic: they help surgeons prepare the procedure without deciding for them.

Dr. Rahal’s story shows what is at stake. As a digestive surgeon, she discovered an unexpected vascular variation in the middle of an operation, which reinforced her conviction that 3D preparation based on imaging matters. The surgical act itself remains hers.

Practices, equipment and populations change. A software update can also alter how the tool behaves. The monitoring plan should include:

• product version and date of changes;
• availability and response time;
• rate of uninterpretable results;
• false positives, false negatives and disagreements reviewed on a sample;
• incidents and near misses;
• performance by site, device and relevant group;
• impact on turnaround times and workload;
• suspension criteria.

The TRIPOD+AI statement and the PROBAST+AI tool provide benchmarks for reporting and assessing prediction models. They do not replace regulatory assessment, but they help identify missing information and risks of bias.

1. Exactly which decision or task does the product support?
2. Which version was evaluated?
3. On which population and which equipment?
4. Is there external and prospective validation?
5. Which cases are excluded?
6. How is uncertainty displayed?
7. Who remains responsible for the reading and the action taken?
8. How are errors and incidents reported?
9. What happens after an update?
10. Which clinical or organizational outcomes will be measured locally?