» Inhaltsübersicht
AI Policy Corner
5 Min.

Governing AI in Healthcare: what´s the value of checklists?

In clinical environments where every decision can affect a patient’s life, governing the use of artificial intelligenceArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. (AI)Artificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. is not just a regulatory matter — it’s a matter of clinical trust, patient safety, and ethical responsibility. With the EU Artificial IntelligenceArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. Act in force [1], tools like the checklist-based governance methodology proposed by Bignami et al [2] are due to become commonplace in hospital boards and risk committees. Checklists are attractive governance tools: they translate regulatory principles into discrete questions, support documentation, and help multidisciplinary teams track readiness. Yet in healthcare — where data is messy, patient populations vary, and clinical workflows are complex — we also see situations where traditional checklists struggle to capture what matters most. As we witness the rapid deployment of AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. in all realms of society, it’s worth asking: Do checklists help us govern AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. well — or do they risk creating a false sense of complacency?

AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. errors, bias and liability challenges

Risks of AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. are well documented. For example, a high-profile research paper introducing Google’s Med-Gemini model in radiology mistakenly referred to a non-existent brain structure, highlighting how even well-resourced systems can generate hallucinations that lapse past clinical reviewers unnoticed. Real-world error modes like this underscore the need for dynamic model validation, real-time monitoring, and robust human oversight — far beyond ticking a box on a checklist [3].

Also well-established is that AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. models trained on non-representative data risk encoding and amplifying healthcare disparities. Survey studies in AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. healthcare research show that predictive tools can perform unequally across demographic groups, raising concerns about fairness in diagnostics and treatment recommendations. As we’ve learned from bias research in clinical AI,Artificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. diverse data and fairness auditing are essential governance components [4].

Finally, experts have warned that as clinicians increasingly rely on algorithmic recommendations, establishing accountability in cases of harm becomes legally complex. Patients and providers alike may struggle to identify whether clinicians, developers, or organisations should bear responsibility — a challenge that checklists alone cannot resolve without clear institutional escalation protocols and ethical frameworks [5]. These examples remind us that clinical AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. governance must be both rigorous and context-aware — not only procedural.

A proposed checklist

To address the challenges above, Bignami et al designed a checklist as an operational tool to help healthcare organisations systematically assess and govern the adoption and use of AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. systems — particularly in clinical environments such as anaesthesia and intensive care units (ICUs) [2]. Its purpose is to support responsible, safe, ethical and legally compliant AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. integration, in line with regulatory obligations under the EU Artificial IntelligenceArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. Act. Importantly, it is question-based rather than prescriptive, helping professionals ask the right questions rather than simply tick boxes.

The checklist is composed of two main domains: “Clinical and Technical Validation” and “Governance and Compliance”. Each domain focuses on different aspects of trustworthy AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. adoption

The clinical and technical validation domain evaluates the performance, robustness, safety and applicability of AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. systems in the specific clinical context where they will be used. Key elements include validating real-world clinical performance, regulatory and GDPRThe General Data Protection Regulation (GDPR) is the European Union’s legal framework governing how organizations collect, store, and process personal data. Its aim is to strengthen individuals’ privacy rights by requiring explicit consent before data collection and ensuring that consent can be withdrawn at any time. The GDPR came into… compliance, representative data, robust post-deployment monitoring, and adequate training of clinicians and staff to ensure safe use and oversight.

The governance and compliance domain focuses on organisational structures, processes, and legal responsibilities that ensure AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. is used ethically, traceably, and in compliance with law and internal policy. Key elements include risk-based legal compliance, clear accountability for AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. oversight and clinical use, robust traceability and auditability, defined human-in-the-loop procedures, role-specific staff training, and regular auditing and updates to keep governance aligned with evidence and regulation.

Do checklists bring value to healthcare and clinical practice?

Checklists may add value in healthcare AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. governance by bringing structure to multidisciplinary review, helping clinicians, data scientists, IT, legal, and quality teams align around shared validation steps, documentation standards, and risk assessments. They also support audit trails and compliance by providing a clear, traceable account of testing, validation, monitoring, responsibilities, and corrective actions when reviewed by regulators or internal auditors. In addition, checklists encourage lifecycle thinking, reinforcing that AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. governance is a continuous process spanning vendor selection, deployment, and ongoing performance monitoring and drift detection, rather than a one-off approval step [6].

Despite their utility, checklists often struggle with the qualitative dimensions of clinical AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. governance:

  • Nuance and clinical context: A checklist item such as “Has model performance been validated?” does not capture how performance varies by sub-population, clinical setting, or data quality — issues that directly affect patient outcomes.
  • Trust and clinician acceptance: Research shows clinicians don’t uniformly “accept or reject” AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. recommendations; instead, they negotiate trust in what the AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. suggests, adjusting advice based on clinical judgment. This kind of adaptation is not easily captured in checklist responses [7].
  • Sociotechnical factors: Patient perception, empathy, workflow integration, and clinician training are all vital for AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. to function safely in practice. These variables are hard to codify in binary governance items but are essential in ethical, human-centred AI.Artificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video.

Toward Richer Governance Practices

AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. governance in healthcare is positioned at the intersection of technology, medicine, and human judgement. Checklists help us organise conversation and document evidence. Checklists provide a structural baseline, but meaningful and safe AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. adoption depends on governance that is grounded in real clinical practice and day-to-day use. Only when paired with active, context-sensitive governance cultures — where clinicians, data scientists, and ethicists continually reflect on outcomes — can we truly steward AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. in ways that enhance safety, equity, and trust [8]. Healthcare organisations thus need to consider:

  • Multidisciplinary AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. governance boards with clinical, ethical, legal, and patient perspectives.
  • Scenario-based validation that asks how AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. performs under different patient conditions or edge cases.
  • Continuous learning loops where post-market monitoring informs updates to governance tools.
  • Ethics and equity audits that examine how AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. impacts different patient populations.

This means moving beyond “checklists as compliance” toward checklists as catalysts for organisational learning and ethical alignment.

In healthcare, the stakes are high. A box checked does not always mean a patient safe. But a culture that scrutinises AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. at every turn just might.

References
  1. Artificial IntelligenceArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. Act. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligenceArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial IntelligenceArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. Act), 13 June 2024, http://data.europa.eu/eli/reg/2024/1689/oj
  2. Bignami E, Darhour LJ, Franco G, Guarnieri M, Bellini V. AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. policy in healthcare: a checklist-based methodology for structured implementation. J Anesth Analg Crit Care. 2025 Sep 25;5(1):56. doi: 10.1186/s44158-025-00278-3.
  3. Chinta SV, Wang Z, Palikhe A. AI-driven healthcare: a review on ensuring fairness and mitigating bias.  arXiv:2407.19655, https://arxiv.org/abs/2407.19655
  4. The Verge, 4.8.2025. Google´s healthcare AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. made up a body part. https://www.theverge.com/health/718049/google-med-gemini-basilar-ganglia-paper-typo-hallucination
  5. Angus DC, Khera R, Lieu T, et al. AI,Artificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. Health, and Health Care Today and Tomorrow: The JAMA Summit Report on Artificial Intelligence.Artificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. JAMA. 2025;334(18):1650–1664. doi:10.1001/jama.2025.18490
  6. Wells BJ, Nguyen HM, McWilliams A et al. A practical framework for appropriate implementation and review of artificial intelligenceArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. (FAIR-AI) in healthcare. npj Digit. Med. 8, 514 (2025). https://doi.org/10.1038/s41746-025-01900-y
  7. Sivaraman V, Bokuwski LA, Levin J et al. Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care.
    arXiv:2302.00096
    , https://arxiv.org/abs/2302.00096
  8. Kim JY, Hasan A, Kueper J et al Establishing organizational AIArtificial Intelligence (AI) implies the attempt to transfer human learning and thinking to computers. Computer systems are intended to solve problems—for which a human would require intelligence—using large datasets. In the field of radiology, for example, artificial intelligence can assist physicians in diagnosing findings and detecting abnormali… Explained in the video. governance in healthcare: a case study in Canada. NPJ Digit Med. 2025 Aug 15;8(1):522. doi: 10.1038/s41746-025-01909-3  

Contact
Chair of Management and
Innovation in Healthcare
Witten/Herdecke University
Alfred-Herrhausen-Straße 50
58448 Witten, Germany