Safety Assurance of Autonomous Intravenous Medication Management Systems (SAM)

The SAM project focused on how autonomy could be safely introduced to the use of intravenous (IV) medication preparation, administration and management and the types of safety assurance needed by patients, healthcare professionals, and regulators.

The challenge

IV medication preparation, administration and management are complex, and many errors are routinely made. Highly automated technology, and technology that takes decisions independently of healthcare professionals, could help to provide personalised treatments while reducing errors.

Considering IV medication management within an ICU setting, this project focused on three areas:

• What kinds of safety assurance do patients, healthcare professionals and regulators require?
• How well are current safety assurance methods able to evaluate highly automated intravenous infusion technology?
• Can these methods form the basis for assurance strategies that are able to satisfy the assurance needs of the different stakeholders?

The research

The focus of the study was the clinical system rather than the technology as such, and it looked at safety assurance challenges at the intersection of engineering and human factors. 

Four use scenarios at different levels of automation and autonomy were identified. These were used to explore stakeholder perceptions about risk, handover, and the investigation of adverse events involving autonomous infusion devices.

Three complementary analysis approaches (Functional Resonance Analysis Method, Systematic Human Error Reduction and Prediction Approach, NHS Digital SMART approach) were used to explore the safety issues around the use of autonomous infusion technology in intensive care.

The results

The project has made six recommendations aimed at technology developers, healthcare providers and regulators:

1. Developers should consider the patient experience and the impact on the patient-clinician relationship.
2. Adoption of RAS should be accompanied by training to enable clinicians to maintain core clinical skills, and to educate clinicians about limitations of AI. 
3. Healthcare providers should consider the introduction of new AI specialist roles.
4. Hazard analysis should be performed at the level of the clinical pathway or clinical system.
5. Developers should design for situation awareness, handover between clinicians and RAS, and human performance variability.
6. Regulators should promote existing best practices and establish an integrated safety governance framework for AI regulation in healthcare.

The team established collaborations and partnerships with a number of bodies, including the Chartered Institute of Ergonomics and Human Factors (CIEHF), NHSX and BSI, where these recommendations are being considered further. This includes the publication of Human Factor in Healthcare AI, a white paper published by CIEHF, representing the outcomes of work by Dr Mark Sujan and colleagues as part of the HF/AI demonstrator project.

Project partners

• Human Reliability Associates Ltd
• NHS Digital
• University Hospitals of Derby and Burton NHS Foundation Trust