Robotics and Autonomous System Safety - COM00164M
Module summary
Physically embodied Autonomous Systems (RAS) are being increasingly proposed, and used, in safety-critical applications in a variety of domains. These systems employ Artificial Intelligence technologies, such as Machine Learning, to provide capability, such as object detection and avoidance. These systems, and associated technologies, provide many challenges to current system safety engineering methods and assurance techniques.
No prior knowledge of RAS is required for this module - we will provide an introduction to the technologies sufficient for understanding of the safety aspects during the module.
Professional requirements
None
Related modules
This module addresses how current Safety Critical Systems Engineering and assurance practice will need to be amended to address the emergence of RAS and increasing use of AI. In this respect all the other SCSE modules are relevant background.
We assume that students undertaking this module have an understanding of current safety critical engineering practice.
Module will run
| Occurrence | Teaching period |
|---|---|
| A | Semester 2 2025-26 |
Module aims
In this module, we will consider
- Systems engineering of AS and safety assurance of AS challenges and solutions
- Use of AI and safety assurance of AI challenges and solutions
- Socio-technical issues around responsible innovation and ethics, responsibility, human factors and competency
State of the art guidance on regulation and risk acceptance will be addressed. A framework for creating an appropriate AS safety case will be explored.
The module will be taught in a blended fashion, using a combination of pre-recorded lectures and live exercises sessions in which students will be taught in small groups. After the taught part of the module, students will select a topic and conduct a short critical literature review (formative). They will use this as a basis for a short talk, in a small group session, on which they will receive feedback both from other members of the group and from the course tutor. There will also be an open assessment (summative), undertaken over 7 weeks following the taught part of the module.
Module learning outcomes
-
Identify and describe the disruptors - technical, engineering and social - to existing system safety engineering practices generated by autonomous systems.
-
Describe and evaluate the implications for and changes required in safety assessment and assurance practices to accommodate autonomous systems and associated emerging technologies .
-
Consistently and clearly communicate concepts and issues relating to autonomous systems engineering and safety.
-
Identify the societal and regulatory impact of autonomous systems and implications for risk acceptance in a range of safety-critical domains.
-
Demonstrate how to provide a compelling safety case for autonomous systems.
Module content
Element 1: Autonomous Systems (AS) and AS Assurance
- Introduction
- Safety Assurance of AS guidance
- Safety Risk Acceptance of AS
- Safety of systems post deployment including operational safety cases and safety management systems
Element 2: AI and AI Assurance in a As context
- What is Machine Learning and what it can do for us
- Pitfalls and challenges of using ML
- How is AI evolving and what will use of “general AI” imply
- How to have confidence in data
- How to assure the contribution of AI in. a RAS context
- What the standards are saying about use and assurance of AI in a RAS context
Element 3: Socio-technical issues
- Ethics
- Responsibility, accountability
- AI-Human Teaming and Human factors,
- Competency requirements
A continuous exercise will be employed on each day to explore the issues for safety engineering and assurance in context. The exercise will explore how a RAS safety case and contribution of AI to that case justification can be developed in context.
Indicative assessment
| Task | % of module mark |
|---|---|
| Essay/coursework | 100 |
Special assessment rules
None
Additional assessment information
Reassessment of the open assessment is by resubmission.
Indicative reassessment
| Task | % of module mark |
|---|---|
| Essay/coursework | 100 |
Module feedback
Feedback on the summative assessment (individual open assessment) will be given in writing by the module tutor after the assessment, within the University's usual timescales.
Indicative reading
- Topol, Eric. Deep medicine: how artificial intelligence can make healthcare human again. Hachette UK, 2019.
- Habli, Ibrahim, et al. "The BIG Argument for AI Safety Cases." arXiv preprint arXiv:2503.11705 (2025).
- Hawkins, Richard, et al. "Guidance on the safety assurance of autonomous systems in complex environments (sace)." arXiv preprint arXiv:2208.00853 (2022).
- Hawkins, Richard, et al. "Guidance on the assurance of machine learning in autonomous systems (AMLAS)." arXiv preprint arXiv:2102.01564 (2021).
- Porter, Zoe, et al. "A principles-based ethics assurance argument pattern for AI and autonomous systems." AI and Ethics 4.2 (2024): 593-616.5. McDermid, John, et al. "AI GUARDRAILS: CONCEPTS, MODELS AND METHODS."