.png)
Assuring the safety of an autonomous system means understanding what must be done to ensure people are confident that an autonomous system, such as a self-driving car or drone, will operate safely.
Throughout my career at the University of York, I have been interested in how we make systems safe, particularly systems that use software to control them. As more autonomy is used for systems, I have started to look at what we have to do differently to make sure we can demonstrate systems are safe when they have increased autonomy and become less reliant on a human operator.
As with any new technology, autonomy brings unique challenges which mean that established approaches to safety assurance aren’t necessarily sufficient. For example, machine learning, which is widely used in autonomous systems for things like identifying objects in the environment, is very different from traditional software, so we need new techniques to assure this. It is even more challenging to demonstrate the safety of autonomous systems since they are often used in very complicated environments.
Making systems safe is not just a technical and engineering task; developing and operating complex systems safely requires us to think about much broader socio-technical issues such as how humans interact with autonomous systems, what the ethical implications are of autonomous operation, how the law might need to change in response, and so on.
To fully understand the issues therefore requires us to involve experts from different fields such as psychologists, lawyers, and philosophers. This means we all play a part in assuring the safety of the system. It helps ensure that our research and the guidance that we develop is relevant to industry, and helps solve the problems they face.
One of the really important aspects of the work of the AAIP is that we do all of our research in very close collaboration with industry and regulators. This means that what we produce is not theoretical; we produce research outputs that reflect real practical needs and challenges.
Centre for Assuring Autonomy