How do humans and machines safely share control of an autonomous car?

The Safe-SCAD project developed a proof-of-concept driver attentiveness management system to support safe shared control of autonomous vehicles. This system comprises a deep neural network (DNN) responsible for predicting the driver control-takeover behaviour, methods for verifying this DNN, and a discrete-event controller that issues optical, acoustic and/or haptic driver alerts based on the predictions of the DNN and the results of its online verification.

Contact us

Assuring Autonomy International Programme

assuring-autonomy@york.ac.uk
+44 (0)1904 325345
Institute for Safe Autonomy, University of York, Deramore Lane, York YO10 5GH

An illustration of the inside of an autonomous vehicle

Project report

The full project report describes the development and integration of the SafeSCAD components and the testing of the SafeSCAD solution.

Final project report

The challenge

Drivers find it very challenging to remain attentive when in charge of vehicles with automated driving systems. How can we measure and map a safety driver’s level of situational awareness while they’re not in control of the vehicle, in order to safely hand-back control to the human?

The research

The team developed a novel DNN-based framework that predicts driver takeover behaviour (e.g. takeover reaction time) to ensure that a driver is able to safely take over the control when engaged in non-driving tasks. They investigated formal analysis techniques for neural networks whose results can feed directly into the system-level design of autonomous systems, and applied them to the DNN developed by the project, to quantify its aleatory uncertainty.

An approach that takes this uncertainty into account in the implementation of conventional software controllers for autonomous systems was then developed. This approach to controller synthesis provides formal guarantees that the autonomous system complies with key safety and performance requirements despite using a deep-learning component to perceive its environment. The DNN, the formal analysis techniques, and the controller synthesis approach devised by the project were used to prototype an autonomous system for maintaining driver attentiveness in shared-control autonomous driving through issuing optical, acoustic and haptic driver alerts as required when driver attention is unacceptably low.

The results

You can read about all of the results from the project in the final project report. The main results are that the project:

  • created a proof-of-concept solution that can be integrated into the future design of shared-control automated vehicles, to facilitate human-system interaction, making it safer and more natural and efficient.
  • outlined formal analysis techniques for neural networks that can be used to quantify the aleatory uncertainty of multiclass DNN classifiers within the operational design domain of their autonomous systems.
  • demonstrated the use of a combination of design-time and online verification of neural networks to synthesise conventional controllers guaranteed to satisfy key safety, dependability and performance requirements of an autonomous system, and to be Pareto optimal with respect to a set of optimisation criteria.
A diagram showing the components of the Safe-SCAD driver attentiveness management system
The Safe-SCAD driver-attentiveness management system

How can the verification of neural networks and of traditional software components be combined to provide assurance evidence for systems comprising both types of components?

Find out more from project PI, Professor Radu Calinescu

Project partners

Contact us

Assuring Autonomy International Programme

assuring-autonomy@york.ac.uk
+44 (0)1904 325345
Institute for Safe Autonomy, University of York, Deramore Lane, York YO10 5GH