How do humans and machines safely share control of an autonomous car?

Ensuring and assuring the safety of shared control in autonomous driving is very challenging due to the uncertainties associated with measuring the level of situational awareness of safety drivers while not in control of the vehicle, and with the mapping of such measures to control hand-back times and likelihood of success.

This project will extend, adapt and integrate the team's recent research and the latest advances from human behaviour and cognitive modelling, verification of deep neural networks, and automated controller synthesis to tackle these challenges.

An advanced semi-autonomous driving simulator will be used to deliver methods for ensuring and assuring the safety of shared control in autonomous driving. The project will make significant and generalisable contributions in the areas of:

  • shared autonomy
  • training and verification of machine learning
  • monitoring of autonomous systems by human operators

Project progress

Four new PhD students have been recruited: one at the University of York, one at Carnegie Mellon University, and two at the University of Virginia (UVA). The team has designed and conducted a human subject study on the driving simulator located at UVA involving 18 participants. This collected data about a human’s response time and physiological signals in different driving scenarios. 

Using the preliminary data from this study, the team trained a neural network model that achieves 90% accuracy of driver takeover time prediction (with the takeover time organised into several categories, e.g. fast, medium and slow). Work has also started to explore the verification of such a neural network and to synthesise a safety controller that will exploit the predictions provided by the neural network model.

The team has identified significant ongoing international efforts to ensure the safety of the control handover in autonomous/semi-autonomous driving. However, these are primarily focused on defining the scope, requirements and potential sensors/actuators of the control handover process, with little published on how these can be brought together. They have exploited the former to define the inputs, actions and requirements for their handover safety controller, and progressed with the modelling work to enable the synthesis of a correct-by-construction controller in the next stages of the project.

The team leveraged the recent open consultation document on the ‘Safe use of Automated Lane Keeping System (ALKS) on GB motorways’ to finalise the selection of the modelled (safety) driver - autonomous car aspects, so the Safe-SCAD controller will provide a driver availability recognition and improvement system compatible with the ALKS solution. As such, they have defined requirements for the controller to reduce the frequency of ‘minimum risk manoeuvres’ (i.e. unplanned stops due to safety concerns) by issuing auditory/visual/haptic warnings to improve driver availability (when required and subject to not overwhelming the driver) and/or by suggesting speed adjustments to the car (e.g. ALKS) controller.

The team has developed a deep learning framework, named DeepTake, for driver takeover behaviour prediction. This work has recently been accepted by the top conference CHI 2021. In addition, they have recently made progress in the following areas:

  1. Attribution and trust in neural networks
  2. Clustering for robustness analysis
  3. Improving robustness based on Marabou counterexamples
  4. Adversarial generation and training using off-the-shelf methods

The team has also completed Phase I of the driver/car modelling and controller synthesis tasks.

Papers and presentations

Body of Knowledge guidance 

Safe-SCAD guidance is available in the Body of Knowledge:

Project team

  • University of Virginia
  • Carnegie Mellon University
  • University of York