How can we develop autonomous vehicles that can explain the decisions they take?
Understanding the decisions taken by an autonomous machine is key to building public trust in robotics and autonomous systems (RAS). This is addressing this issue of explainability through designing, developing, and demonstrating fundamental AI technologies in real-world applications.
The aim of the project is to build robots, or autonomous vehicles, that can:
- sense and fully understand their environment
- assess their own capabilities
- provide causal explanations for their own decisions
In on-road and off-road driving scenarios, the project team is studying the requirements of explanations for key stakeholders (users, system developers, regulators). These requirements are informing the development of the algorithms that will generate the causal explanations.
The work is focused on scenarios in which the performance of traditional sensors (e.g cameras) significantly degrades or completely fails (e.g. in harsh weather conditions). The project is developing methods that can assess the performance of perception systems and adapt to environmental changes by switching to another sensor model or a different sensor modality. For the latter, alternative sensing devices (incl. radar and acoustic sensors) are being investigated in order to guarantee robust perception in situations when traditional sensors fail.
The team has extended their methods for interpreting and representing observations of the environment in human-understandable terms. This is to enable end-users to comprehend algorithmic decisions. To this end, they have improved the way that traditional sensors (e.g. cameras, lasers) are used in complex and rare traffic situations. However, due to limitations of these sensors under certain environmental conditions (e.g. bad weather, poor illumination), the team has also investigated non-traditional sensors (e.g. radars, microphones) for understanding the drivable surface and for assessing the ability to measure the vehicle's ego-motion.
The team takes a more nuanced approach than solely relying on one type of sensor. Towards this, they have developed a system which facilitates introspection (knowing when we don’t know) using radar. They released a radar dataset advocating for the increased exploitation of these unusual sensors, and will use the lessons learned here in the collection and release of a SAX dataset capturing complex, challenging driving scenarios outside of already well-investigated urban environments.
This data collection has covered urban, suburban, and motorway driving conditions, as well as off-road driving in the Scottish Highlands. The data collected is focused on the requirement for very robust sensing in challenging driving conditions and will be representative of a wide swathe of UK driving scenarios.
The team has also been introducing audio as information to use in conjunction with other sensors. In London they recorded detailed in-cabin audio narration from a professional driving instructor, which will allow future AI systems to better explain actions on the road.
Presentations and papers
Sense-Assess-eXplain (SAX): Building Trust in Autonomous Vehicles in Challenging Real-World Driving Scenarios”, in Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Workshop on Ensuring and Validating Safety for Automated Vehicles (EVSAV), 2020
- Broome, M., Gadd, M., De Martini, D., and Newman, P. “On the Road: Route Proposal from Radar Self-Supervised by Fuzzy LiDAR Traversability,” AI. Multidisciplinary Digital Publishing Institute (MDPI), November 2020
- Williams, D., Gadd, M., De Martini, D., and Newman, P. “Fool me once: robust selective segmentation via out-of-distribution detection with contrastive learning,” in IEEE International Conference on Robotics and Automation (ICRA), 2021. Preprint available.
- Omeiza, D., Kollnig, K., Web, H., Jirotka, M., and Kunze, L. "Why not explain? Effects of explanations on human perceptions of autonomous driving: a user study". 2021 IEEE International Conference on Advanced Robotics and its Social Impacts (ARSO), 2021
- Omeiza, D., Web, H., Jirotka, M., and Kunze, L. "Towards accountability: providing intelligible explanations in autonomous driving". 2021 IEEE Intelligent Vehicles Symposium (IV), 2021
- De Martini, D., Gadd, M., and Newman, P. "kRadar++: Coarse-to-Fine FMCW Scanning Radar Localisation", in Sensors, Special Issue on Sensing Applications in Robotics, vol. 20, no. 21, p. 6002, 2020. Multidisciplinary Digital Publishing Institute (MDPI), October 2020
- Gadd, M., De Martini, D., Marchegiani, L., Newman, P., Kunze, L. “Sense-Assess-eXplain (SAX): Building Trust in Autonomous Vehicles in Challenging Real-World Driving Scenarios,” in Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Workshop on Ensuring and Validating Safety for Automated Vehicles (EVSAV), (Las Vegas, NV, USA), October 2020
- Williams, D., De Martini, D., Gadd, M., Marchegiani, L., and Newman, P. “Keep off the Grass: Permissible Driving Routes from Radar with Weak Audio Supervision,” in IEEE Intelligent Transportation SystemsConference (ITSC), (Rhodes, Greece), September 2020
- Kaul, P., De Martini, D., Gadd, M., Newman, P. “RSS-Net: Weakly-Supervised Multi-Class Semantic Segmentation with FMCW Radar,” in Proceedings of the IEEE Intelligent Vehicles Symposium (IV), June 2020
- Gadd, M., De Martini, D., Newman, P. “Look Around You: Sequence-based Radar Place Recognition with Learned Rotational Invariance” in IEEE/ION Position, Location and Navigation Symposium (PLANS), April 2020
- Barnes, D., Gadd, M., Murcutt, P., Newman, P., and Posner, I. The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset, in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2020.
- Saftescu, S., Gadd, M., De Martini, D., Barnes, D., and Newman, P. Kidnapped Radar: Topological Radar Localisation using Rotationally-Invariant Metric Learning, in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2020.
- Tang, TY., De Martini, D., Barnes, D., and Newman, P. RSL-Net: Localising in Satellite Images From a Radar on the Ground, in IEEE Robotics and Automation Letters (RA-L) 2020
- Dr Lars Kunze co-organised the 2nd Workshop on 3D-Deep Learning for Automated Driving at the IEEE Intelligent Vehicles Symposium (IV’2020) - Las Vegas, NV, United States, June 2020.
- Dr Lars Kunze is a co-author of Marina Jirotka’s talk “Towards Responsible Innovation in Autonomous Vehicles” at the Driverless Futures? research workshop on “The politics of autonomous vehicles”, 16-17 December 2019, University College London.
- Oxford Robotics Institute (ORI) at the University of Oxford
The team undertaking off-road trials in December 2019 with a specially adapted ORI Land Rover, equipped with vision and Lidar sensors to gather data to test algorithms for localisation and perception for autonomy in challenging environments. From left to right: Matt Towlson (Hardware Engineer, Principal Research Technician), Matthew Gadd (Postdoctoral Research Assistant, Researcher Co-Investigator), Lars Kunze (Departmental Lecturer, Co-Investigator), Daniele De Martini (Postdoctoral Research Assistant, Researcher Co-Investigator), and Oliver Bartlett (Trials Manager, Trials and Communications Organiser).