Autonomous vehicles and safety: outcomes from an ethics workshop

News | Posted on Thursday 17 December 2020

Is it appropriate for autonomous vehicles to be able to detect and make decisions based on characteristics protected by anti-discrimination legislation?

In November 2019, the Centre for Connected and Autonomous Vehicles and the Assuring Autonomy International Programme, brought together experts from across academia, industry, and Government to explore the ability and appropriateness of autonomous vehicles (AVs) being able to detect and make decisions based on characteristics protected by anti-discrimination legislation.

The group debated technical and ethical considerations of AVs being programmed to perceive and use protected characteristics as a way to ensure safe outcomes. They discussed how the public could be engaged in such debates and the role of developers in ensuring the diversity and fairness of the data they use.

The workshop concluded that the technology is not yet sophisticated enough to detect, understand, and make decisions based upon protected characteristics. The group agreed on the following primary concerns going forwards:

  • system safety and accuracy
  • transparency from industry
  • public engagement

Inputs vs outcomes

Ethics was discussed in terms of ‘input’ ethics and ‘outcome’ ethics. The former being the decisions made by developers about how the vehicle will be programmed and the process for doing this (e.g. the training data used). Output ethics focuses on the expectations we have for the behaviour of the vehicle, such as allowing all pedestrians time to step out of the road after crossing.

On the whole, the group agreed that it is unclear how to dictate input ethics and that this could even be counter-productive to the aims of equality and fairness. It was agreed that establishing the right outcomes for AV behaviour was considered the best way to approach the problem

System safety

The view of the group was that it was better to rely on accurate trajectory prediction, and other general safety mechanisms, to avoid impacting objects than identifying protected characteristics to inform decision-making.

Transparency

A reasonable degree of transparency from industry was concluded to be desirable, although the group was uncertain about how this should best be achieved. Such openness would allow the public and others to understand the reasoning for choices made by industry at the input stage and enable an understanding of what went wrong in the case of incidents and accidents.

Public engagement

Engaging with the public was considered crucial to the success of identifying the right outcomes.  Public engagement requires the following aspects: honesty, credible voices, transparency, a consultative approach, and exposure to the technology.

Ultimately, the group agreed that AVs should raise the standards of safety for everyone not target specific people for better safety outcomes and that this dialogue must be revisited as the policy area develops and vehicles are deployed without safety drivers.

CCAV AAIP ethics workshop report (PDF , 62kb)