As we become more comfortable with the use of artificial intelligence (AI) and machine learning for applications such as banking systems that predict when and how much to save or TV that recommends what to watch next based, our public engagement work is asking what if this AI or autonomy was used in a safety-critical situation such as in a car or a hospital? Would we want to know more about the risks?
In 2020 we held five focus groups to talk with people about their opinions on technology and autonomy generally and in relation to two specific scenarios: autonomous cars and healthcare.
Attitudes towards technology
The groups we spoke with mostly enjoyed the benefits of technology, such as saving time or enabling them to connect with family and friends across the world. They could also see benefits for wider society, for instance more environmentally-friendly transport options or improved medical treatments.
“You know, shopping. It’s saving us having to go out, so we’ve got more time for ourselves. Even booking an appointment at the doctors now I do it online.”
They had concerns about the impact of an increasing use of technology in our lives. These were mostly around an over-reliance or an intrusion of technology, and a worry that the technology could go wrong. Some were also concerned that technology moved too fast and was more for company profit than to help society.
“Are we still the master of the technology? Or do you develop a robot or a program that does the thinking for us?”
“It’s acceptably safe” is how one member of a focus group described driving a car at the moment. “It’s clearly not safe, but it’s a means of getting around so I’m more than happy to take the risk,” they continued.
When considering autonomous vehicles, the groups divided into three main categories – supporters, potentials and rejectors.
Supporters suggested that the technology can outperform humans, would maximise leisure time and would be less stressful than driving.
“ABS does a pump 500 times a minute. We can’t pump brakes as fast as that. So there is an example. Technology is working and a car would react and see a person quicker than we can.”
Potentials want more information. They question the technology’s capabilities and want to know more about the risks and who would be liable in the event of an accident.
“It’s like when you take your driving theory test and particularly when you see the hazards, they’re not actually something happening yet but as a human you can say, right, I think that person might be thinking about crossing the road, where a computer couldn’t necessarily judge.”
Rejectors were very sceptical, had a mistrust of the technology and were not always keen to listen to more detail about it.
“I was going to say, they’ll have to be out for about ten years before I’d trust it. After that maybe.”
There was also an unease about the use of autonomous vehicles alongside existing cars, with groups worried that until the infrastructure is in place to fully support autonomous cars, the mix of old and new technology on the road would be riskier.
“I consider the most dangerous time will be the interface between a mix of autonomous vehicles and when other drivers are still on the road.”
Cars that still require a safety driver to be alert at all times were thought to be “pointless”. Some of the people were onboard with autonomous vehicles, but they wanted to be able to hand over full control to the system.
“If I got an autonomous car I’d want to be chilling in the back.”
“Was the driver expected to be paying attention to what was going on outside the car while it was in full computer mode? If it was, what’s the point of full computer mode?”
To demonstrate the use of AI in healthcare, the groups were given a scenario of a hospital ward using an autonomous infusion pump to provide medication.
The discussions highlighted an inherent trust in the healthcare sector, with wide acceptance of the use of the autonomous infusion pump, and a clear understanding of the benefits.
“They’re professionals, they know what they’re doing and, yeah, [I would] completely accept it.”
“So I think we just have to assume that they’ve got systems and controls in place within their organisation to make sure that the machine works.”
The groups did still have reservations about the use of the technology, but generally their verdict was much more positive than in the automotive scenario.
“I’d still be a little concerned… If it was one computer on one person not a problem, because that’s what it is basically now. But if it was ten people [monitored by one computer] there is still a chance that something could go wrong.”
Future focus groups
We are running the groups again in 2021 to see if and how Covid-19 might have impacted the public’s understanding and acceptance of the risks associated with autonomous systems.