Scarlett Johansson’s AI row has echoes of Silicon Valley’s bad old days

So far, the AI giants have largely played ball on paper. At the world’s first AI Safety Summit six months ago, a bunch of tech bosses signed a voluntary pledge to create responsible, safe products that would maximise the benefits of AI technology and minimise its risks.

Those risks, originally identified by the event organisers, were the proper stuff of nightmares. When I asked back then about the more down-to-earth threats to people posed by AI tools discriminating against them, or replacing them in their jobs, I was quite firmly told that this gathering was dedicated to discussing the absolute worst-case scenarios only – this was Terminator, Doomsday, AI-goes-rogue-and-destroys-humanity territory.

Six months later, when the summit reconvened, the word “safety” had been removed entirely from the conference title.

Last week, a draft UK government report from a group of 30 independent experts concluded that there was “no evidence yet”, external that AI could generate a biological weapon or carry out a sophisticated cyber attack. The plausibility of humans losing control of AI was “highly contentious”, it said.

Some people in the field have been saying for quite a while that the more immediate threat from AI tools was that they will replace jobs or cannot recognise skin colours. AI ethics expert Dr Rumman Chowdhury says these are “the real problems”.

The AI Safety Institute declined to say whether it had safety-tested any of the new AI products that have been launched in recent days; notably OpenAI’s GPT-4o, and Google’s Project Astra, both of which are among the most powerful and advanced generative AI systems available to the public that I have seen so far. In the meantime, Microsoft has unveiled a new laptop containing AI hardware – the start of AI tools being physically embedded in our devices.

The independent report also states that there is currently no reliable way of understanding exactly why AI tools generate the output that they do – even among developers – and that the established safety testing practice of Red Teaming, in which evaluators deliberately try to get an AI tool to misbehave, has no best-practice guidelines.

At the follow-up summit running this week, hosted jointly by the UK and South Korea in Seoul, the firms have committed to shelving a product if it doesn’t meet certain safety thresholds – but these will not be set until the next gathering in 2025.

Some fear that all these commitments and pledges don’t go far enough.

“Volunteer agreements essentially are just a means of firms marking their own homework,” says Andrew Straight, associate director of the Ada Lovelace Institute, an independent research organisation. “It’s essentially no replacement for legally binding and enforceable rules which are required to incentivise responsible development of these technologies.”

OpenAI has just published its own 10-point safety process which it says it is committed to – but one of its senior safety-focused engineers recently resigned, writing on X that his department had been “sailing against the wind” internally.

“Over the past years, safety culture and processes have taken a backseat to shiny products,” posted Jan Leike.

There are, of course, other teams at OpenAI who continue to focus on safety and security.

Currently though, there’s no official, independent oversight of what any of them are actually doing.

“We have no guarantee that these companies are sticking to their pledges,” says Professor Dame Wendy Hall, one of the UK’s leading computer scientists.

“How do we hold them to account on what they’re saying, like we do with drugs companies or in other sectors where there is high risk?”

We also may find that these powerful tech leaders grow less amenable once push comes to shove and the voluntary agreements become a bit more enforceable.

When the UK government said it wanted the power to pause the rollout of security features from big tech companies if there was the potential for them to compromise national security, Apple threatened to remove services from Britain, describing it as an “unprecedented overreach” by lawmakers.

The legislation went through and so far, Apple is still here.