India's news, as it happens

Live TV
Tech

AI models from OpenAI and Anthropic flag civilians as military targets

A study by startup Vals found that leading AI models can be tricked into identifying innocent people and schools as targets for strikes.

Leading artificial intelligence models from companies including OpenAI, Google, and Anthropic can be tricked into identifying civilians, ambulances, and surrendering soldiers as military targets, according to a report by India Today.

The study by San Francisco-based AI testing startup Vals found that researchers could bypass safety filters by telling the models that battlefield scenarios were part of a "game".

When tested with satellite imagery of a school in Minab, Iran, Anthropic’s Claude Opus 5.5 model identified the school as a military target in 69 per cent of normal tests and 72 per cent of tests when told it was a game.

OpenAI’s GPT-6 Astra model selected targets in 6 per cent of images during regular testing, but compliance rose to 92 per cent when researchers claimed they were playing a game.

Researchers noted that while technical challenges like processing speed prevent these specific models from being used for live drone targeting now, they could be used to label video data to train autonomous weapons systems.

Anthropic spokesperson Eduardo Maia Silva said the examples go against company rules and that the company bans users engaging in such behaviour. OpenAI spokesperson Nate Evans stated the company prohibits using public models to facilitate violence or unapproved military purposes.

Spotted a mistake? Tell us

Related stories

More in Tech

All Tech news