SOUTH CHINA SEA — The order came down to board her. A US destroyer, operating in international waters, had been tracking a Chinese cargo vessel that its artificial-intelligence targeting system had flagged as a carrier of banned weapons. The commander requested permission to send a boarding party across.
Then, at the last minute, human analysts stepped in. The AI was wrong.
The system had misread the ship’s cargo manifests and satellite imagery, according to officials familiar with the matter, flagging the vessel through flawed pattern-matching algorithms. The discrepancy surfaced only when people — not machines — looked again at the assessment and saw what the software had missed. The boarding was halted.
No shots were fired. No one was injured.
The episode, first reported by [source], has drawn intense scrutiny to the military’s use of artificial intelligence and now sits at the center of a widening debate in Washington over how much trust the armed forces place in machines that can be wrong in ways no one predicts.
A near-miss in international waters
By nightfall, the scene had shifted from the deck of a warship in contested waters to the corridors of the Pentagon, where officials declined to comment on the specific incident but acknowledged that AI protocols are under review. The Chinese government has not yet issued a formal response. That silence may not hold.
Washington and Beijing have been locked in a trade and technological rivalry that has already reshaped supply chains, semiconductor policy and military planning across the Indo-Pacific. A US warship preparing to board a Chinese-flagged cargo vessel — even one stopped short of execution — is the kind of event that can escalate on its own momentum, regardless of intent.
The Chinese ship was en route from one port to another carrying its declared cargo when the AI flagged it. The source material does not name the vessel, the ports or the cargo type. What is known is that the system’s error was not a single glitch but a chain of them: manifests misread, imagery misinterpreted, a conclusion assembled from bad inputs and presented to a commander as a targeting recommendation.
The human backstop — and how close it came
Human oversight is the last line, and this time it held. The Pentagon’s AI ethics guidelines, updated in 2020, require human involvement for lethal decisions. Critics argue those rules are insufficient — and the near-miss in the South China Sea gives them a concrete case to point to.
It is not an isolated one. In 2023, an AI system misidentified a civilian aircraft as hostile during a NATO exercise.
That error, too, was caught before anyone fired. But the pattern is what worries lawmakers: systems trained on imperfect data, deployed in high-stakes environments, where a single mistake could trigger armed conflict between nuclear-armed powers. Lawmakers on Capitol Hill have called for hearings, citing the need for robust oversight of military AI.
The incident remains under investigation. The stakes are not theoretical.
As both the US and China continue to develop advanced autonomous systems, the risk of miscalculation grows — not because either side wants a fight, but because the machines advising commanders may not know the difference between a threat and a misread manifest until a human catches it. What to watch next: whether the Chinese government issues a formal response, whether Congress moves from calls for hearings to actual oversight legislation, and whether the Pentagon’s review of AI protocols produces changes to how targeting recommendations reach a commander’s desk.
The answers will shape how — and whether — the US Navy trusts its algorithms the next time a ship is flagged in contested waters.
