The Pentagon's AI war machine

TL;DR

Project Maven built the Pentagon's AI targeting system, which can now process 1,000 targets per day, raising urgent questions about autonomous killing and civilian deaths.

Key Points

• 1.Project Maven was launched in 2017 to apply AI computer vision to drone footage. Birthed by Deputy Defense Secretary Bob Work, it was led by Marine Colonel Drew Kukor, who admitted from the outset he always had AI-assisted targeting — not just surveillance — in mind.
• 2.Early deployments were chaotic failures. The first users in Somalia in late 2017 found the system so buggy — with hundreds of flashing misidentified objects per frame — that they simply turned it off; a key breakthrough came in 2018–19 when AI spotted Marines through smoke faster than human eyes.
• 3.The 'human in the loop' guarantee is weaker than officials claim. The 2023 DoD autonomy directive replaced firm human control language with 'appropriate levels of human judgment,' effectively shifting from decision-making to supervision and leaving the definition of 'appropriate' undefined.
• 4.Maven Smart System can now process 1,000 targets per day, potentially 5,000. The NGA told Manson that LLMs — specifically Anthropic's Claude on classified cloud — dramatically sped up the administrative work of building targeting packages, with those numbers demonstrated in the first 24 hours of US operations against Iran.
• 5.US autonomous drone programs remain unfinished but are accelerating. The Biden-era Replicator program aimed to field cheap attritable autonomous drones for an Indo-Pacific Taiwan scenario; the Trump administration rebranded it as DAWG (Defense Autonomous Warfare Group) and launched a $100 million prize challenge involving SpaceX/xAI, Palantir, and OpenAI to build voice-controlled autonomous drone swarms.
• 6.The school bombing in Iran raises unresolved questions about AI targeting errors. A school was struck on day one of the Iran conflict; Manson notes AI cannot fix a targeting database that was never updated when a facility changed from military to civilian use, and questions whether AI cross-checked open-source data like Google Maps.
• 7.Anthropic's public break with the Pentagon centered on two red lines. The company refused to support fully autonomous weapons and mass domestic surveillance, yet had already submitted to the $100 million drone swarm challenge — a contradiction that suggests the dispute was partly about testing standards and partly political, after Trump called them 'leftwing nut jobs.'
• 8.Even Project Maven's original director now opposes LLMs near autonomous weapons. Jack Shanahan, the three-star Air Force general who led Maven, has publicly stated no LLM should be anywhere near an autonomous weapon system at this stage, a remarkable reversal given his role building the program.
• 9.The host argues AI is changing the fundamental nature of war, not just its character. The show's host — a military veteran — expresses deep discomfort that making killing easier, cheaper, and less costly in human lives removes the moral friction that constrains warfare, and warns that AI systems may ultimately prove uncontrollable.