The black-box trap
The scenario raised by the French Ministry of the Armed Forces Red Team highlights a critical risk in modern warfare: dependence on a monolithic AI whose decision-making path is opaque. In a high-intensity combat environment, corrupted data—whether caused by software, electronic warfare or systemic bias—could lead an AI to make a catastrophic decision without any human operator being able to understand its root cause. Without traceability, command is left with an unacceptable explanatory gap, both operationally and legally.