The black box trap
The scenario raised by the French Ministry of the Armed Forces Red Team highlights a critical risk of modern warfare: dependence on a monolithic AI whose decision path is opaque. In a high-intensity combat environment, corrupted data caused by software, electronic warfare or systemic bias could lead an AI to make a catastrophic decision without any human operator understanding its root cause. Without traceability, command is left with an unacceptable explanatory gap, both operationally and legally.