ArXiv: Mathematical Proof of the Impossibility of Full Accountability in Human-AI Collectives

A theoretical framework for regulators

While public debate on AI accountability is most often conducted at the level of intuition (“someone must be to blame”), author Tibebu approaches the problem formally. The paper The Accountability Horizon, published on ArXiv on April 10, proves an impossibility theorem for systems that combine humans and AI agents.

Four properties of accountability

Tibebu defines four properties we expect from any “accountable” sociotechnical system:

1. Attribution — For every action there must be an identifiable actor
2. Intelligibility — The reasons for a decision must be comprehensible to an oversight body
3. Sanctionability — There must be a mechanism for punishing incorrect decisions
4. Correctability — The system must be able to learn from mistakes and not repeat them

The main theorem

Above a certain threshold of AI agent autonomy (the author calls it the “accountability horizon”), all four properties cannot simultaneously hold. In other words, the more autonomy we grant AI systems, the less meaningfully we can speak of accountability.

Concrete examples of tensions:

• Attribution weakens when multiple agents coordinate (see the ACIArena paper from the same day)
• Intelligibility weakens when agents use latent representations that do not map to human concepts
• Sanctionability weakens when decisions involve distributed computation
• Correctability weakens when RLHF updates have unpredictable side effects

Implications for the EU AI Act and other regulations

The paper has practical consequences for regulations that try to “distribute” accountability between developers, deployers, and users of AI systems. Tibebu suggests that such attempts cannot succeed if autonomy crosses a certain threshold — and that regulators should set hard upper bounds on the level of autonomy instead of trying to distribute accountability post-hoc.