when one or more agents screw up, do large language models assign blame the way a counterfactual model predicts?

This equation is the degree of blameworthiness assigned to a single agent, taken directly from Meir Friedenberg and Joseph Y. Halpern’s Blameworthiness in Multi-Agent Settings (2019).¹ At its core is the Shapley value (introduced by Lloyd Shapley in 1951), which measures an agent’s average marginal contribution across every coalition it might have joined. Friedenberg and Halpern borrow that idea to answer a more specific question: when several autonomous agents act together and something goes wrong, how much of the blame does each one deserve? The formula above is their answer, weighting each agent’s contribution to the outcome over all the ways events could have unfolded.

What struck me when I first read it is that counterfactual reasoning sits at the heart of this. Blame, formalized this way, is not about what happened, but instead about what would have happened otherwise. An agent is blameworthy to the extent that the bad outcome depended on it. And that is precisely the kind of judgement we increasingly expect from large language models as we hand them more agency, or even delegate the apportioning of blame itself. So the question this series of posts circles around is: when an LLM assigns fault, do its judgments move in the direction and shape this kind of framework predicts?

It has been just over a year since I was first drawn to the emergent behavior that surfaces when multiple autonomous agents interact toward a shared goal. This sits at an odd intersection of philosophy, psychology, game theory, and agentic AI that feels niche from an engineering perspective today but, I’d argue, won’t stay niche for long.

I plan on writing a series of posts documenting my active learnings while exploring this and related topics. Over the next three to four posts I’ll work through this in the open: starting from the formal theory of blame, moving to the human intuitions it tries to capture, and finally testing whether LLMs actually exhibit the counterfactual reasoning the framework assumes.

Next up: unpacking what that equation is really saying.

References:

₁₎ M. Friedenberg and J. Y. Halpern. Blameworthiness in Multi-Agent Settings. en. In: Proceedings of the AAAI Conference on Artificial Intelligence 33.01 (July 2019), pp. 525-532. doi: 10.1609/aaai.v33i01.3301525. url: https://ojs.aaai.org/index.php/AAAI/article/view/3826 (visited on 04/19/2026)