A study led by Professor of Psychology Pekka Santtila and published in Law and Human Behavior breaks new ground by exploring whether advanced artificial intelligence systems can replicate human jury decision-making. In this experiment, large language models (LLMs)—including GPT-4o, GPT-o1, and Claude 3.5 Sonnet—were asked to act as jurors in sexual assault cases to see whether they could make fair and consistent legal judgments.
The results were interesting. On one hand, LLMs were more consistent and conservative than human jurors, showing less random variation and a higher threshold for finding defendants guilty. On the other hand, they mirrored the same psychological biases that influenced human reasoning. The models, for example, judged White defendants more harshly and showed greater belief in guilt when reports were made sooner, replicating human biases to social and temporal cues.
What makes the study innovative is not only that AI systems were evaluated as decision-makers but that their internal “reasoning” was analyzed using mediation models—the same methods used to understand human bias. This revealed that the LLMs’ judgments followed recognizable cognitive pathways such as stereotype congruence and credibility attribution, suggesting that AI models applied human patterns of thought, including their distortions.
According to Professor Santtila, “This study represents a step toward systematically understanding how AI models ‘reason’ about legal evidence. By treating LLMs as simulated jurors, we can map their implicit biases and benchmark their reasoning processes which may allow human–AI collaboration in the future.”