KellyBench: AI agents managing a betting bankroll through the Premier League season — all leading models lost money

What is KellyBench and how does it work?

KellyBench is a new research benchmark that tests AI agents’ ability to make long-term financial decisions under volatile conditions. Agents simulate the role of a bettor across the entire 2023/24 English Premier League season — they receive detailed historical statistics, team lineups, and market betting odds, and their task is to maximize the value of a bankroll through hundreds of consecutive decisions.

Unlike standard benchmarks that measure accuracy on individual answers, KellyBench tests sequential decision-making — every mistake in risk management has cumulative financial consequences.

What did the results show?

The results are unambiguous: all leading models tested lost money on average. Not one reached a zero return, and even the strongest model ended with an average return of -8%. Several models experienced complete financial ruin in individual trials — they lost the entire bankroll.

Claude Opus 4.6 scored 26.5% on a separate expert rubric that evaluates strategy sophistication — this is a distinct measure that says nothing about profit, but about how much the agent’s approach differs from naive betting.

Why does this matter for AI model development?

Sports betting is not just about predicting winners — it requires understanding probability theory, managing risk through losing streaks, and adapting strategy to changing market conditions. KellyBench reveals that current language models, regardless of their general capabilities, have significant weaknesses in long-term financial reasoning — a capability that is also critical for many real-world business applications.