UK AISI: Agent Evaluations Must Account for Compute Budget
The UK AI Security Institute shows that AI agent evaluations with a fixed token budget systematically underestimate frontier capabilities. Scaling the budget from one to ten million tokens raises performance by up to 25 percent on software-engineering tasks and 22 percent on math benchmarks. AISI calls on regulators to move from benchmark scores to capability curves that account for variable compute budgets.