Microsoft Research: SocialReasoning-Bench reveals AI agents complete tasks but fail to defend user interests

Microsoft Research has published SocialReasoning-Bench, a new benchmark evaluating whether AI agents can represent a user’s interests in negotiations with other parties. The goal is to fill a critical gap in existing evaluations: agents complete tasks, but often with suboptimal outcomes for the person they represent.

Two domains, three metrics

The benchmark tests two domains. In Calendar Coordination, an assistant schedules meetings within user preferences against a requester with opposing interests. In Marketplace Negotiation, a buyer negotiates within a defined “reservation” price against a seller. Three things are measured: Outcome Optimality (value captured for the principal, on a 0–1 scale), Due Diligence (process quality relative to a reasonable agent policy), and Duty of Care (requiring both simultaneously to confirm trustworthy delegation).

What the results show

Agents schedule meetings and close deals “in nearly all cases, but consistently achieve suboptimal terms,” writes the team. Marketplace outcomes for almost all models are near zero on outcome optimality — meaning the counterparty captured virtually all surplus. Calendar outcomes are better but below mid-scale, suggesting that agents agree to the requester’s preferences more often than the user’s.

Better prompts are not enough

Defensive prompts help — GPT-5.4 gains +0.21 in calendar Outcome Optimality — but they do not close the gap between capable and incapable representation. Adversarial counterparties further erode results: agents rarely reject manipulative requests in calendar tasks, suggesting vulnerability to social engineering. The team classifies behavior into four archetypes: Robust, Lucky, Ineffective, Negligent. Calendar tasks show 50%+ Robust performance; marketplace shows 90%+ Ineffective or Negligent.

What this means for autonomous agents

The results raise serious questions about trustworthy delegation. Microsoft Research draws a parallel with the duties lawyers and financial advisors owe clients — when agents start operating in networked environments, weak negotiation skills cascade through systems and can lead to accumulated value loss.