arXiv:2605.06660: VHG — verifier-backed framework for generating hard mathematical problems

The research paper “Verifier-backed Hard Problem Generation” (Lai et al., arXiv:2605.06660), published 7 May 2026, addresses an important challenge in training large language models: how to automatically create new, valid, and sufficiently hard mathematical problems. The team from Oxford and collaborators demonstrates that an independent verifier within a self-play loop prevents the reward hacking that plagues classic setter-solver approaches.

What problem does VHG solve?

Although LLMs are increasingly capable of solving mathematical problems, they cannot themselves reliably produce valid, challenging, and original problems. This capability is essential for model advancement and autonomous scientific discovery. Classic setter-solver systems suffer from reward hacking: the setter can trivially maximise solver failure by generating poorly defined or unsolvable problems.

Three-party self-play with a verifier

VHG introduces a third component — an independent verifier — so the setter’s reward now depends on both validity (confirmed by the verifier) and difficulty (estimated by solver failure). The team tested two verifier variants: a hard symbolic verifier (strict mathematical validator) and a soft LLM-based verifier (more flexible, neural). Both variants successfully suppress invalid outputs.

Results and implications

The evaluation covered indefinite integral problems and broader mathematical reasoning. VHG “significantly outperforms all baseline methods clearly”, suggesting the approach is not domain-specific. For RL training of mathematical models, the framework opens the path to autonomous curriculum generation — the model can itself create progressively harder problems for its own training without human curation, which is a prerequisite for superhuman mathematical reasoning.