arXiv:2606.07040: Beyond Rubrics — Exploration-Driven Evaluation Skills for Reward Modeling

Researchers published a paper on June 5, 2026, on the arXiv repository (identifier arXiv:2606.07040) titled “Beyond Rubrics,” which introduces the Eval-Skill method for exploration-driven reward modeling. The goal is more efficient and consistent scoring of language model responses without repeating the costly generation of criteria for every prompt.

What is the problem with classic rubrics?

In systems where one model scores the responses of another model — a setup known as LLM-as-a-judge (a language model in the role of judge) — it is common to generate a separate set of scoring criteria, a so-called rubric, for each individual prompt. This approach is flexible but expensive: generating new criteria for each example consumes significant computing resources and introduces inconsistency.

The paper arXiv:2606.07040 starts from the idea that instead of per-prompt criteria, one can build broader, reusable scoring patterns that apply to an entire domain. The work is thus done once and used many times.

How does Eval-Skill work?

Eval-Skill synthesizes domain evaluation skills through exploration — structured knowledge about what makes a good answer in a given domain. The method works in two phases: first a workflow is generated, and then principles (principle generation) are derived from it to guide the scoring.

Surprisingly little data is used to build these skills — only 100 cases per domain. The learned skills are then embedded directly into the judge’s context, so the judge model already has relevant guidelines available when scoring, rather than reinventing them each time. This reduces compute overhead (additional computational load).

What results does the method achieve?

Effectiveness was measured on the RewardBench 2 benchmark, a standard measure for reward models. Eval-Skill delivers an improvement of +13.44% for the Qwen3-8B model and as much as +18.51% for DeepSeek-V4-Flash, both compared to baseline scoring (the basic approach without learned skills).

It is especially worth highlighting that the gains are consistent across different judge architectures, suggesting the method is not tied to one particular model but transfers across multiple systems. This is an important indicator of robustness.

Why does this matter for the community?

Reliable scoring of responses is the foundation of aligning and training language models. If the judge is cheaper to run while also being more accurate, the costs of the entire reward modeling process drop and the quality of the feedback signal that models receive during learning improves.

The authors released the code as open source, enabling other teams to try the Eval-Skill method on their own domains and judges. For researchers working on reinforcement learning and automatic evaluation, this is a practical tool with measurable gains and a low entry cost of just a hundred examples per domain.

The broader significance of the paper lies in shifting the emphasis from ad-hoc criteria toward systematically learned, transferable scoring skills. Such an approach could reduce the variability that accompanies hand-written rubrics and make automatic evaluation more predictable. As more and more phases of model training rely on automatic scores, the quality and cost of the judge itself become strategically important, and methods like Eval-Skill address both of those questions directly.