arXiv:2605.31584: LongTraceRL Learns Long-Context Reasoning from Search-Agent Traces

The paper arXiv:2605.31584 introduces LongTraceRL, a reinforcement learning (RL) method that addresses the challenge of long-context reasoning — situations in which large language models struggle to find and connect key information amid a large volume of distracting content.

What is long-context reasoning?

Long-context reasoning means the model must reach a conclusion based on a very long input, for example multiple documents at once. The problem is that relevant data is often “diluted” among many irrelevant passages. LongTraceRL uses RL with verifiable rewards (RLVR), going beyond approaches limited by weak distractors and sparse feedback signals.

How is the training data created?

The data is built from search-agent traces (trajectories) with two levels of distractors. The first consists of documents the agent opened but did not cite — these are highly confusing because they looked relevant. The second consists of documents that appeared in search results but the agent never opened, which makes them low-confusability. This tiered approach outperforms random sampling or construction from a single search.

What are rubric rewards?

Rubric rewards use gold entities along each reasoning chain as fine-grained, entity-level process supervision. This makes it possible to guide intermediate steps, not just to check the final answer. The system applies a self-positive reward strategy: reasoning quality is rewarded only for correct answers, which prevents “reward hacking.”

What are the results?

The testing covers five long-context benchmarks and models ranging in size from 4 to 30 billion parameters. LongTraceRL shows consistent improvements over strong baseline methods, encouraging thorough, evidence-grounded reasoning. The materials are available in the authors’ GitHub repository.