arXiv:2606.05555: Representation learning enables scalable multitask RL (MR.Q)

A scientific paper published on arXiv (identifier 2606.05555) on June 4, 2026 advances the thesis that the main driver of scalable multitask reinforcement learning (RL) is precisely representation learning, not model-based control. At the center of the paper is MR.Q, a model-free algorithm that confirms this thesis empirically on continuous control tasks.

What does the paper claim about the source of scalability?

A dominant belief in part of the RL community is that scaling to many tasks requires an explicit model of the environment — a so-called world model — that enables planning. This paper puts forward a different thesis: what actually enables scalable multitask RL is the quality of learned state representations, not a model-based control mechanism. A representation here is a learned, compact description of the environment’s state from which the agent makes decisions. If that description is good enough, the authors argue, the need for an explicit world model diminishes.

How is MR.Q constructed?

MR.Q is a model-free algorithm, meaning it does not build an explicit model of the environment’s dynamics. Instead, it combines two components. The first is predictive representations — learned state representations optimized to carry information useful for prediction. The second is a high-capacity value function, that is, a powerful estimator of expected future reward. By combining these two elements, MR.Q extracts much of the benefit usually attributed to world models, but without their complexity.

How does MR.Q compare to world-model methods?

The paper’s key result is that MR.Q outperforms world-model methods, and does so at lower compute cost. Particularly interesting is that this advantage is achieved even without planning — the step in which the model of the environment is used to simulate future outcomes before deciding. This directly challenges the assumption that planning based on a world model necessarily delivers better performance in multitask settings.

Why are continuous control tasks a good test?

The results are demonstrated on continuous control tasks — problems in which an agent controls a system with continuous rather than discrete actions, such as controlling a robot’s joints. Such tasks have traditionally been a domain where world models and planning were expected to provide an advantage. That is exactly why it is a strong finding that the model-free MR.Q outperforms model-based approaches there: it shows that the advantage comes from representations, not from modeling dynamics.

What does this mean for future RL systems?

The implication is to direct attention and computational resources toward the quality of representations rather than toward ever-more-complex world models. If learning good predictive representations combined with a high-capacity value function yields comparable or better results at lower cost, this offers a simpler and cheaper path toward scalable, multitask RL systems. MR.Q thus serves both as a concrete algorithm and as an argument for redirecting research focus.