arXiv:2606.24014: RL training on health domain transfers alignment to 80%+ OOD benchmarks
Google Research researchers showed that RL training on beneficial properties such as truthfulness, fairness, and corrigibility improves performance on more than 80% of 50+ independent OOD benchmarks — including domains outside health on which the model was trained.
This article was generated using artificial intelligence from primary sources.
What is alignment transfer and why does it matter?
Alignment transfer refers to a model’s ability to apply beneficial properties learned in one domain — such as healthcare — in completely different contexts without additional training. Google Research published the paper “Reinforcement Learning Towards Broadly and Persistently Beneficial Models” (authors: Akshay V. Jagadeesh, Rahul K. Arora, Khaled Saab, and collaborators), showing that this transfer is possible and measurable at scale.
How was the RL training conducted?
Researchers constructed datasets measuring four beneficial properties: truthfulness, fairness, risk awareness, and corrigibility (the ability for a model to be corrected or stopped). Training was conducted primarily in the health, science, and education domains. The key result: improvements were recorded on more than 80% of a total of 50+ independent OOD (out-of-distribution) benchmarks — i.e., on evaluations outside the training domain. Unlike the classical approach where each use case is aligned separately, this model achieves generalization from a single training pass.
What does this mean in practice?
Models trained with this approach show greater resistance to adversarial prompts — attempts by users to steer them toward harmful responses — as well as to harmful fine-tuning, where an attacker attempts to make a subsequently trained model harmful. At the same time, the approach reduces reward hacking, where a model optimizes the reward metric without actually learning the desired behavior. Health RL specifically produces broad improvements on non-health alignment evaluations — suggesting that domain-specific training need not be a silo.
Why is this a breakthrough?
Previous models required separate alignment for each application. This paper demonstrates that beneficial behavior is transferable — similar to how a physician who develops ethical habits in medicine applies the same principles to business decisions. The paper was submitted on 2026-06-22 and raises the question of whether a single well-constructed RL training phase will become a standard part of the pipeline for every large model.
Frequently Asked Questions
- What does OOD mean in the context of AI alignment?
- OOD (out-of-distribution) refers to benchmarks or domains the model has not seen during training — a genuine test of generalization, as the model must apply learned principles in entirely new situations.
- Can alignment transfer replace training for each domain separately?
- Not entirely, but the results show that health RL brings improvements on non-health evaluations, suggesting that beneficial properties have general rather than domain-specific effects.