arXiv: TruDi — diffusion policies for massively parallel on-policy RL
TruDi is the first method that enables diffusion policies to train stably in a massively parallel on-policy RL setting, by applying trust-region optimization with a KL-divergence constraint enforced across the entire diffusion trajectory. Across 73 tasks spanning 4 benchmarks it achieves results competitive with or better than strong baselines, with clear gains on humanoid control.