3 upvotes
New Study Exposes Training-Inference Gap in LLM Reinforcement Learning
What is On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning?
The paper shows that on-policy RL for large language models assumes the same sampling during training and inference, but real systems use separate engines, creating a mismatch. It traces how this gap forms and how it affects policy quality over time.
Who it is for
Anyone fine-tuning LLMs with RL should read it to spot hidden bias in their pipelines and consider fixes.