11 upvotes
Researchers fix gradient glitch in multi-token prediction training
What is Multi-Token Prediction Done Right?
Standard multi-token prediction training leaves a stray gradient component that harms downstream accuracy and slows speculative decoding. The new method removes that residual term, delivering cleaner gradients and better model performance.
Who it is for
Anyone building or fine-tuning large language models should adopt the fix to boost both training quality and inference speed.