5 upvotes
Study reveals which unmasking policies speed up diffusion language models
What is What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models?
The authors compare different learned unmasking strategies for masked diffusion language models and measure how they affect the speed-accuracy trade-off. Their controlled experiments isolate the impact of supervision, input features, and model capacity.
Who it is for
Anyone building fast text generators should read it to pick the most efficient unmasking approach.