Learning to Parallel Decoding

dLLM.decoding.demo.mov

🔥News

• [2025-10-14] Dream integration coming soon!

💡Methods

1. Learning to Parallel Decoding

Extremely Greedy Parallel strategy: compares the predicted tokens with the reference answer and only remasks the tokens that do not match in these comparisons. Use a trained filter $f_\theta$ that simulate the Extremely Greedy Parallel strategy after each decoding step to select tokens and decide whether to remask them.

2. End-of-Text Prediction

Upon detection of an $[EoT]$ token, we throw away all the tokens after the $[EoT]$ token in the next diffusion step. When the specified output length is very long (for example, 1024), this method can significantly reduce computation by dynamically reducing the input size during the diffusion process.

🏎️Performance

Experiments on GSM8K, MATH, HumanEval, and MBPP show that our approach significantly improves throughput (by up to 22.58 times faster) while maintaining model accuracy, demonstrating outstanding generalization and practicality. Each method was evaluated using two generation lengths (256 and 1024) across four datasets. Performance is measured using three metrics: TPS (tokens/sec), speedup, and accuracy score. The highest throughput and speedup values for each configuration are highlighted in bold.

How to run

1. Install dependencies

pip install -r requirements.txt

2. Run the program
   1. Test single questions

   python generate.py
   

   2. Run evaluations

   ./eval_llada.sh
   

Generate data for training

1. Download the FLAN dataset to small_model_train/flan
2. Run the following script

./generate_training_data.sh

Training Filter

You can directly use training.ipynb to train new filter models with your own datasets.

Acknowledgments

We would like to thank the authors of LLaDA and Fast-dLLM for their excellent work and open-source contributions.

Citation

If you find our work useful, please consider citing our paper.

@misc{bao2025learningparallelacceleratingdiffusion,
      title={Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding}, 
      author={Wenrui Bao and Zhiben Chen and Dan Xu and Yuzhang Shang},
      year={2025},
      eprint={2509.25188},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2509.25188}, 
}