P-EAGLE: Faster LLM inference with Parallel Speculative Decoding in vLLM

P-EAGLE enables parallel generation of draft tokens for speculative decoding in vLLM. The method produces K draft tokens in a single forward pass by using hidden states from the target model and shared mask embeddings for additional positions. This replaces the autoregressive drafting in EAGLE, which required one forward pass per token. The update delivers speedups ranging from 1.05 times to 1.69 times over EAGLE-3 on benchmarks including MT-Bench, HumanEval, and SpeedBench with GPT-OSS 20B on B200 GPUs. Integration into vLLM started with version 0.16.0 through pull request 32887. A configuration flag called parallel_drafting set to true activates the feature in the serving pipeline. Pre-trained parallel drafter heads exist on HuggingFace for GPT-OSS 120B, GPT-OSS 20B, and Qwen3-Coder 30B. The drafter processes prompt positions with corresponding hidden states and uses learnable mask tokens for multi-token prediction slots. A Triton kernel fuses the batch expansion and metadata generation to limit overhead. A sequence partition algorithm supports training on long sequences by splitting positions while preserving attention dependencies. The changes also extend CUDA graph capture ranges to accommodate the parallel slots.