P-EAGLE: Faster LLM inference with Parallel Speculative Decoding in vLLM

This article introduces P-EAGLE, a parallel speculative decoding method that accelerates LLM inference in vLLM by generating multiple draft tokens in a single forward pass instead of sequentially.

• P-EAGLE generates K draft tokens in one forward pass, eliminating autoregressive EAGLE's sequential bottleneck
• Achieves up to 1.69x speedup over vanilla EAGLE-3 on NVIDIA B200 GPUs with real workloads
• Pre-trained P-EAGLE heads available on HuggingFace for GPT-OSS 120B, GPT-OSS 20B, and Qwen3-Coder 30B
• Integrated into vLLM v0.16.0+ with simple configuration: set "parallel_drafting": true
• Uses learnable mask tokens and hidden states as placeholders for future token positions
• Implements fused Triton kernel to minimize batch metadata rebuild overhead
• Consistently achieves higher acceptance lengths than EAGLE-3 across benchmarks
• Performance gains range from 55-69% at low concurrency to 5-25% at high concurrency

P-EAGLE removes the sequential drafting ceiling, enabling deeper speculation without additional latency overhead, making it suitable for production LLM deployments.