vLLM Benchmarks Five Speculative Decoding Methods on AMD Instinct GPUs, Reporting Gains Up to 2.87x
AMD and Embedded LLM tested five speculative-decoding drafting methods in vLLM on AMD Instinct GPUs, with some configurations reaching up to 2.87x throughput over the non-speculative baseline.
Overview
vLLM, the open-source inference and serving engine for large language models, is described on its own GitHub repository as “a high-throughput and memory-efficient inference and serving engine for LLMs” released under the Apache License 2.0. On August 23, 2026, a joint team from AMD and Embedded LLM published benchmarks on the vLLM Blog testing five speculative-decoding drafting methods on AMD Instinct GPUs, reporting that some configurations reached throughput ratios as high as 2.87x over the non-speculative baseline — while other configurations produced smaller gains or fell below baseline.
What We Know
Speculative decoding works by having a lightweight “draft” component propose candidate future tokens, which the full target model then verifies in a single pass, allowing multiple tokens to be committed at once instead of one at a time. The vLLM Blog post examined five such drafting approaches supported by vLLM: native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark. In vLLM, the feature is turned on through the --speculative-config flag, with mtp, eagle3, dflash, and dspark available as method values, according to the vLLM Blog.
According to the vLLM Blog, “several model-workload combinations produced throughput ratios above 2×,” with the largest measured gains including 2.87x for DFlash on the gemma-4-26B-A4B-it model, 2.83x for Gemma 4 MTP on the same model, and 2.68x for DFlash on Kimi-K2.5. The team tested nine target models in total — two Gemma 4 variants, five Qwen models, Kimi-K2.5, and MiniMax-M3-MXFP8 — across the GSM8K, MATH500, HumanEval, and MBPP benchmark datasets, per the vLLM Blog.
The benchmarks ran on two hardware configurations: eight AMD Instinct MI300X GPUs paired with two AMD EPYC 9654 processors, and eight AMD Instinct MI355X GPUs paired with two AMD EPYC 9575F processors for the MiniMax-M3-MXFP8 experiment, using the ROCm open software platform, according to the vLLM Blog. The post credits AMD’s Hongxia Yang and Peng Sun, along with Embedded LLM’s Pin Siang Tan, Jun Kang Chow, and Ye Hur Cheong, for the collaboration, per the vLLM Blog.
Three days after the blog post, on August 26, 2026, the vLLM project shipped version 0.28.0, a release the project describes as containing “584 commits from 270 contributors (76 new).” That release folded in further speculative-decoding work, including “DFlash2 with local convolution and a candidate selector” and “DSpark confidence-scheduled verification,” and it credits an adaptive speculative token budget with delivering “~60% better DSpark TTFT” — time-to-first-token — for the Kimi-K3 model, alongside optional shared-expert sharding that the release notes say saves “~17 GiB of memory per GPU,” according to the v0.28.0 release notes.
What We Don’t Know
The AMD/Embedded LLM benchmarks were run on a pre-release development build of vLLM rather than on the v0.28.0 mainline release, so the specific throughput figures in the blog post do not necessarily carry over unchanged to the version now shipping generally. The blog post itself cautions that “measurements varied by target model, drafting method, workload, and proposal length,” and that increasing the number of speculative tokens requested “sometimes increased throughput over the first few settings, while larger values could lead to a plateau or lower throughput” — meaning there is no single configuration the authors identify as universally optimal, according to the vLLM Blog. Neither source specifies how these AMD GPU results compare to equivalent speculative-decoding benchmarks on Nvidia hardware.