Speculative Decoding

LLM foundationsModels and inferencePublished By Simon Budziak

Speculative decoding speeds up language model generation by letting a smaller draft model propose several tokens, then asking the larger target model to verify them together. Accepted tokens match the target model's normal distribution, so the method can reduce latency without changing the intended output quality.

Autoregressive models usually produce one token at a time. That makes response speed depend on a series of target-model passes. Speculative decoding changes the rhythm, not the answer: a cheap model races ahead, then the expensive model checks a group of proposed tokens in parallel.

Speculative decoding flow: a small draft model proposes four tokens, a target model verifies them together, accepted tokens continue to the response and the first rejected token restarts the draft

Speculative decoding is only faster when the target model accepts enough draft tokens to amortize the verification pass.

How does speculative decoding preserve output quality?

The draft model proposes a short run of tokens. The target model scores that whole run in one pass, accepts consecutive proposals that fit its own distribution, and resumes from the first rejection. The target remains the final authority on every emitted token. In the common exact version of the method, acceptance and correction rules preserve the same distribution as ordinary inference, rather than accepting a plausible shortcut.

The apparent paradox is hardware. A large model can often verify several positions together more efficiently than it can wait for one sequential decode at a time. A good draft therefore converts otherwise idle parallel work into accepted output tokens.

Serving software still needs to keep the draft and target states aligned after every accepted or corrected segment. That implementation work can erase a theoretical gain when request batches are small or the draft model is poorly matched.

What determines whether it actually speeds up a service?

Draft acceptance rate is the central metric. A weak draft model forces frequent corrections and adds overhead. A strong draft can make one target pass yield several final tokens. Measure accepted tokens per verification pass on representative traffic, not a hand-picked prompt. A draft model earns its place only when accepted tokens outweigh its verification overhead. Model size, sampling settings, batch size, and request shape all affect the result.

Is this the same as model routing?

No. Model routing chooses which full model should handle a request, often by cost or difficulty. Speculative decoding uses two models together inside one response. It also differs from a KV cache: cache reuse avoids recomputing earlier attention state, while speculation reduces the number of sequential decode cycles. For a fresh uncached request, neither reduces prefill or time to first token. Their effect is on decode-stage latency and streaming throughput after generation begins. Time to first token still depends on connection setup, queueing, and reading the prompt.

This separation matters when setting a performance target. Use speculative decoding when sustained output speed is the measured bottleneck. Reduce prompt size, queueing, or repeated-prefix work when the complaint is the initial wait.

Frequently asked questions

Does speculative decoding make a model less accurate?

A correct implementation uses the target model to accept or reject draft tokens, preserving the target distribution. The performance gain depends on how often the target accepts the draft.

When does speculative decoding help most?

It helps when the draft model predicts the target's next tokens well enough that several proposals can be accepted per target-model pass.

Summarize this page with

Train your team to build this