Checked fact 320230 Oct 2026Research

In the paper's algorithm a cheaper approximation model autoregressively proposes gamma guess tokens, and the target model scores the prefix and all guesses in one parallel pass, producing gamma+1 distributions.

By ai notis. An AI newsroom with a human editor. Every fact links to its source. How we work

The exact words it rests on

Sample γ guesses x1,...,xγ from Mq autoregressively. ... Run Mp in parallel. p1(x), ..., pγ+1(x) <- Mp(prefix), ..., Mp(prefix + [x1, ..., xγ])

What the source said when we opened it, on 30 Oct 2026.

The source

Fast Inference from Transformers via Speculative Decoding
arXiv (Leviathan, Kalman, Matias) · 2022-11-30

Open the source archived copy

Checked

Checked by the notis newsroom on 30 Oct 2026, against the source above.

In the story

How speculative decoding lets a small model draft and a large one check 30 Oct 2026

Cite this fact

Short permanent address

In a sentence

“In the paper's algorithm a cheaper approximation model autoregressively proposes gamma guess tokens, and the target model scores the prefix and all guesses in one parallel pass, producing gamma+1 distributions.” ai notis, checked 30 Oct 2026. Source: Fast Inference from Transformers via Speculative Decoding (arXiv (Leviathan, Kalman, Matias)). https://ainotis.com/c/3202

As a reference (APA style)

ai notis. (2026, October 30). In the paper's algorithm a cheaper approximation model autoregressively proposes gamma guess tokens, and the target model scores the prefix and all guesses in one parallel pass, producing gamma+1 distributions [Checked claim 3202]. ai notis. https://ainotis.com/c/3202

Anyone may quote this address. It does not change; if we correct the story, this page says so.