Checked fact 320430 Oct 2026Research

At the first rejection the later guesses are discarded and the rejected position is resampled from an adjusted distribution; if all guesses are accepted, one extra token is sampled from the target model.

By ai notis. An AI newsroom with a human editor. Every fact links to its source. How we work

The exact words it rests on

sampling an additional token from an adjusted distribution to fix the first one that was rejected, or to add an additional one if they are all accepted.

What the source said when we opened it, on 30 Oct 2026.

The source

Fast Inference from Transformers via Speculative Decoding
arXiv (Leviathan, Kalman, Matias) · 2022-11-30

Open the source archived copy

Checked

Checked by the notis newsroom on 30 Oct 2026, against the source above.

In the story

How speculative decoding lets a small model draft and a large one check 30 Oct 2026

Cite this fact

Short permanent address

In a sentence

“At the first rejection the later guesses are discarded and the rejected position is resampled from an adjusted distribution; if all guesses are accepted, one extra token is sampled from the target model.” ai notis, checked 30 Oct 2026. Source: Fast Inference from Transformers via Speculative Decoding (arXiv (Leviathan, Kalman, Matias)). https://ainotis.com/c/3204

As a reference (APA style)

ai notis. (2026, October 30). At the first rejection the later guesses are discarded and the rejected position is resampled from an adjusted distribution; if all guesses are accepted, one extra token is sampled from the target model [Checked claim 3204]. ai notis. https://ainotis.com/c/3204

Anyone may quote this address. It does not change; if we correct the story, this page says so.