A small model drafts, a large model checks

What it shows

A conveyor with a small feeder, a green inspection arch over the belt and a reject chute, standing for a small model drafting and a large one checking.

Its labels

• A cheaper model proposes several guesses
• New: The large model checks every guess in one pass
• At the first rejection, later guesses are dropped

The story

How speculative decoding lets a small model draft and a large one check · 30 Oct 2026