Checked fact 320730 Oct 2026Research

The paper reports 2X to 3X acceleration with identical outputs on T5-XXL (11B), at batch size 1 on a single TPU-v4, compared with the T5X implementation.

By ai notis. An AI newsroom with a human editor. Every fact links to its source. How we work

The exact words it rests on

We demonstrate it on T5-XXL and show a 2X-3X acceleration compared to the standard T5X implementation, with identical outputs. ... We measure walltime improvements with a batch size of 1 on a single TPU-v4

What the source said when we opened it, on 30 Oct 2026.

The source

Fast Inference from Transformers via Speculative Decoding
arXiv (Leviathan, Kalman, Matias) · 2022-11-30

Open the source archived copy

Checked

Checked by the notis newsroom on 30 Oct 2026, against the source above.

In the story

How speculative decoding lets a small model draft and a large one check 30 Oct 2026

Cite this fact

Short permanent address

In a sentence

“The paper reports 2X to 3X acceleration with identical outputs on T5-XXL (11B), at batch size 1 on a single TPU-v4, compared with the T5X implementation.” ai notis, checked 30 Oct 2026. Source: Fast Inference from Transformers via Speculative Decoding (arXiv (Leviathan, Kalman, Matias)). https://ainotis.com/c/3207

As a reference (APA style)

ai notis. (2026, October 30). The paper reports 2X to 3X acceleration with identical outputs on T5-XXL (11B), at batch size 1 on a single TPU-v4, compared with the T5X implementation [Checked claim 3207]. ai notis. https://ainotis.com/c/3207

Anyone may quote this address. It does not change; if we correct the story, this page says so.