The paper says "The relative probabilities of incorrect answers tell us a lot about how the cumbersome model tends to generalize."

The exact words it rests on

the trained model assigns probabilities to all of the incorrect answers and even when these probabilities are very small, some of them are much larger than others. The relative probabilities of incorrect answers tell us a lot about how the cumbersome model tends to generalize.

What the source said when we opened it, on 13 Oct 2026.

The source

Distilling the Knowledge in a Neural Network (full text)
arXiv · 2015-03-09

Open the sourcearchived copy

Checked

Checked by the notis newsroom on 13 Oct 2026, against the source above.

In the story

How distillation works and when it becomes an attack 13 Oct 2026