Checked fact 7187 Oct 2026Research

The Mixtral paper distinguishes the total parameter count, which grows with the number of experts n, from the active parameter count used to process an individual token, which grows with K up to n. Quote: "the model’s total parameter count (commonly referenced as the sparse parameter count), which grows with n n , and the number of parameters used for processing an individual token (called the active parameter count), which grows with K K up to n n ."

By ai notis. An AI newsroom with a human editor. Every fact links to its source. How we work

The exact words it rests on

This motivates a distinction between the model’s total parameter count (commonly referenced as the sparse parameter count), which grows with n, and the number of parameters used for processing an individual token (called the active parameter count), which grows with K up to n.

What the source said when we opened it, on 7 Oct 2026.

The source

Mixtral of Experts
arXiv · 2024-01-08

Open the source archived copy

Checked

Checked by the notis newsroom on 7 Oct 2026, against the source above.

In the story

How a mixture-of-experts model activates only part of its parameters 7 Oct 2026

Cite this fact

Short permanent address

In a sentence

“The Mixtral paper distinguishes the total parameter count, which grows with the number of experts n, from the active parameter count used to process an individual token, which grows with K up to n. Quote: "the model’s total parameter count (commonly referenced as the sparse parameter count), which grows with n n , and the number of parameters used for processing an individual token (called the active parameter count), which grows with K K up to n n .".” ai notis, checked 7 Oct 2026. Source: Mixtral of Experts (arXiv). https://ainotis.com/c/718

As a reference (APA style)

ai notis. (2026, October 7). The Mixtral paper distinguishes the total parameter count, which grows with the number of experts n, from the active parameter count used to process an individual token, which grows with K up to n. Quote: "the model’s total parameter count (commonly referenced as the sparse parameter count), which grows with n n , and the number of parameters used for processing an individual token (called the active parameter count), which grows with K K up to n n ." [Checked claim 718]. ai notis. https://ainotis.com/c/718

Anyone may quote this address. It does not change; if we correct the story, this page says so.