A language model can be described as having tens of billions of parameters while using a smaller number for each token. In a sparse mixture-of-experts model, those two statements can both be accurate.
The distinction is between the model’s stored capacity and the parts selected for a particular computation. It helps explain the architecture, but does not by itself predict the model’s quality, speed, or hardware requirements.
What the router selects
The original Mixtral of Experts paper describes eight feed-forward expert blocks per layer, with a router selecting two for each token at each layer. Its abstract reports approximately 47 billion parameters available in total and 13 billion active during inference for a token.
Those experts are learned neural-network components. The label does not mean there is necessarily a neatly separated doctor, lawyer, or historian inside the model. The selected components can differ across tokens and layers.
An analogy is a workshop with several machines available but only some engaged for one operation. The analogy explains selective use; it should not be pushed into claiming that neural routing works like a person assigning jobs to human specialists.
Use the two figures for different questions
The total count matters when considering the model’s stored weights. The active count helps describe computation per token, but memory use and practical performance also depend on implementation, numerical precision, context length, batching, and hardware.
Suppose two deployment proposals advertise the same active-parameter count. That alone does not establish that they require the same memory or answer equally quickly. Ask for measurements under the intended workload, including the input lengths and concurrency you expect.
Likewise, a model’s name is not always a multiplication exercise. Shared components and the exact architecture affect the totals. Use the paper or model documentation rather than inventing a parameter count from branding.
For a comparison sheet, keep architecture, total parameters, active parameters, test conditions, and observed performance in separate fields. That preserves the distinction between what a model is built from and what it actually does in your application. Neither count should quietly become a substitute for an evaluation.