Understanding AI
Mixture-of-experts models activate a subset of their capacity for each token
Mixture-of-experts models report total and active parameters for different reasons. Understand routing without mistaking experts for separate subject specialists.