---
title: "Mixture of experts (MoE)"
canonical: "https://router.xark.io/glossary/mixture-of-experts"
description: "A mixture-of-experts model divides its parameters into many specialised sub-networks and routes each token through only a few of them."
section: "glossary"
updated: "2026-09-02"
source: "https://router.xark.io/glossary/mixture-of-experts.md"
---

# Mixture of experts (MoE)

*Also known as: MoE, sparse model, active parameters, sparse mixture of experts.*

A mixture-of-experts model divides its parameters into many specialised sub-networks and routes each token through only a few of them, so the parameters actually used per token — the active parameters — are a small fraction of the model's total size.

A dense model puts every token through every parameter. An MoE model puts each token through a router that selects a handful of experts, so compute per token scales with the active parameter count rather than the total. That is how a very large model can generate at the speed of a much smaller one.

The naming convention encodes both numbers. A model described as 235B A22B has around 235 billion total parameters and around 22 billion active per token. Total size sets the memory the weights occupy; active size sets the compute and memory bandwidth each token costs.

Those two numbers pull serving economics in opposite directions. All the weights must be resident even though only a fraction is used per token, so an MoE model is memory-hungry to host and cheap to run once hosted. That favours providers operating at enough scale to keep the weights loaded and busy.

Most of the strongest open-weight models released recently use this architecture, so it is now the common case rather than an exotic variant.

## What it costs you

Serving cost tracks active parameters and memory footprint, not the headline parameter count, which is why a model with a very large total size is not automatically the expensive one on a rate card. Reading a price purely off a parameter count will mislead in both directions — and it also explains why per-token prices for large open-weight models have fallen faster than model sizes have grown.

## Related terms

- [Quantization](https://router.xark.io/glossary/quantization.md) — Quantization is the compression of a model's weights from higher-precision numbers to lower-precision ones — typically 16-bit down to 8-bit or 4-bit — which reduces the memory and compute needed to serve it, at some cost to output quality.
- [Distillation](https://router.xark.io/glossary/distillation.md) — Distillation is the training of a small model to reproduce the outputs of a larger one, so that the smaller student performs close to the teacher across a particular range of work while costing a fraction as much to serve.
- [Open-weight model](https://router.xark.io/glossary/open-weight-model.md) — An open-weight model is one whose trained parameters are published under a licence permitting others to download, run and serve them, which is what makes it possible for anyone other than the original lab to offer that model as an API.
- [Throughput (tokens per second)](https://router.xark.io/glossary/throughput.md) — Throughput is the rate at which a model emits output tokens once generation has started, usually quoted in tokens per second, and together with time to first token it determines how long a complete response takes.

## See also

- [Every model and its architecture](https://router.xark.io/models)
- [Full pricing table](https://router.xark.io/pricing)