Models
Mixture of experts
Mixture of experts (MoE) routes inputs to specialized sub-models.
Quick definition
Mixture of experts (MoE) routes inputs to specialized sub-models.
- Category: Models
- Focus: model capability and fit
- Used in: Choosing a model that fits latency and cost constraints.
What it means
It can improve quality while keeping compute manageable. In models workflows, mixture of experts often shapes model capability and fit.
How it works
Model architecture and scale determine capability. Context length, parameter count, and modality support vary across models.
Why it matters
Model architecture affects capability, context length, and speed.
Common use cases
- Choosing a model that fits latency and cost constraints.
- Selecting longer context for document-heavy workflows.
- Using specialized models for code, vision, or speech.
Example
A router selects the best expert for each token.
Pitfalls and tips
Bigger is not always better. Match the model to the task and evaluate in production.
In BoltAI
In BoltAI, this shows up in model selection and configuration.
