Quick definition
Edge inference runs models close to the user instead of a central cloud.
- Category: Deployment
- Focus: hosting and runtime tradeoffs
- Used in: Running local models for privacy-sensitive workflows.
What it means
It improves latency and can increase reliability. In deployment workflows, edge inference often shapes hosting and runtime tradeoffs.
How it works
Deployment choices include cloud APIs, local inference, or hybrid setups. Each option trades off privacy, cost, and performance.
Why it matters
Deployment choices affect privacy, performance, and cost.
Common use cases
- Running local models for privacy-sensitive workflows.
- Using managed APIs for fast iteration and scaling.
- Hybrid setups that keep data local but call cloud models.
Example
Run inference on a local gateway or device.
Pitfalls and tips
Local deployments require hardware planning and updates. Cloud deployments require governance and cost control.
In BoltAI
In BoltAI, this appears in provider, hosting, or local model settings.
