Deployment
On-device AI
On-device AI runs models directly on user hardware.
Quick definition
On-device AI runs models directly on user hardware.
- Category: Deployment
- Focus: hosting and runtime tradeoffs
- Used in: Running local models for privacy-sensitive workflows.
What it means
It reduces latency and keeps data local. In deployment workflows, on-device ai often shapes hosting and runtime tradeoffs.
How it works
Deployment choices include cloud APIs, local inference, or hybrid setups. Each option trades off privacy, cost, and performance.
Why it matters
Deployment choices affect privacy, performance, and cost.
Common use cases
- Running local models for privacy-sensitive workflows.
- Using managed APIs for fast iteration and scaling.
- Hybrid setups that keep data local but call cloud models.
Example
Use a Mac to run an offline assistant.
Pitfalls and tips
Local deployments require hardware planning and updates. Cloud deployments require governance and cost control.
In BoltAI
In BoltAI, this appears in provider, hosting, or local model settings.
