ModelRefs / On-Device Stack — Architecture Pattern
On-Device Stack — Architecture Pattern
Quantized model running fully on device with local cache and optional cloud fallback.
Overview
Privacy-preserving inference: a quantized small model runs on the user's device for common cases; rare cases fall back to a cloud model.
When to use it: You need privacy, offline support, or zero per-call cost for the common case.
Pattern details
| Pattern class | edge |
|---|---|
| Difficulty | advanced |
| Topology | hybrid |
| Also known as | on-device llm, edge inference |
| Last reviewed | 2026-06-07 |
Known failure modes
- Battery drain — Inference exhausts battery. Mitigation: Throttle; prefer Neural Engine.
- Model rot — Old on-device model behind cloud. Mitigation: OTA model updates with gated rollout.
When not to use it
- Sending all queries to cloud when device can serve them.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to On-Device Stack — Architecture Pattern.
Frequently asked questions
When should I adopt the On-Device Stack?
You need privacy, offline support, or zero per-call cost for the common case.
What are common failure modes of On-Device Stack?
Battery drain • Model rot
Is On-Device Stack production-ready?
Yes when paired with the safety controls and observability hooks documented on the pattern page.