The first act of the generative-AI era was an arms race for size: more parameters, more data, more datacenter. The second act, unfolding now, is the opposite. The most consequential models of 2026 are the small ones — compact enough to run on the device in your hand.
The efficiency turn
Distillation, quantisation and better training data have compressed near-frontier quality into a few billion parameters. A model small enough to fit in a phone's memory now does what took a server-class GPU eighteen months ago.
The question stopped being how big can we make it, and became how much can we take away.
Why local wins
- Privacy — your data never leaves the device
- Latency — no network round-trip, instant responses
- Cost — no per-token bill for every interaction
- Resilience — it works on a plane, in a tunnel, offline
The tradeoff
Small models still trail the giants on the hardest reasoning. The emerging pattern is a hybrid: a local model handles the common case instantly and privately, and quietly escalates to a large cloud model only when the task genuinely needs it. Most of the time, it doesn't.