Today we're launching Inworld Inference, designed from the ground up for consume
By Kylangibbs · October 16, 2026 · Curated by George's Blog
Today we're launching Inworld Inference, designed from the ground up for consumer AI.
AI's first wave was built for companies. The bigger wave is built for people.
We are now serving well into the tens of trillions of tokens a month. Up 25x since March. As we've learned what matters for consumer-facing developers, our inference has grown hand in hand with top apps and we are excited to open it up publicly today.
Building for people is a different engineering problem:
- Latency correlates with revenue: when a reply is slow, people go back to tapping buttons. Every turn resends the whole history, so prompts run up to 40 times longer than replies.
- Cost structures differ: one interaction fans out into many model calls, from moderation and memory to ranking and the reply itself, yet consumers pay less, so each of those calls has to cost almost nothing
- Reliability is make or break: when a consumer app goes down for ten minutes, its users leave for another one, while an enterprise workflow just waits.
We built for the first token, the cache and the slowest requests:
- Nearly 8x faster first token than most other providers.
- ~40% lower cost.
- 99.999% reliability.
The same research drives all three:
- Speculative decoding with per-model drafts, or the model's own multi-token prediction: 1.6x faster streaming on DeepSeek V3.2
- Custom attention kernels for the verify step: up to 7.5x faster attention on Gemma 4 26B
- Fused mixture-of-experts kernels: 3x faster first token on MiMo 2.6 Pro
- KV-cache-aware routing: up to 90% of each prompt from cache on DeepSeek V4 Flash
- A serving build per model: 2.8x faster first token on Gemma 4 31B at the same scores
With your own model, we go further: post-training against your evals, a draft trained on your traffic, tuned kernels and a dedicated deployment. One customer's fine-tune now runs at ~30% lower cost.
Who are you building for?