Simon Willison highlights that cheaper AI models are winning adoption over Anthropic’s flagship. The lesson: default to the cost-quality sweet spot, not the absolute “best” model.
Why this matters
Model choice shapes margins, user experience, and roadmap velocity. Overpaying for every request taxes budgets and slows iteration.
Action playbook: ship quality, pay less
- Set a mid-tier default. Most workloads run fine on balanced models; reserve premium for critical cases.
- Route by task criticality. Use simple rules or a small classifier to send only high-stakes requests to the flagship model.
- Build a lightweight eval harness. Track quality, latency, and token cost on a small golden set; re-run on every model or prompt change.
- Add budget guardrails. Cap max tokens, constrain outputs (JSON mode/functions), set timeouts, and fail gracefully with retries.
- Cut prompt/token waste. Keep system prompts tight, chunk and dedupe RAG context, and summarize histories instead of replaying full threads.
- Cache what you can. Template prompts, store semantic/exact-match results, and reuse deterministically generated outputs.
- Stream and batch. Stream for perceived latency; batch vector queries and non-urgent jobs to reduce overhead.
- Watch prices and drift. Re-benchmark monthly and monitor vendor pricing pages—markets shift fast.
When to use the flagship model
- High-stakes outputs: legal, compliance, medical, or safety-critical content with human-in-the-loop review.
- Complex, multi-step reasoning where failure costs are high and accuracy beats speed or price.
- Premium user tiers or enterprise SLAs that explicitly pay for the best available quality.
Quick model portfolio template
- Premium brain (5–10% of calls): only for P0 tasks and escalations.
- Workhorse mid-tier (70–85%): default for most chat, RAG, and agent steps.
- Bulk/cheap (5–25%): classification, embeddings, summarization, and batch jobs.
Sources
Context via Simon Willison: Anthropic’s best AI model struggles to attract users as cheaper tiers win. Pricing references: Anthropic pricing, OpenAI pricing.
Takeaway
Make a mid-tier model your default, route only critical tasks to the flagship, and keep a tight eval loop. You will protect margins without sacrificing quality.
Like this? Get one practical AI nugget in your inbox weekly: Subscribe to our newsletter.
