GitHub Models has been retired. If your prototypes or workloads depend on it, act now to avoid outages. Here’s a fast, vendor‑neutral migration playbook grounded in lessons from platform deprecations. Source: Simon Willison.
What changed and why it matters
Hosted model endpoints come and go. The fix is to decouple your app from any single provider so you can swap endpoints in hours, not weeks.
Quick migration checklist (60 minutes)
- Inventory: List every place your app calls GitHub Models (backends, notebooks, evals, batch jobs).
- Abstract: Wrap model calls behind a single interface (e.g.,
llm.generate(prompt, model, params)). - Switch: Point that interface to an alternative, starting with an OpenAI‑compatible endpoint (set
OPENAI_BASE_URLandOPENAI_API_KEY). - Evaluate: Re‑run key prompts; log diffs in latency, cost, and output quality. Approve a fallback model per task.
- Guardrails: Enforce timeouts, retries, rate limits, and max tokens at the adapter layer—not per call site.
- Cache: Add deterministic caches for frequent prompts and enable transcript logging for rollback.
- Monitor: Track provider health, spend, and error codes. Alert on model/version drift.
Providers to consider
- OpenAI‑compatible endpoints (swap via
OPENAI_BASE_URL): OpenAI, Together, Fireworks, Groq, Replicate, Mistral-hosted, and others. - Cloud suites: Azure AI model endpoints and Google Vertex AI (note: different SDKs; use your adapter).
- Self/edge: Ollama or vLLM for private or on‑prem use; good for regulated data and cost control.
Best practice: build an API adapter
Create a thin adapter translating your app’s calls to provider‑specific SDKs. Keep prompts, safety settings, and sampling params in one place so you can swap models by config, not code. Tools like LiteLLM can help if you prefer a maintained router.
Minimal interface sketch: generate({model, input, temperature, max_tokens, system, tools}) -> {text, usage, latency, provider}. Start with the superset of params you need across providers; ignore the rest.
Source and further reading
- News: GitHub Models is now retired — Simon Willison.
- Reference: OpenAI API reference (many providers offer OpenAI‑compatible endpoints).
- Router option: LiteLLM (multi‑provider adapter with cost/latency routing).
Takeaway
Treat model providers as interchangeable. Wrap calls, centralize config, and test fallbacks now so a sunset notice never takes your app down again.
Get weekly, no‑fluff playbooks like this in your inbox—subscribe to our newsletter: theainuggets.com/newsletter.
