The model that stopped existing

A dependency you did not write can be withdrawn from under you, and the withdrawal can look exactly like a bug in your own code.

Our documentation chat is a thin thing. A question goes to a worker, the worker adds a fixed knowledge base as context, calls a hosted model, and returns the answer. Perhaps two hundred lines. It had been working for days.

Then I rotated the API key — routine hygiene, nothing to do with the feature — and the chat stopped answering. Same code. Same deployment. Same request shape. A new key.

What the logs actually said

The worker logs the provider's status code for every attempt, which turned out to be the only reason this took four minutes instead of an evening:

[chat] primary-model returned 429
[chat] fallback-model returned 404
[chat] all attempts failed, last status 404

Two different failures, and the difference between them is the whole story. A 429 is a rate limit: the model exists, the request was understood, you are asking too often. A 404 is the provider saying the thing you named is not there.

Except it was there. The provider's own model-listing endpoint returned it, with a description, a token limit, and an explicit statement that it supported the exact method we were calling. Listed, described, documented — and dead.

Why the key mattered

The model was being retired. Ahead of the published shutdown date, the provider had closed it to new credentials while leaving existing ones working. Our old key was grandfathered in. The new one was not. Rotating the key did not break the model; it revealed that we had been holding the only kind of access that still worked.

This is worth sitting with, because the ordinary mental model of a credential is that it authenticates you and nothing more. Here the key also carried an entitlement — access to a model generation — and that entitlement did not survive rotation. Nothing in the error said so.

What we changed

The fix was not the interesting part; we moved to a current model generation and the chat came back. The interesting part is what we changed around it.

Pin the primary, float the fallback

The primary model is now pinned to a specific released version, because a support bot that has been carefully tuned not to overstate what the product does should not have its behaviour change underneath it without warning. The fallback points at the provider's floating latest alias instead. If the pinned version is retired the way this one was, the alias resolves to something alive and the feature degrades rather than dies.

Stability where you want predictable behaviour. A moving target where you want survival.

Never leave a dead default in the code

Our source still carried the old model name as a hardcoded default, masked by an environment variable that overrode it. That is a trap set for a future maintainer: delete the variable and the application silently reaches for something that no longer exists. Defaults in code should be values you would be content to run on.

Log the provider's status, not your own summary

Users saw one message: the model could not be reached. That is the right thing to show a person. It is the wrong thing to be the only record. Had the worker not written the upstream status code to its own logs, a 404 and a 429 would have been indistinguishable, and the obvious suspect — the key I had just changed — would have been the wrong one.

The general shape

Hosted model APIs are not like other dependencies. A library you have pinned stays pinned. A model you call over HTTP can be deprecated, restricted, or withdrawn by someone else's roadmap, and your deployment pipeline will report success throughout, because from its point of view nothing failed. The code shipped. It is the far end that moved.

We met this at midnight, with nobody using the feature, because a routine key rotation happened to trip it. The alternative was meeting it in October, on the published shutdown date, with customers in the room. That was luck. The logging was not.