Two names for one thing fail closed
Every news draft was blocked as LLM unavailable, and the model was not down at all.
By Andrew Pyle
A provider name spelled two ways looked exactly like an outage, because the safe failure path was built to look like one.
01The error text
The block reason read like an outage
The block reason on every news draft began with llm_unavailable. The rest of the string ended with Unknown provider: 'anthropic'. The first half is what I would expect to see if the model provider were having an outage. The second half is a lookup failing on a name. Both halves sat in the same field, and the first half is the one that gets read first: the model is down, so wait it out.
llm_unavailable:...Unknown provider: 'anthropic'Two builders were affected, one that drafts prep football news and one that drafts sports news. Each has four settings at the top of its file: a provider and a model for drafting, and a provider and a model for verifying what the draft says. Both provider settings in both files said anthropic. On every run of the daily content engine, every draft from those two builders was blocked and marked as needing a human check.
02Two lookups
Generation and the probe asked different registries
The drafting code was fine. Generation and verification both go through a small factory that takes a provider name and hands back a client. That factory treats anthropic and claude as the same thing. Had a draft reached that step, the name would have resolved.
The drafts never reached it. Before any attempt to write about real people or real businesses, the engine runs a cheap availability probe. The probe takes a different path, through a provider manager with its own registry, and that registry knows the provider only as claude. It looked up anthropic, found nothing, and raised.
The history explains how both sides could be right. The two builders were written in late June and early July, against the factory, where anthropic works. The probe arrived in early September, as part of a daily content engine that started life dormant and dry run first. Each piece was correct on the day it was written. The mismatch only existed once both sat in the same call path.
03Closed by design
Failing closed made a typo look like capacity
The probe is meant to fail closed. Its docstring says it runs before a real person or real business generation attempt, and that anything that raises fails closed. Most model failures do not raise at that layer. They are recorded as an errored run and returned quietly. An unknown provider was different. It raised, a broad exception handler caught it, and the probe returned false with the exception name and message attached.
That is the right policy for content that names real people. I would rather block a draft than publish one the verifier never saw. The price of the policy is that a configuration typo and a real outage land in the same bucket, under the same label, and only the tail of the message tells them apart.
One more thing kept it hidden. The probe only runs for sites that name real entities, and it is skipped entirely in dry run mode. The dry run branch returns a would generate decision before the probe is ever called. So every dry run I could have watched was structurally unable to see this. Only a live run on a site that names real entities would trip it.
04One name, one test
Rename one side and guard the class
Four lines, on one side
The change itself was four lines. I renamed the provider setting for drafting and for verifying in each builder from anthropic to claude, so both paths now see a name they know. I left the registry alone. The factory still accepts the old spelling, and older stored templates that say anthropic are overridden by an explicit argument where the job runs, so nothing that leaned on the old name broke. Cost estimates were safe too, because pricing keys on the client class's own provider attribute, not on the builder's string.
A test for the whole class
The second half of the change is a guard test. It covers both builders and both provider settings, eight cases in all, and asks two questions of each name: is it in the manager's list of registered providers, and does the factory return the expected client for it. The failure message says plainly that the probe will block every draft. The test has to import the providers module for its side effect, because importing it is what registers them.
To prove the guard could fail, I set one builder back to anthropic and ran it: one failed, seven passed. With the rename in place, ninety tests passed across the guard, both news builders, and the engine's own suite. Because the test asks whether a name is registered, a third spelling would fail it too.
Two things are still open. The test proves the registry lookup, not a live call, and I have not yet seen a live run come back without an llm_unavailable block on the news builders. A check on the next run is filed for that. The guard also covers only these two builders. Any other module that names a provider by bare string sits outside it.
These same files already carried a warning from an earlier round: a comment noting that some pinned model ids had started returning 404 at the API and must not be reused. So names in this code go stale from both sides. The model behind a name changes, and the lookup in front of it changes. A string that two parts of a system share needs one place that defines it, or a test that proves both sides still agree.
Related
writing
A close that would not stick
I was about to clean up a third of my task board, and the code that filed those rows would have put them all back.
writing
A queue with no backoff floods on every tick
My approval queue filled with the same pull request on every tick, and the thing it kept trying to merge could never merge.
writing
A dry run that spent money
The flag meant to make a rehearsal safe was making paid model calls and writing real rows.
writing
Dead is not asleep
A liveness alarm kept telling me to revive a laptop that was healthy and asleep, until I changed what the alarm counts as dead.
writing
Active was measuring the wrong thing
My overnight builder thought I was awake from 1 to 7 AM, because the signal it trusted for awake was being touched by something that was not me.