The publishing pipeline that lied to me for four days
A green log, a stale site, and the difference between a job that ran and a job that did the thing.
By Andrew Pyle
For four days my site published nothing, and every dashboard I had told me it was fine.
01Green log, stale site
The two statements that disagreed
The setup was supposed to be simple. I write short essays, each one dated for a future day, and they reveal themselves one at a time. The site hides anything dated later than today, so a queued essay just appears on its morning. A small job runs nightly and regenerates a schedule document so I can see what is live and what is coming. Its log was clean. Every night it said: sixty-nine essays, live, queued. Healthy.
The site said otherwise. The newest essay was four days old.
The gap between those two statements is the whole lesson. The nightly job was not publishing anything. It read my essay files, counted how many had dates in the past, and wrote that count into a planning note. That is all it ever did. I had built a reporter and been treating it as a publisher. It ran perfectly every night and moved nothing, because moving something was never its job.
02Nothing to do
Then the fix lied too
The real publishing step was a command I ran by hand, and had not run in four days. The queue of pre-written essays had drained to empty. There was no new work to reveal and nothing automated to notice.
Then it got worse, in the instructive way. When I went to re-date a stack of essays that had all landed on the same day, the publish command told me, for several of them, that they were already in their target state and there was nothing to do. It was wrong. The date had changed. The command just was not looking at the date — its idea of unchanged compared the title, the body, and the metadata, everything except the one field I was trying to change. So it skipped the work and reported success.
03Built means observed
Same shape, every time
Two failures, one shape. A job that reports instead of acts. A check that confirms the wrong thing. Both produce the most dangerous output a system can produce: a green light over a task that did not happen.
I keep meeting this pattern. A deploy that passes and ships a feature that renders nothing. A status that is declared rather than observed. The fix is never cleverer automation. It is insisting that done means the real thing was seen to happen, in the place it actually lives.
Not the job exited zero. Not the log looks right. The essay is on the site, dated today, or it is not.
So the check compares the date now. The publisher is the thing that publishes, and the reporter is labeled a reporter. And I went and looked at the live site with my own eyes before I called it fixed — which, embarrassingly, is the step that would have caught all of this on day one.
Related
writing
A close that would not stick
I was about to clean up a third of my task board, and the code that filed those rows would have put them all back.
writing
A queue with no backoff floods on every tick
My approval queue filled with the same pull request on every tick, and the thing it kept trying to merge could never merge.
writing
A dry run that spent money
The flag meant to make a rehearsal safe was making paid model calls and writing real rows.
writing
Active was measuring the wrong thing
My overnight builder thought I was awake from 1 to 7 AM, because the signal it trusted for awake was being touched by something that was not me.
writing
Built means exercised: why a green deploy isn't done
A related-essays card compiled, passed CI, and deployed green. It rendered nothing on all 52 pages. Here is what that taught me about the actual definition of done.