It Happened Again: The Deploy Ate 11 More Images

On May 30 I published a post mortem about seven images disappearing from this site. I explained what broke. I said the real fix was "an afternoon of work I haven't done yet." Then I did not do the afternoon of work.

It happened again on June 28. Eleven images this time. I found out today.

The commit

Here is the thing that deleted them:

bc566fd  2026-06-28  add llms.txt for AI crawler discoverability

A commit whose entire purpose was to add a 2.4KB text file removed eleven PNGs. I did not touch those images. I did not edit anything near them. I added one file and eleven others stopped existing.

The commit before it, 6ba3619 from June 15, has all eleven. I checked the trees side by side. They were there, then they were not, and nothing in that commit’s contents explains it.

Why nobody said anything

Thirty-four days. The Lighthouse post and the lives and ranks update both served empty boxes to every visitor for over a month, in the exact same way they did in May, on the exact same two pages.

Last time I blamed not looking closely enough. That excuse only works once. The second time it's a process problem, and the process was me remembering to check, which is not a process.

While I was in there I found something worse. The site references og-image.png in its Open Graph tags — the preview card that shows up when someone shares a link. That file has never existed. Not deleted, not lost. It was never in a single commit in the history of this repo. Every link anyone has ever shared to TypeVelocity had a broken preview, from day one, and I only noticed because I was auditing every image reference on the site for unrelated reasons.

The actual root cause

TypeVelocity deploys through the GitHub Git Data API instead of git push. The script uploads each file as a blob, assembles them into a tree, commits the tree, moves the branch. Here is the line that has been quietly deleting my work:

// Create tree (no base_tree = clean repo, only these files)
const tree = await ghApi('git/trees', 'POST', { tree: treeEntries });

When you create a tree without base_tree, GitHub does not merge your files into the existing repository. It builds a brand new tree containing only what you passed. Everything else is gone. Not modified, not flagged — absent from the commit, and therefore deleted.

So the deploy rebuilds the entire repo from a hardcoded list on every single push. Anything not in that list — or in the list but missing from my phone at that moment — evaporates.

I wrote that comment myself. I even documented the behavior. I just never followed the sentence through to its conclusion, which is that every deploy silently reduces the repo to whatever happened to be sitting on my phone at the time.

And that second clause turned out to matter far more than the list ever did.

The part I had wrong in May

In the first post mortem I assumed the image entries had been accidentally deleted out of the FILES list during a big update. Reasonable guess. Wrong.

When I opened the script this week, all eleven images were still listed. They had never left. The list was fine.

What actually happens is upstream of the list. The images kept vanishing off my phone — I develop on Android in Termux, and files in that storage disappear on me for reasons I still don't fully understand. Cleanup, permissions, something in the middle. Then the deploy runs:

if (!fs.existsSync(localFile)) {
    deployResults.push({ status: 'skip', size: 0 });
    continue;
}

File isn't on disk, so it never becomes a blob, so it never enters treeEntries, so it isn't in the new tree, so it's deleted from GitHub, so it's deleted from the live site. A local file going missing gets silently upgraded into permanent remote deletion.

And the report for that? A yellow dash and the words "not found," printed among forty-odd green checkmarks, followed by exit 0. The script is telling the truth. It just says it in the same tone as everything else.

That's the chain I never traced in May. I was looking for the human error that removed entries from a list, and the actual bug was that my deploy treated "missing locally" as "delete remotely" — quietly, by design, on every single push.

Why the last fix wasn't a fix

In the May post I said I'd added "a pre-deploy check that logs a warning for any file referenced in a blog post that isn't in the FILES list." I described it as not automated but at least visible.

A warning you have to read is not a check. It scrolls past in a wall of green checkmarks while you're waiting for a deploy to finish on mobile data. Mine did exactly that, assuming it ran at all. Loud means the build stops. Anything softer is decoration.

What I did instead

The fix was not to make the deploy better at carrying images. It was to stop the deploy from touching images at all.

Images now live in a separate public repo and are served over a CDN. The code repo is private, and jsDelivr will not serve from private repos, so they had to be split anyway. The site references absolute CDN URLs. Nothing about rendering this blog depends on a file existing on my phone.

That last part is the whole point. I deleted every image off my machine and reloaded the site to check, and every screenshot still rendered. A deploy can no longer delete something it does not ship. Yesterday's deploy pushed 44 files and not one of them was a PNG.

Three things back it up:

A manifest. One JSON file lists every image on the site with its dimensions, byte size, and SHA-256. Adding an image means adding an entry. There is exactly one list now instead of a list plus my memory.

A gate, not a warning. The deploy scans every HTML file for image references. If one isn't in the manifest, or still points at a local path instead of the CDN, the deploy exits 1 and nothing ships. It also prints exactly which files a deploy is about to remove from the repo, so the clean-tree behavior stops being invisible.

A runtime fallback. Every image walks a chain: jsDelivr, then raw.githubusercontent, then same-origin. If all three fail it renders a bordered placeholder with the caption text instead of a broken-image icon. Failure degrades to something that looks deliberate.

The fallback earned its keep within a minute of being written. One image was still cold on jsDelivr's edge and the mirror served it instead, live, without me doing anything.

Getting them back

My local clone had exactly one commit. Whatever squashed it took the history with it, so the trick from last time — dig through git for the blobs — was not available locally.

The remote still had all 87 commits. So the recovery walks the private repo's history newest-first, finds the last tree that still contained each missing file, pulls the blob, and writes it back to disk. All eleven came back byte-identical. That's a command now, not an afternoon of archaeology:

node assets.js recover

Then I generated the og-image that never existed. No image tooling on Android, so it's a hand-rolled PNG encoder — raw scanlines, zlib deflate, CRC32 chunks, and a 5x7 bitmap font. Roughly 200 lines to draw a rectangle with the logo on it. Worth it, because the og:image tag went in on February 13 and had been pointing at nothing ever since.

The detail that stung

Recovering the files meant reading their real dimensions out of the PNG headers. The Lighthouse post had been declaring width="960" height="540" on all four screenshots. The actual images are 698x800, 695x797, 691x831, and 719x839.

Every one of them was portrait. I'd told the browser they were widescreen. So even during the months when those images did load, they were reserving the wrong space and shifting the layout — on the post where I bragged about getting CLS down to 0.001. The dimensions are correct now, and they're generated from the files rather than typed by me.

The lesson, if there is one

Last time I wrote that the gap worth closing was "make breakage loud." I still think that's right. I just did not act on it, and the thing about a silent failure is that it does not remind you.

The sharper version: if your deploy can delete files, it will eventually delete files. Not because you'll make a dramatic mistake, but because you'll add llms.txt on a Sunday and something unrelated will quietly stop existing. The blast radius of a deploy should be the things you meant to change.

Mine wasn't, across two incidents and eighteen broken images. It is now.