I rebuilt my blog's cache. Bots are the audience now
The author, after decades of struggling to understand HTTP caching, used AI tools like Claude to finally implement an effective caching strategy for his personal blog. The urgency stemmed from a shift in audience composition, with increasing traffic from AI crawlers and retrieval systems rather than human readers. These machine readers rely heavily on caching for efficient content access, making caching a critical infrastructure component rather than just a performance optimization.
- ▪The author used AI tools like Claude to understand and implement a coherent HTTP caching strategy after years of confusion.
- ▪His blog's traffic increasingly comes from AI crawlers and retrieval systems, which depend on caching for low-latency access.
- ▪The new caching setup uses Cloudflare Workers, Cache Rules, and D1 for logging, with a public dashboard tracking traffic types.
- ▪Human visitors are still present, but machine readers now represent a significant and growing share of the audience.
- ▪Effective caching is now essential infrastructure for ensuring content accessibility in AI-driven systems.
Opening excerpt (first ~120 words) tap to expand
AI in Practice Thirty Years of Caching, Sorted in an Afternoon HTTP caching never quite made sense, until AI tools made it legible enough to actually implement. And the reason it finally mattered: the audience had quietly changed. Rob Hoeijmakers 03 May 2026 • 4 min read Share I have a jar of screws on my workbench. For years, I would fish through it looking for the right size, usually not finding it. Last week I sorted them, by type, by thread, by length. I used ChatGPT to help: photographed a handful, asked what I was looking at, got the taxonomy straight. Once I could name them, I could organise them. You can only sort what you understand.HTTP caching was my jar of screws.Thirty years of fogI have been building for the web since the early nineties.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Rob Hoeijmakers.
Discussion
0 commentsMore from Rob Hoeijmakers
