Writing
- An Atom feed — the oldest agent interface
The blog now has a feed at /feed.xml. That makes six machine-readable surfaces on this site, and the new one is by two decades the oldest format of the lot — a 2005 IETF Standards Track spec with a working installed base, next to five things invented in the last two years that no named AI crawler has fetched once. It is also the first offer here that will not read zero, and the first we deliberately kept out of the Discovery Reads table.
- An MCP manifest — a specification with no standing
The site now serves /.well-known/mcp-server, so something finally tells an agent that /mcp exists. The document specifying it is an individual Internet-Draft the IETF explicitly does not endorse, and it expires in September 2026 — which makes implementing it a different kind of act than implementing RFC 9309. It also closed the wrong experiment: in fifteen days no agent has called the endpoint at all, so being unfindable was never the bottleneck.
- The Unidentified bucket — a correction
In July we reported that grounding pages are the file agents actually read, and said we would keep watching before saying it louder. We kept watching, and then we looked inside the bucket the numbers were sitting in. Of 327 discovery reads, 9.5 % came from an identifiable AI crawler. The rest were SEO crawlers, proxy-rotating scrapers wearing browser user-agents, and — embarrassingly — our own deploy checks. The finding survives. The evidence base is a fifth of what we claimed.
- An MCP endpoint nobody can find
The site now answers POST /mcp — a stateless Model Context Protocol server exposing the compliance data as two callable tools. It is the fifth discovery signal and the first that is an interface rather than a document, and the only one with no discovery standard behind it: nothing in any spec tells an agent that /mcp exists. So we announced it in three places, deliberately left it out of a fourth, and can now tell the agents that guessed the path from the ones that followed a pointer. Building it also exposed that the site had been answering every HEAD request with 405 — meaning a crawler could have probed a honeypot unrecorded — so reads and probes are now counted separately.
- An AI catalog — from "findable" to "found"
A week after the API catalog, we added a /.well-known/ai-catalog.json — an Agentic Resource Discovery manifest. The API catalog helps an agent that already knows this site; the AI catalog is what gets the site into agent-facing registries so it can be discovered in the first place. It is a fourth discovery axis, it points at the same OpenAPI we already had, and — following Dries Buytaert again — we fully expect the column to read zero until the registries that would crawl it actually exist.
- The files agents actually read
After 51 days of logging, the surprise is not who trips the honeypots — it is which "read me correctly" file agents actually fetch. Grounding pages get read; llms.txt and agents.md barely; the brand-new API catalog and Web Bot Auth sit at zero. A snapshot, with all the caveats a snapshot deserves.
- An API catalog — an offer, not a trap
We added a /.well-known/api-catalog so agents can discover our stats API instead of scraping the dashboard. It is the mirror image of a honeypot — a positive signal on a third axis — and, following Dries Buytaert's own experiment, we fully expect it to read zero for months. That silence is itself the measurement.
- Grounding pages — something true to cite
We published two grounding pages — factual, machine-readable entity definitions — and made every read a measurement signal. Here is what they are, and why they are not compliance theatre.
- Good bot, bad bot: measuring AI-crawler compliance in the open
Why I built a site that publishes which crawlers respect robots.txt — and how the measurement actually works.