# goodbot-badbot — agents.md

> This site is an *observer* of AI agents and crawlers, not an agent itself.
> It exposes exactly one callable surface — a read-only MCP server over its own
> measurements. Nothing here can be changed by a caller.

## What this is

goodbot-badbot.com measures whether crawlers respect robots.txt. Six honeypot
paths are listed as `Disallow` in /robots.txt. Any request to one of them is
logged as a violation, regardless of user-agent, and published live.

## What an agent can read here

- [Live scoreboard](https://goodbot-badbot.com/)
- [Machine-readable stats](https://goodbot-badbot.com/api/stats)
- [LLM-oriented summary](https://goodbot-badbot.com/llms.txt)
- [Crawl rules](https://goodbot-badbot.com/robots.txt)

Please honour /robots.txt. The `Disallow` list is the entire experiment.

## What an agent can call here

`POST https://goodbot-badbot.com/mcp` — a stateless MCP server speaking
protocol revision `2026-07-28`. No `initialize` handshake, no sessions: send
`server/discover`, `tools/list` or `tools/call` as a single JSON-RPC POST with
the `MCP-Protocol-Version` and `Mcp-Method` headers. Two read-only tools:

- `get_compliance_stats` — the full crawler scoreboard
- `check_bot` — one crawler, by display name or User-Agent string

Because the revision is stateless there is no handshake to learn the rules
from: the first request either carries every required header or is rejected
with `-32020 HeaderMismatch`. A complete, working call:

```
curl -X POST https://goodbot-badbot.com/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -H 'MCP-Protocol-Version: 2026-07-28' \
  -H 'Mcp-Method: tools/list' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
```

`Mcp-Method` must equal the body's `method`. For `tools/call` add
`Mcp-Name: <tool name>`, matching `params.name`. `GET` and `DELETE` return
`405` — this revision removed the GET stream and protocol-level sessions.

There is no A2A endpoint and no write operation of any kind.

## Transparency

Requests to this file and to /mcp are logged (user-agent plus a
SHA-256-truncated IP hash, the same privacy model as the rest of the site) so
we can measure which agents probe for them. MCP has no *ratified* discovery
standard; since August 2026 the site also serves /.well-known/mcp-server per
draft-serra-mcp-discovery-uri, an individual Internet-Draft with no IETF
standing. So how an agent finds /mcp — by reading this file, by reading the ARD
manifest or that draft manifest, or by guessing the path outright — is itself
the measurement.