Commit Your Code 26 ·
Why We Should View Docs as a Product
Documentation in the age of AI — why docs deserve the same product discipline as the experience they support.
A talk from Commit Your Code 26. AI isn’t making documentation less important. It’s changing how people get to it: they might open a page themselves, or ask an agent to read it for them. Either way, the answer still depends on the docs being accurate and having someone responsible for them. That’s the case I make in this talk.
Docs deserve the same care as the product they explain.
What gets tricky is what happens when the docs are unclear or stale. An agent may keep going, and then the user’s task fails. So the talk comes back to a few practical questions. Who is this for? What are they trying to do? Who keeps the docs system running, and which team owns the feature details? Traffic tells you that a request happened. It doesn’t tell you whether the person finished what they came to do.
The numbers behind the talk
Three headline measurements anchor the traffic shift discussed in the talk. Thales measures bot and human traffic across the wider web, while GitBook and Mintlify measure agent and human requests on their own documentation platforms. A Cloudflare measurement provides AI-specific context. They cover different populations, windows, units, and classification rules, so they are separate observations — not one trend line.
53%
bot · 47% human
Bots accounted for 53% of global web traffic in Thales’s full-year 2025 analysis, exceeding human traffic at 47%.
Scope: bots in aggregate, not AI agents alone. Thales says its report combines analysis from its Threat Research and Security Analyst Services teams.
Thales, April 2026 ↗
Slide: The Wider Web
51.8%
AI · 48.2% human
In a one-week GitBook sample, AI-agent requests were 51.8% of requests GitBook classified as intentional reads.
Scope: page requests, not unique readers; GitBook-hosted sites only, with crawler and other-bot categories excluded.
GitBook, May 2026 ↗
Slide: Docs Traffic · GitBook
66%
213M agent · 105M human
On docs powered by Mintlify, agent web requests were 66% of measured traffic in July 2026 — up from a 15.2% share at the start of the year.
Scope: July 2026, Mintlify-hosted docs only; agent requests and human page loads are counted differently, and not comparable with the GitBook figure.
Mintlify, July 2026 ↗
Slide: Docs Traffic · Mintlify
The broader web result is a bot majority, not an AI majority. In its 2025 network analysis, Cloudflare measured identified AI bots at an average 4.2% of HTML requests, with dual-purpose Googlebot broken out separately at 4.5%. That distinction is why the talk moves from the broad automation result to the two documentation-specific measurements instead of treating every bot request as an agent reading docs.
Make docs work on the web first
Before I get into formats built for agents, I want to start with the web. Docs need to work there first. The mechanisms below already have defining specifications, which gives us a good place to start.
| Mechanism | What it does | Defining spec |
|---|---|---|
| Semantic headings | Machine-readable document structure and outline | WHATWG HTML ↗ |
| Durable URLs | Stable identifiers that don’t rot over time | W3C Cool URIs ↗ |
| Canonical link | One authoritative URL per piece of content | RFC 6596 ↗ |
| Sitemaps | Enumerate canonical URLs for discovery | sitemaps.org ↗ |
| robots.txt | Crawl policy for automated clients | RFC 9309 ↗ |
| OpenAPI | Machine-readable description of an API | OpenAPI 3.2.0 ↗ |
| Copy as Markdown | Serve a page as plain | RFC 7763 ↗ |
| llms.txt / llms-full.txt | Curated entry points and exports for LLMs | llmstxt.org ↗ |
I’d still keep one maintained set of docs, with readable pages for people and machine-friendly formats for agents when they need them. Both paths should lead back to the same source.