Skip to main content
← All talks & appearances

Commit Your Code 26 ·

Why We Should View Docs as a Product

Documentation in the age of AI — why docs deserve the same product discipline as the experience they support.

A talk from Commit Your Code 26. AI isn’t making documentation less important. It’s changing how people get to it: they might open a page themselves, or ask an agent to read it for them. Either way, the answer still depends on the docs being accurate and having someone responsible for them. That’s the case I make in this talk.

Docs deserve the same care as the product they explain.

What gets tricky is what happens when the docs are unclear or stale. An agent may keep going, and then the user’s task fails. So the talk comes back to a few practical questions. Who is this for? What are they trying to do? Who keeps the docs system running, and which team owns the feature details? Traffic tells you that a request happened. It doesn’t tell you whether the person finished what they came to do.

The numbers behind the talk

Three headline measurements anchor the traffic shift discussed in the talk. Thales measures bot and human traffic across the wider web, while GitBook and Mintlify measure agent and human requests on their own documentation platforms. A Cloudflare measurement provides AI-specific context. They cover different populations, windows, units, and classification rules, so they are separate observations — not one trend line.

53%

bot · 47% human

Bots accounted for 53% of global web traffic in Thales’s full-year 2025 analysis, exceeding human traffic at 47%.

Scope: bots in aggregate, not AI agents alone. Thales says its report combines analysis from its Threat Research and Security Analyst Services teams.

Thales, April 2026 ↗

Slide: The Wider Web

51.8%

AI · 48.2% human

In a one-week GitBook sample, AI-agent requests were 51.8% of requests GitBook classified as intentional reads.

Scope: page requests, not unique readers; GitBook-hosted sites only, with crawler and other-bot categories excluded.

GitBook, May 2026 ↗

Slide: Docs Traffic · GitBook

66%

213M agent · 105M human

On docs powered by Mintlify, agent web requests were 66% of measured traffic in July 2026 — up from a 15.2% share at the start of the year.

Scope: July 2026, Mintlify-hosted docs only; agent requests and human page loads are counted differently, and not comparable with the GitBook figure.

Mintlify, July 2026 ↗

Slide: Docs Traffic · Mintlify

The broader web result is a bot majority, not an AI majority. In its 2025 network analysis, Cloudflare measured identified AI bots at an average 4.2% of HTML requests, with dual-purpose Googlebot broken out separately at 4.5%. That distinction is why the talk moves from the broad automation result to the two documentation-specific measurements instead of treating every bot request as an agent reading docs.

Make docs work on the web first

Before I get into formats built for agents, I want to start with the web. Docs need to work there first. The mechanisms below already have defining specifications, which gives us a good place to start.

MechanismWhat it doesDefining spec
Semantic headings

Machine-readable document structure and outline

WHATWG HTML ↗

Durable URLs

Stable identifiers that don’t rot over time

W3C Cool URIs ↗

Canonical link

One authoritative URL per piece of content

RFC 6596 ↗

Sitemaps

Enumerate canonical URLs for discovery

sitemaps.org ↗

robots.txt

Crawl policy for automated clients

RFC 9309 ↗

OpenAPI

Machine-readable description of an API

OpenAPI 3.2.0 ↗

Copy as Markdown

Serve a page as plain .md for machines

RFC 7763 ↗

llms.txt / llms-full.txt

Curated entry points and exports for LLMs

llmstxt.org ↗

I’d still keep one maintained set of docs, with readable pages for people and machine-friendly formats for agents when they need them. Both paths should lead back to the same source.

← Back to speaking