Your website has a new audience. It's not human.

Almost half of documentation traffic is now AI agents. And I think this applies to all websites, not just docs.

This week I came across a blog post from Mintlify that made me stop and think.

If you don’t know Mintlify, they power the documentation sites for some of the biggest tech companies in the world, like Anthropic, Microsoft, Notion, PayPal, Perplexity, and many others. These are not small companies. These are the ones building the future of AI, payments, and search.

And Mintlify just shared data showing that almost half of their documentation traffic is now coming from AI agents.

Not humans. Agents.

When I read that, it felt like a real confirmation of the shift I’ve been writing about since Issue #1. The companies at the center of this change are already seeing it in their own data.

What Mintlify is seeing

Mintlify described it simply. Agents show up with a task. They look for an answer. If they find it, they take it and leave. If they don’t, they move on. No support ticket. No complaint. They just quietly go somewhere else. Or worse, they make something up.

That’s the scary part. When agents fail on your site, you don’t hear about it. There’s no signal. There’s just a developer somewhere getting a wrong answer and blaming your product.

How Mintlify is solving it

I’ve been reading through how they’re approaching this, and I’m learning a lot from it. Two things stood out.

First: llms.txt

This is a simple file that sits on your site and tells agents what content exists, how it’s organized, and what they can skip. Think of it as a map. But for AI, not humans.

We’ve had sitemaps for search engines for years. But agents need something different. They don’t want to crawl everything. They want to know where to go right away. Mintlify builds this into their platform so agents don’t have to guess.

Second: content negotiation.

This means serving the same content in different ways depending on who’s asking. A human visitor gets the nice, designed page. An agent gets a clean, simple version, like markdown, that’s easy to read and process.

This connects to something I covered in Issue #3. Cloudflare launched “Markdown for Agents”, which converts HTML pages to clean markdown when an agent asks for it. 80% less tokens. Same information, different format.

Mintlify is building this same idea into their docs platform from the start. One source of content. Two ways to read it.

Here’s what got me thinking

Mintlify is doing all of this for documentation sites. But as I was reading through their approach, I kept asking myself: why would this only apply to docs?

If agents are already making up almost half the traffic on documentation sites, the same thing will eventually happen to other types of websites too. Product pages. Landing pages. Service pages. Portfolios. Blogs. Online stores.

The pattern is the same one I’ve been learning about since Issue #1. Humans are sending agents to find information, compare options, and make decisions for them. Agents don’t care about your animations or clever headlines. They care about whether they can understand what you do and whether it’s relevant to the task they were sent to do.

And if they can’t figure that out, they leave. Quietly.

What I think this means for websites

I don’t have a full answer yet. I’m still figuring this out. But watching how Mintlify and Cloudflare are handling agent traffic is giving me ideas about what “agent-ready” might look like for regular websites.

A machine-readable map. Something like llms.txt that tells agents what’s on your site and where to find it.

Clear, structured content. Not just visually clear, but clear in meaning.

Different formats for different readers. If an agent requests your pricing page, maybe it should get a clean version instead of the same heavy JavaScript page a human sees.

Agent analytics. Without knowing when agents visit your site and what they do there, you can’t improve anything. You don’t know if they’re finding what they need or leaving confused.

Why I’m paying attention to this

At our company, we currently sell website templates and build custom websites for clients. And the more I learn from companies like Mintlify and Cloudflare, the more I realize that building websites only for humans means building for a shrinking part of the audience.

The companies that learn this early and make their sites work for both humans and agents will have a big advantage. Their products will get picked up by AI assistants. Their services will show up when agents compare options. Their content will be what agents trust and share.

The ones who don’t adapt will slowly become invisible. Not overnight. But quietly, steadily, and then all at once.

I’m going to keep learning from the companies that are ahead on this and sharing what I find. How do we make websites, not just docs, but all websites, ready for the agentic web?

If you’re thinking about this too, I’d love to hear what you’re seeing.

- Farhad