Cloudflare announces a new feature that automatically converts HTML content into Markdown in real time, specifically designed for AI agents. The starting observation is simple: HTML is extremely inefficient for consumption by language models. A simple heading that requires 3 tokens in Markdown consumes 12 to 15 in HTML. At the scale of a full web page, the conversion delivers an 80% reduction in the number of tokens used, with direct gains in cost and latency.
The mechanism relies on HTTP content negotiation, an existing web standard. When an AI agent sends a request with the Accept: text/markdown header, Cloudflare's edge network intercepts the origin server's HTML response and converts it on the fly into Markdown before returning it to the agent. This approach is particularly elegant because it requires no modification on the origin server side: everything happens on Cloudflare's infrastructure.
The article also presents the Content Signals framework, which allows website publishers to express their preferences regarding the use of their content. Three signals are defined: ai-train (training permission), search (indexing for search) and ai-input (use as input by AI agents). This mechanism gives publishers granular control over how agents interact with their content.
The authors point out that popular coding agents such as Claude Code already send the Accept: text/markdown header, which reflects organic adoption of this approach. For more complex use cases, Cloudflare offers complementary alternatives: Workers AI with the AI.toMarkdown() method for programmatic conversion, and the Browser Rendering API for pages requiring JavaScript rendering.
Cloudflare Radar makes it possible to track markdown usage patterns across different AI crawlers, offering valuable visibility into the evolution of this ecosystem. In addition, Cloudflare is launching a Toolshed bringing together more than 400 MCP (Model Context Protocol) tools, positioning itself as a central platform for AI agent tooling.
The feature is available in beta for subscribers to the Pro, Business and Enterprise plans. This initiative is part of a broader trend in which web infrastructure adapts to serve not only human browsers but also autonomous AI agents, which are becoming major consumers of web content.