How to Serve Markdown to AI Agents from WordPress

To serve Markdown to AI agents, return a Markdown version of a page when the request asks for one: with an Accept: text/markdown header, a .md URL or a ?format=md query string. The header method, called content negotiation, is the standard one and the one Claude Code, Cursor, and OpenCode use.

Be clear about who benefits. In published tests and server logs, coding agents and some crawlers take the Markdown. ChatGPT, the Claude web app, and Perplexity, answering people’s questions, still read your HTML.

WordPress does none of this out of the box. On our WordPress 7.1.3 test site, an Accept: text/markdown request and ?format=md both returned the normal HTML page, and /post-name.md returned a 404. Below are the three methods, who uses them, and a tested way to add them. If you’re setting up llms.txt as well, start with our guide to llms.txt on WordPress.

Content negotiation and the other two ways to ask for Markdown

An agent can ask for Markdown with an Accept: text/markdown header on the normal URL, by adding .md to the URL, or with a ?format=md query string. The Accept header is the request header where a client lists the formats it wants, in order of preference.

Content negotiation is answering that header on the same URL: one address, two formats, chosen by what the client asks for.

Method	What the request looks like	Who uses it
Accept header	GET /my-post/ with Accept: text/markdown	Coding agents that negotiate (Claude Code, Cursor, OpenCode)
.md URL	GET /my-post.md	Agents and crawlers that follow a published link to it
Query string	GET /my-post/?format=md	Tools you control; the easiest to test in a browser

A .md URL is the page address with .md added, which is the form the llms.txt proposal recommends for Markdown copies of pages. A successful negotiated response looks like this:

HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
Vary: Accept
X-Robots-Tag: noindex

Support the header first, add .md for agents that only follow links, and keep ?format=md as a convenience. The .md and ?format=md versions have their own URLs, so caches keep them apart without extra headers. Only the Accept method shares a URL with the HTML page.

Which agents ask for Markdown

Coding agents such as Claude Code, Cursor, and OpenCode ask for Markdown first, while consumer chat assistants like ChatGPT, the Claude web app, and Perplexity still fetch the HTML page. An AI agent here means any tool that fetches web pages for a person, from a coding assistant reading API docs to a chatbot.

Checkly tested the web-fetch tools of seven coding agents in February 2026. Claude Code 2.1.38 sent text/markdown, text/html, */*, and Cursor 2.4.28 and OpenCode 1.2.5 also ranked Markdown first. OpenAI Codex, Gemini CLI, GitHub Copilot, and Windsurf did not ask for it.

GoodBarber’s server logs from September 2026 show where one site’s requests came from over two weeks:

• About 80,000 requests went to its .md links, roughly half from Amazon, Meta and OpenAI crawlers. Googlebot made none.
• About 22,000 asked through the Accept header, mostly from ShapBot, ExaSearchBot and Claude Code (around 1,400 requests).

Everyday chat assistants behaved differently. ChatGPT, the Claude web app and Perplexity fetched HTML more than 13,000 times between them and asked for Markdown once.

So Markdown pays off for documentation, reference pages and anything developers point their agents at. Pages are also far smaller in Markdown (GoodBarber measured about 80% less), which our HTML vs Markdown token test on WordPress pages covers in detail.

Tell agents a Markdown version exists

Add a rel="alternate" link with type="text/markdown", in the page head or as an HTTP Link header, so agents that don’t send the Accept header can still find the Markdown version. A rel="alternate" link is a standard way for a page to say “the same content is also available here, in this format”. The llms.txt proposal recommends exactly this:

<link rel="alternate" type="text/markdown" href="https://example.com/my-post.md">

The header form works too and needs no theme changes:

Link: <https://example.com/my-post.md>; rel="alternate"; type="text/markdown"

Plugins that serve Markdown can add both for you. Ours puts the rel="alternate" tag in each HTML page’s head and sends the matching Link header, which you can switch off under Settings → Enable HTTP Link Headers. It leaves both out of 404 and search pages (our documentation).

Crawlers follow links you publish, which is why GoodBarber saw so many .md requests from crawlers that didn’t send the Accept header. Your llms.txt can point to the same .md URLs; how to write those entries is in our annotated llms.txt examples.

Keep HTML as the page search engines index

Send Vary: Accept on pages that can return both formats, mark Markdown responses noindex, and never switch formats by user agent, so browsers and Google keep getting HTML. The Vary header tells caches that the response depends on a request header. A cache then stores the HTML and Markdown versions separately instead of handing one to everyone.

• Send Vary: Accept on every page that can answer with Markdown, including HTML responses. Without it, a page cache can serve a stored Markdown copy to your visitors. Not every page cache honours Vary, so our serve-markdown.php mu-plugin also tells them not to store Markdown responses. Making LiteSpeed, nginx and Cloudflare respect it is covered in fixing caches that serve AI bots the wrong version.
• Add X-Robots-Tag: noindex to Markdown responses so .md URLs don’t appear in search results next to the real page.
• Only serve Markdown when the request asks for it. Guessing from the user agent and sending bots different content risks being treated as cloaking, which Google defines as showing users and search engines different content.

Google says you don’t need “Markdown to appear in Google Search”, and that Search itself doesn’t use such files. The HTML page stays the one that ranks; Markdown is an extra for agents.

Our plugin follows these rules by default, per the documentation linked below. Every Markdown response carries X-Robots-Tag: noindex, follow and a canonical Link header pointing to the HTML page. If your SEO plugin sets a custom canonical, that one is used. Negotiated responses are sent no-store, so a CDN doesn’t keep them and visitors never see Markdown.

Serve Markdown to AI agents from WordPress with a small plugin

A must-use plugin can answer all three request methods, add the discovery link and Vary header, and convert post content to Markdown; we tested this one on WordPress 7.1.3 with nginx and Apache. Save it as wp-content/mu-plugins/serve-markdown.php (create the mu-plugins folder if it doesn’t exist) with SFTP or your host’s file manager. To remove it, delete the file. Must-use plugins load automatically and can’t be switched off from the Plugins screen, so test on a staging copy first.

With Plain permalinks the plugin advertises ?format=md instead of .md, and a static front page gets /index.md.

<?php
/**
 * Plugin Name: Serve Markdown to AI agents
 * Description: Markdown versions of posts and pages via Accept: text/markdown, .md URLs and ?format=md.
 */

// Which format does the request want? HTML unless Markdown is preferred.
function smd_wants_markdown() {
	if ( isset( $_GET['format'] ) && 'md' === sanitize_key( wp_unslash( $_GET['format'] ) ) ) {
		return true;
	}
	$accept = $_SERVER['HTTP_ACCEPT'] ?? '';
	$best   = array( 'type' => '', 'q' => -1 );
	foreach ( explode( ',', $accept ) as $i => $part ) {
		$bits = array_map( 'trim', explode( ';', $part ) );
		$type = strtolower( $bits[0] );
		$q    = 1.0;
		foreach ( $bits as $b ) {
			if ( 0 === strpos( $b, 'q=' ) ) {
				$q = (float) substr( $b, 2 );
			}
		}
		if ( $q <= 0 ) {
			continue; // q=0 means "not acceptable".
		}
		$q -= $i / 1000; // Equal q-values: the first listed type wins.
		if ( in_array( $type, array( 'text/markdown', 'text/html' ), true ) && $q > $best['q'] ) {
			$best = array( 'type' => $type, 'q' => $q );
		}
	}
	return 'text/markdown' === $best['type'];
}

// Map /some-post.md to /some-post/ before WordPress parses the request.
add_action( 'parse_request', function ( $wp ) {
	if ( preg_match( '#^(.+)\.md$#', $wp->request, $m ) ) {
		$front   = 'page' === get_option( 'show_on_front' ) ? (int) get_option( 'page_on_front' ) : 0;
		$post_id = 'index' === $m[1] ? $front : url_to_postid( home_url( $m[1] . '/' ) );
		if ( $post_id ) {
			$wp->query_vars        = array( 'p' => $post_id, 'post_type' => get_post_type( $post_id ) );
			$GLOBALS['smd_force'] = true;
			remove_action( 'template_redirect', 'redirect_canonical' );
		}
	}
} );

// Only public, published, unprotected content gets a Markdown version.
function smd_can_serve( $post ) {
	return $post instanceof WP_Post && 'publish' === $post->post_status
		&& is_post_type_viewable( $post->post_type ) && ! post_password_required( $post );
}

// The Markdown address for a post: .md, index.md for a static front page, ?format=md without pretty permalinks.
function smd_md_url( $post ) {
	if ( ! get_option( 'permalink_structure' ) ) {
		return add_query_arg( 'format', 'md', get_permalink( $post ) );
	}
	if ( 'page' === get_option( 'show_on_front' ) && (int) get_option( 'page_on_front' ) === $post->ID ) {
		return home_url( '/index.md' );
	}
	return untrailingslashit( get_permalink( $post ) ) . '.md';
}

// Runs late (priority 99) so membership, private-site and maintenance plugins act first.
add_action( 'template_redirect', function () {
	$post = get_queried_object();
	if ( ! is_singular() || ! smd_can_serve( $post ) ) {
		if ( ! empty( $GLOBALS['smd_force'] ) ) {
			header( 'X-Robots-Tag: noindex' ); // A .md URL that can't be served shows HTML; keep it out of search.
		}
		return;
	}
	header( 'Vary: Accept', false );
	header( 'Link: <' . esc_url_raw( smd_md_url( $post ) ) . '>; rel="alternate"; type="text/markdown"', false );
	if ( ( empty( $GLOBALS['smd_force'] ) && ! smd_wants_markdown() ) || ! class_exists( 'DOMDocument' ) ) {
		return;
	}
	if ( ! defined( 'DONOTCACHEPAGE' ) ) {
		define( 'DONOTCACHEPAGE', true ); // Page-cache plugins: don't store the Markdown version.
	}
	nocache_headers();
	header( 'Content-Type: text/markdown; charset=utf-8' );
	header( 'X-Content-Type-Options: nosniff' );
	header( 'X-Robots-Tag: noindex' );
	echo '# ' . html_entity_decode( get_the_title( $post ), ENT_QUOTES, 'UTF-8' ) . "\n\n";
	echo smd_to_markdown( apply_filters( 'the_content', $post->post_content ) );
	exit;
}, 99 );

add_action( 'wp_head', function () {
	if ( is_singular() && smd_can_serve( get_queried_object() ) ) {
		echo '<link rel="alternate" type="text/markdown" href="' . esc_url( smd_md_url( get_queried_object() ) ) . '">' . "\n";
	}
} );

// A small HTML-to-Markdown converter for common post markup.
function smd_to_markdown( $html ) {
	$doc = new DOMDocument();
	libxml_use_internal_errors( true );
	$doc->loadHTML( '<?xml encoding="utf-8"?><div>' . $html . '</div>' );
	return trim( preg_replace( "/\n{3,}/", "\n\n", smd_node( $doc->getElementsByTagName( 'div' )->item( 0 ) ) ) ) . "\n";
}

function smd_node( $node ) {
	$out = '';
	foreach ( $node->childNodes as $c ) {
		if ( XML_TEXT_NODE === $c->nodeType ) {
			$out .= preg_replace( '/\s+/', ' ', $c->textContent );
			continue;
		}
		if ( XML_ELEMENT_NODE !== $c->nodeType ) {
			continue;
		}
		$tag   = strtolower( $c->nodeName );
		$inner = smd_node( $c );
		switch ( $tag ) {
			case 'h1': case 'h2': case 'h3': case 'h4': case 'h5': case 'h6':
				$out .= "\n\n" . str_repeat( '#', (int) $tag[1] ) . ' ' . trim( $inner ) . "\n\n";
				break;
			case 'p':
				$out .= "\n\n" . trim( $inner ) . "\n\n";
				break;
			case 'br':
				$out .= "\n";
				break;
			case 'strong': case 'b':
				$out .= '**' . $inner . '**';
				break;
			case 'em': case 'i':
				$out .= '*' . $inner . '*';
				break;
			case 'code':
				$out .= 'pre' === strtolower( $c->parentNode->nodeName ) ? $c->textContent : '`' . $c->textContent . '`';
				break;
			case 'pre':
				$fence = str_repeat( chr( 96 ), 3 );
				$out  .= "\n\n" . $fence . "\n" . rtrim( $c->textContent ) . "\n" . $fence . "\n\n";
				break;
			case 'a':
				$out .= '[' . trim( $inner ) . '](' . $c->getAttribute( 'href' ) . ')';
				break;
			case 'img':
				$out .= '![' . $c->getAttribute( 'alt' ) . '](' . $c->getAttribute( 'src' ) . ')';
				break;
			case 'li':
				$mark = 'ol' === strtolower( $c->parentNode->nodeName ) ? '1.' : '-';
				$out .= "\n" . $mark . ' ' . trim( $inner );
				break;
			case 'ul': case 'ol':
				$out .= "\n" . $inner . "\n\n";
				break;
			case 'table':
				$rows = array();
				foreach ( $c->getElementsByTagName( 'tr' ) as $r => $tr ) {
					$cells = array();
					foreach ( $tr->childNodes as $td ) {
						if ( XML_ELEMENT_NODE === $td->nodeType ) {
							$cells[] = trim( str_replace( '|', '\\|', smd_node( $td ) ) );
						}
					}
					$rows[] = '| ' . implode( ' | ', $cells ) . ' |';
					if ( 0 === $r ) {
						$rows[] = '|' . str_repeat( ' --- |', count( $cells ) );
					}
				}
				$out .= "\n\n" . implode( "\n", $rows ) . "\n\n";
				break;
			case 'figure': case 'figcaption': case 'blockquote':
				$out .= "\n\n" . ( 'blockquote' === $tag ? '> ' : '' ) . trim( $inner ) . "\n\n";
				break;
			case 'script': case 'style': case 'noscript':
				break;
			default:
				$out .= $inner;
		}
	}
	return $out;
}

The converter handles headings, paragraphs, bold and italic, links, lists, code, images, and tables. Nested lists come out flat, and anything other plugins add to post content (share buttons, related posts) is included. Complex layouts, shortcodes that output scripts, and page builders will come out rough; for those, use one of the plugins in the plugins and CDN table at the end of this page.

Check it with curl:

curl -sI -H "Accept: text/markdown" https://yoursite.com/my-post/ | grep -i "content-type\|vary\|x-robots"
curl -sI https://yoursite.com/my-post/ | grep -i "content-type\|link"

On Windows, run curl.exe instead of curl and drop the | grep … part, or open https://yoursite.com/my-post/?format=md in a browser.

If ?format=md works but the Accept test returns text/html, a page cache answered before WordPress ran. Purging confirms it, but the cache refills with HTML until it varies on Accept; the caching guide linked in the section on keeping HTML indexable covers each layer.

What our test site returned:

Request	Response
Normal browser request	HTML, with Vary: Accept and the Markdown Link header
/my-post.md, ?format=md, or Accept: text/markdown	text/markdown; charset=utf-8, noindex
Claude Code’s header (text/markdown, text/html, */*)	Markdown
Accept: text/html, text/markdown;q=0.5, text/markdown;q=0, or Accept: */*	HTML
A password-protected post asked for Markdown	HTML password form; the content stayed hidden
/no-such-post.md	404
A maintenance-style plugin blocking the post (priority 10)	The block applies to .md, ?format=md and Accept requests too
Static front page, or Plain permalinks	Discovery link points to /index.md, or to ?format=md

q-values are the 0–1 preference weights in the Accept header. A client that rates HTML higher gets HTML, and when weights are equal, the type listed first wins. Browsers, and agents like Gemini CLI that send */*, see your normal page.

If you’d rather not maintain this code, a plugin does the same job; the comparison at the end of this guide includes ours.

Frequently Asked Questions

Plugins and CDN options

Two plugins list all three methods: Roots’ Post Content to Markdown and our own plugin (per its documentation). AIOSEO served .md URLs but ignored the Accept header in Roots’ April 2026 test; the free AIOSEO 5.0.3 on our test site had no .md URLs. One CDN option, Cloudflare, converts pages at the edge. Check any plugin with the two curl commands from the plugin section before trusting its feature list.

Option	Accept header	.md URL	Query string	Vary / noindex	Notes
Roots Post Content to Markdown	Yes (q-values)	Yes	?format=markdown	Both	Installed with Composer (needs SSH, not the Plugins screen); posts only by default
Skyboot AI Markdown (ours)	Yes	Yes	?format=md	noindex header; negotiated responses sent no-store	Paid; also builds llms.txt and sends a Link header (from our docs)
Lunatec Article to Markdown	Yes	Yes	Not stated	Not stated	Version 1.0.0, under 10 installs, tested up to WordPress 6.9.10
AgentMark	Not stated	Yes	Not stated	Not stated	Version 1.0.1; also llms.txt
AIOSEO	No (HTML returned, per Roots)	Yes in Roots’ April 2026 test; none in free 5.0.3 on our site	Not stated	Not stated; no discovery link	Check your version with curl
Cloudflare Markdown for Agents	Yes	No	No	Adds Vary: Accept; noindex not stated	Converts at the edge; Pro plan and up

Cloudflare’s option is the least work if your site already runs through it, since nothing changes in WordPress. A WordPress plugin gives you more control over what goes into the Markdown, such as front matter and which post types are included.

In ours, that covers posts, pages, products, categories, tags, author archives, and the homepage. Each Markdown page opens with front matter: title, meta description, canonical URL, publish and modified dates, author and language, plus price, stock, SKU, and brand on product pages. You choose which content types get a Markdown version under AI Visibility → Settings → Content types. AI Markdown → Checker fetches any of your pages the way an agent does, so you can confirm the result without curl.