---
title: llms.txt and Markdown Pages in Skyboot AI Markdown
description: "How Skyboot AI Markdown creates llms.txt, llms-full.txt and a clean Markdown version of every WordPress page, and keeps them out of Google's index."
url: "https://skybootstrap.com/docs/skyboot-ai-markdown/llms-txt-and-markdown/"
canonical_url: "https://skybootstrap.com/docs/skyboot-ai-markdown/llms-txt-and-markdown/"
date: "2026-10-24T21:02:07+06:00"
modified: "2026-10-24T21:02:07+06:00"
author: Skybootstrap Team
type: article
site: SkyBootstrap
language: en-US
---

# llms.txt and Markdown Pages in Skyboot AI Markdown

Skyboot AI Markdown gives every WordPress page a clean Markdown version and publishes an index of your site at `/llms.txt`. AI agents read plain Markdown more reliably than full HTML pages with menus, scripts and ads, so these files make your content easier to fetch, understand and quote.

## How do I get the Markdown version of a page?

Every post, page, product, category, tag, author archive and the homepage has a Markdown version. You can request it in three ways:

| Method | Example | Typical use |
| --- | --- | --- |
| `.md` suffix | `https://example.com/my-post.md` | Links in `llms.txt`, AI agents |
| Query argument | `https://example.com/my-post/?format=md` | Homepage, testing |
| Content negotiation | `Accept: text/markdown` header on the normal URL | AI agents that ask for Markdown |

Each Markdown response has three parts:

- YAML front matter with the title, meta description, URL, canonical URL, publish date, last modified date, author, page type and site language. Products add price, currency, availability, SKU, GTIN, brand and rating.
- The content: headings, paragraphs, lists, tables, links and images converted from your page. Shortcodes and breadcrumb widgets are removed, and the text inside shortcodes is kept.
- On listings (category, tag, blog and author pages), each post with a short summary. Products show price and stock.

## Will Markdown pages cause duplicate content in Google?

No. Markdown versions never compete with your real pages:

- Every Markdown response sends `X-Robots-Tag: noindex, follow`, which Google documents in its [robots meta tag guide](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag).
- Every Markdown response sends a `Link: rel="canonical"` header that points to the HTML page, as described in Google’s guide to [consolidating duplicate URLs](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls). If your SEO plugin sets a custom canonical, that one is used.
- When a normal URL answers with Markdown through content negotiation, the response is marked `no-store`, so CDNs never cache it and visitors never see Markdown.

## How do AI agents find the Markdown version?

Each HTML page has a `<link rel="alternate" type="text/markdown` tag in the page head and an HTTP `Link` header that point to its Markdown version. You can switch the headers off under **Settings > Enable HTTP Link Headers**. 404 and search result pages get no Markdown link.

## What is in llms.txt?

`/llms.txt` follows the [llms.txt proposal](https://llmstxt.org/). It contains:

1. Your site name as the title.
2. Your summary as a quote. Write it under **Settings > llms.txt Blockquote Summary**, or click **Write with AI** to draft it from your pages and topics. If it is empty, your site tagline is used.
3. A link to the Markdown version of every published page and post, and of every entry of other post types (documentation, products, portfolios), each type in its own section. Each link has a one-line description, which is your SEO meta description when you set one. Product lines start with price and availability.
4. Your product categories, brands and collections, with product counts.
5. An optional link to `/llms-full.txt`.

These are left out automatically:

- Posts set to noindex in your SEO plugin
- Password-protected posts
- Hidden WooCommerce products
- Cart, checkout, account, receipt and order history pages of any supported store plugin
- Privacy policy and terms pages
- Content types you untick (see below)

`llms.txt` updates itself whenever content is published, updated or deleted. To rebuild it by hand, click **Sync & Regenerate /llms.txt File** on the Settings tab.

## Choose which content AI gets

Under **AI Visibility > Settings > Content types**, tick the post types that should get a Markdown version, an `llms.txt` entry, IndexNow pings and a readiness score. All of them are ticked by default, and a post type added later is included automatically.

Unticked types keep their normal HTML page only: no `.md` version, no Markdown link in the page head and no `llms.txt` entry. Builder templates (Elementor, Divi, Bricks and similar) are never listed.

## What is llms-full.txt?

`/llms-full.txt` contains the full Markdown content of your 50 most recently updated posts and pages in one file. It is off by default. Turn it on under **Settings > Expose /llms-full.txt Endpoint**. It leaves out noindex, password-protected and store utility pages, like `llms.txt`.

## How are llms.txt and Markdown cached?

| URL | Browser cache | CDN cache |
| --- | --- | --- |
| `/llms.txt`, `/llms-full.txt` | 1 hour | 1 day |
| `.md` URLs | **Markdown Response Cache TTL** (default 300 seconds) | Up to 1 day |
| `?format=md` and content negotiation | Not cached | Not cached |

The plugin also keeps its own file cache of generated Markdown and clears it when content changes. To clear it by hand, click **Purge Markdown Cache**. If you use Cloudflare or another CDN, purge `/llms.txt` there after large changes so AI agents get the new version right away.

## How do I check what AI agents receive?

**AI Markdown > Checker** fetches any URL on your site the way an AI agent does. It shows the status code, content type, Link headers, whether front matter is present and the raw Markdown. **Test /llms.txt**, **Test /llms-full.txt** and **Test API Catalog** run quick checks. For security, the Checker only tests URLs on your own site.

## Can I show a “Read as Markdown” link on my pages?

Yes. Add this shortcode anywhere in your content:

```
[skyboot_ai_markdown_link text="Read this page as Markdown"]
```

## FAQ

### What is llms.txt?

llms.txt is a plain-text file at the root of a website that lists its most useful pages for AI tools, with a short summary of the site. The format was proposed at [llmstxt.org](https://llmstxt.org/) in 2024. Skyboot AI Markdown builds and updates it for you.

### Does Google use llms.txt for ranking?

No. Google’s Search team has said Google Search does not use llms.txt, and Google’s guide to [AI features in Search](https://developers.google.com/search/docs/appearance/ai-features) does not ask for one. It is read by AI agents and tools that look for it. The Markdown pages, IndexNow and crawler checks help in either case.

### How do I create an llms.txt file in WordPress?

Install and activate Skyboot AI Markdown. `/llms.txt` is live right away and lists your published content. Add a site summary under **Settings > llms.txt Blockquote Summary** so AI tools know what your site is about.

### Why is my llms.txt not updating?

A CDN is probably serving a cached copy. Purge `/llms.txt` in Cloudflare or your CDN, then click **Sync & Regenerate /llms.txt File** on the Settings tab. If another plugin (for example Rank Math) also creates `llms.txt`, turn its version off.

### Can I remove a post type from llms.txt?

Yes. Untick it under **AI Visibility > Settings > Content types**. To remove single posts, set them to noindex in your SEO plugin, or use the `skyboot_ai_md_llms_exclude_post` filter described in [Developer Hooks](https://skybootstrap.com/docs/skyboot-ai-markdown/developer-hooks/).