---
title: "ClaudeBot, Claude-SearchBot and Claude-User Explained for WordPress Owners"
description: "ClaudeBot is Anthropic's training crawler. What it, Claude-SearchBot and Claude-User do, what blocking each costs, tested robots.txt rules and IP checks."
url: "https://skybootstrap.com/claudebot/"
canonical_url: "https://skybootstrap.com/claudebot/"
date: "2026-11-03T14:23:00+06:00"
modified: "2026-11-03T12:52:36+06:00"
author: Skybootstrap Team
type: article
site: SkyBootstrap
language: en-US
---

# ClaudeBot, Claude-SearchBot and Claude-User Explained for WordPress Owners

ClaudeBot is Anthropic’s training crawler: it collects public web pages Anthropic may use to train Claude models. Anthropic runs two more bots with different jobs: Claude-SearchBot for search, and Claude-User for fetching a page when someone asks Claude. Each one can be allowed or blocked on its own.

On tlinky.com, one of our own WordPress sites, requests using Anthropic’s bot names had the two largest counts in about one week of data: 404 and 252, against 222 IP-verified bingbot hits. But a name in a log proves nothing, which is why the IP check further down matters.

## Anthropic’s three bots, and what blocking each one costs

ClaudeBot gathers training material for Anthropic’s Claude models, Claude-SearchBot indexes pages for search, and Claude-User fetches a page when someone asks Claude a question. [Anthropic’s crawler page](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler), dated 7 April 2026, describes all three and what blocking each one does.

| Bot | Job | If you block it, Anthropic says |
| --- | --- | --- |
| ClaudeBot | Collects public content that may help train Claude models | Your future content should be left out of training datasets |
| Claude-SearchBot | Crawls to improve the quality of search results | Your content isn’t indexed for search, which may reduce your visibility and accuracy in user search results |
| Claude-User | Opens pages when a person asks Claude something | Claude can’t retrieve your content for that person, which may reduce visibility in user-directed web search |

Blocking ClaudeBot is a training decision. The other two are how Claude finds and opens your pages when people use it, so Anthropic says blocking them may reduce your visibility when people use Claude.

Anthropic’s page names only these three. Older names you may see in old robots.txt templates, such as Claude-Web and anthropic-ai, aren’t on it. Rules written only for them don’t cover the bots Anthropic runs today.

OpenAI’s bots page splits its own crawlers by job too, with [GPTBot, OAI-SearchBot and ChatGPT-User](https://skybootstrap.com/gptbot/), so you can make one decision per job across both companies.

## How much Anthropic traffic a WordPress site gets

Requests using Anthropic’s bot names on tlinky.com came to 404 calling themselves Claude-User and 252 calling themselves ClaudeBot, over roughly seven days ending 28 October 2026. Those were the two biggest counts on the first page of the plugin’s crawler table; bingbot was next with 222 verified hits. The site runs our [Skyboot AI Markdown plugin](https://skybootstrap.com/skyboot-ai-markdown/), and its analytics count these requests by user agent (the name a bot sends with each request) only.

That caveat matters. The plugin checks crawler IPs against official lists from OpenAI, Google, Bing, Perplexity, Apple and DuckDuckGo. It doesn’t check Anthropic’s list yet, so it shows Claude rows as “No public IP list” and leaves them out of its verified totals ([our documentation](https://skybootstrap.com/docs/skyboot-ai-markdown/ai-traffic-analytics/)). Anthropic does publish a list now, and the section after next shows how to check it yourself.

The activity log shows what ClaudeBot did. Over about 33 minutes it requested category, author and post pages and the home page. For some pages it asked twice, once for the normal HTML page and once for the Markdown version our plugin serves. One Claude-User request fetched a statistics page in its Markdown version.

![Latest activity on tlinky.com in Skyboot AI Markdown: ClaudeBot requesting html and md versions of category, author and post pages](https://skybootstrap.com/wp-content/uploads/2026/11/claudebot-tlinky-latest-activity-1024x685.webp "Latest activity on tlinky.com in Skyboot AI Markdown: ClaudeBot requesting html and md versions of category, author and post pages")

Visits back were few. The same week, Claude sent 5 of the 35 visits the plugin counted from AI assistants (each visitor counted once per page and day). [Cloudflare’s network data](https://blog.cloudflare.com/crawlers-click-ai-bots-training/) points the same way at a larger scale. It measured about 38,066 Anthropic crawls per referral by July 2025, down from about 286,930 in January 2025.

## robots.txt rules for Anthropic’s bots

Anthropic says its bots honour robots.txt and that it supports the non-standard Crawl-delay line; its example names ClaudeBot only. A group for the training bot with `Disallow: /` opts you out of training, and `Crawl-delay` asks it to slow down.

To opt out of training only, add one group and leave Claude-SearchBot and Claude-User alone:

```
User-agent: ClaudeBot
Disallow: /
```

To keep the crawler but slow it down, use Crawl-delay, a line asking the bot to pause for that many seconds between fetches. Copy every rule from your `*` group into this one, admin rules included, because a bot with its own group ignores your `*` group. Back up robots.txt before you change it:

```
User-agent: ClaudeBot
Crawl-delay: 10
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
```

We ran a WordPress robots.txt holding that group through Protego 0.7.0, an open-source Python library that reads robots.txt the way Google’s spec describes. The parser gave ClaudeBot a 10-second delay. Of the three paths we tested (`/`, a members page and `/wp-admin/`), only `/wp-admin/` was disallowed for it.

Where to put these lines depends on how your site manages robots.txt; our [WordPress robots.txt guide for AI crawlers](https://skybootstrap.com/robots-txt-ai-bots-wordpress/) covers Yoast, Rank Math, AIOSEO, and a physical file.

If you’d rather click than type, our plugin’s training-crawler setting covers it. Set to Block, it adds Disallow lines for Anthropic’s training bot along with the other training bots it lists. That works as long as WordPress builds the robots.txt itself, and it never blocks Claude-SearchBot or Claude-User ([our documentation](https://skybootstrap.com/docs/skyboot-ai-markdown/ai-visibility/)). Its crawler access test also requests your homepage as Claude-SearchBot and Claude-User, so you can see they still get through.

## Check a ClaudeBot hit against Anthropic’s IP list

Anthropic publishes its crawler IP addresses at `claude.com/crawling/bots.json` and says a crawler whose address is on that list is coming from Anthropic. A request calling itself ClaudeBot from any other address is unverified. Anthropic still recommends robots.txt for opting out, and warns that blocking by IP may not work correctly or guarantee an opt-out.

Our copy, downloaded on 28 October 2026, carried a creation date of 20 October 2026 and held 14 entries: one range, `216.73.216.0/22` (216.73.216.0 to 216.73.219.255), and 13 single addresses. A test script we wrote matched two addresses and rejected one just outside the range:

```
$ python3 verify_crawler_ip.py claude-bots.json 216.73.217.4 216.73.220.1 34.162.230.222
list claude-bots.json: 14 prefixes, creationTime 2026-10-20T00:00:00Z
216.73.217.4: MATCH 216.73.216.0/22
216.73.220.1: no match (treat as unverified)
34.162.230.222: MATCH 34.162.230.222/32
```

Lists change, so download a fresh copy each time rather than reusing the ranges shown here. Our plugin doesn’t keep visitor IPs, so take the address from your host’s access log; our guide to tracking AI crawlers shows where to find it. The script itself, a PHP version for WordPress, and how to read a visitor’s IP behind Cloudflare are in our [crawler IP verification guide](https://skybootstrap.com/verify-gptbot-ip/).

## Set Anthropic’s bots in one pass

Set Anthropic’s three bots in one pass by deciding on the training bot first. Keep Claude-SearchBot and Claude-User unless you want out of Claude altogether, and check hits against bots.json a week later. Google-Extended works differently from all three because it’s a token, not a crawler. Read [how Google-Extended works](https://skybootstrap.com/google-extended/) before treating it like these three.

For Perplexity, Meta, ByteDance, and the rest, the [WordPress AI crawler hub](https://skybootstrap.com/ai-crawlers-wordpress/) sets each company’s bots side by side.

## Frequently Asked Questions

### Are Claude-Web and anthropic-ai still Anthropic’s crawler names?

Not on Anthropic’s current crawler page. It lists three bots: ClaudeBot, Claude-SearchBot, and Claude-User. Some older templates block Claude-Web or anthropic-ai, and a third-party guide describes those as deprecated. If your robots.txt only names the old tokens, add a group for ClaudeBot, which is the training crawler Anthropic documents today.

### Will blocking ClaudeBot affect my Google rankings?

No direct link exists between them. ClaudeBot is Anthropic’s crawler, and Google crawls for Search with its own bots, mainly Googlebot. A robots.txt group only applies to the bot it names. What you give up is your content’s use in Anthropic’s training, not anything in Google.

### Who do I contact if Anthropic’s crawler misbehaves?

Anthropic, by email. The address is on [Anthropic’s crawler page](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler), which asks you to write from an email address at the domain you’re asking about. The same page offers a form to be told about substantial changes to its crawlers. Check the requests against bots.json first, because a misbehaving “ClaudeBot” may not be Anthropic at all.

### How often does ClaudeBot visit a WordPress site?

Visit frequency differs from site to site. On tlinky.com, requests calling themselves ClaudeBot reached 252 on 100 pages in about one week, and Claude-User 404 on 135 pages. Our plugin counted those by user agent only, so some may be fakes. Cloudflare reported ClaudeBot’s share of AI and search crawler traffic, Googlebot and Bingbot included, rising from 6% in July 2024 to 9.9% in July 2025.

## How we tested

- robots.txt: a WordPress file with a ClaudeBot group (Crawl-delay 10, /wp-admin/ disallowed) run through Protego 0.7.0 on 28 October 2026. The parser shows how the file reads, not what Anthropic does.
- IP check: Anthropic’s bots.json downloaded on 28 October 2026 (dated 20 October 2026, 14 entries) and three addresses tested with our verify_crawler_ip.py script.
- Live data: Skyboot AI Markdown’s AI Analytics on tlinky.com (plugin version 3.0.0), the seven or so days before 28 October 2026. Claude rows are counted by user agent only, because the plugin doesn’t check Anthropic’s list yet.
- What Anthropic does with each bot is taken from its own crawler page; our tests can’t show that.