PerplexityBot

Operator

Perplexity

User agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

Robots token

PerplexityBot

Obeys robots.txt

Yes, per the operator

Verification

Published IP ranges. Perplexity publishes CIDR ranges at https://www.perplexity.com/perplexitybot.json and a separate file for Perplexity-User. No reverse DNS hostname is documented.

Operator docs

https://docs.perplexity.ai/guides/bots

Perplexity documents two agents with nearly identical strings and very different behavior.

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)

Tokens: PerplexityBot and Perplexity-User.

What each one feeds

PerplexityBot is the index. Pages it crawls can appear as cited sources, with a link, in Perplexity answers. That is a referral surface, and it is the reason Perplexity's own documentation recommends allowing it.

Perplexity-User is a live fetch. Someone asked a question, answering it needs your page, and the fetcher goes and gets it. Perplexity states that this agent generally ignores robots.txt, because the request originated with a person rather than with a crawl schedule. Whether you accept that reasoning is a policy question, but it is documented rather than hidden, which is more than several operators manage.

Perplexity states that neither bot collects content for training foundation models. That is a narrower claim than it looks, and it is worth reading precisely: it is about training, not about retrieval or display.

Does it honor robots.txt

PerplexityBot does. Perplexity-User does not, by design and by published statement. If you need the second one stopped, robots.txt is the wrong tool and an edge rule on the published ranges is the right one.

How to verify a request

Match against the published JSON:

https://www.perplexity.com/perplexitybot.json
https://www.perplexity.com/perplexity-user.json

Both return CIDR prefixes with a creationTime. Perplexity's documentation says to always use the current lists rather than a copy, and the file contents do change. No reverse DNS hostname is documented, so unlike Googlebot there is no DNS fallback when the file is unreachable.

Allow it

User-agent: PerplexityBot
Allow: /

Block it

User-agent: PerplexityBot
Disallow: /

Blocking Perplexity-User in robots.txt is not effective, because the operator has said it ignores those rules. Enforce it at the edge:

# nginx, matched against the published ranges rather than the string
if ($http_user_agent ~* "Perplexity-User") { return 403; }

A user agent test alone is weak, since the string is trivially copied. Pair it with the range file if the traffic volume justifies the work.

What blocking costs you

Citations, and the clicks attached to them. Perplexity links its sources, so unlike a training crawl there is a visible referral path. Traffic volume is modest next to Google for most sites, and the queries that reach you tend to be specific.

If crawl volume rather than policy is the problem, look at the response side first. A crawler retrying through 429 and 503 responses generates far more requests than one getting clean answers, and slow time to first byte makes every crawler look heavier than it is. Compare with GPTBot, where the training and retrieval split works the same way under different names.

Find it in your logs

awk '$0 ~ /Perplexity/ {print $1, $12}' access.log | sort | uniq -c | sort -rn | head

Printing the agent field alongside the address separates PerplexityBot from Perplexity-User, which matters because only one of them reads your robots.txt. Steady low-rate traffic across many URLs is the crawler. Bursts of one or two URLs following a spike in interest are the fetcher, arriving because people are asking about you.

The dispute worth knowing about

Perplexity's crawling has been publicly contested, with infrastructure providers reporting fetches from addresses outside the published ranges and with agent strings that do not identify the operator. Perplexity's own documentation describes the two agents above and the ranges they use. Those two pictures do not match, and this page is not in a position to settle it.

What follows from that is practical. Treat the published range files as the definition of verified Perplexity traffic and nothing else. If you decide to block, an edge rule keyed on the ranges is enforcement; a robots.txt Disallow is a request that only the compliant half was ever going to read. The same reasoning applies anywhere verification is available, including Amazonbot, and it is unavailable entirely for Bytespider.

Serving the crawler well

If you want the citations, the useful work is not in robots.txt. It is making sure the answer to the question is in the HTML your server returns, with a canonical URL that points at the version you want cited, and without a soft 404 sitting where a real page should be.

Check your site against PerplexityBot