Perplexity-User
Crawler operated by Perplexity. This crawler fetches one specific page because a user just asked for it. Traffic is low and deliberate; blocking it means declining to answer someone who is looking for you.
- Operator
- Perplexity
- Purpose
- User request
- Honours robots.txt
- no
- User agent string
- Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)
Allow it
Goes in the robots.txt served at the root of your domain. robots.txt groups do not stack: a crawler applies only the most specific group naming it, so a broader, more permissive rule will not be added to this one.
Allowing it costs a few requests a day, every one of them triggered by a person who was looking for you. It is the most intentional traffic a crawler can produce.
User-agent: Perplexity-User Allow: /
Refuse it
The refusal takes effect immediately for crawlers that honour robots.txt, and is not retroactive: whatever has already been collected stays collected.
Refusing it tells that person the page cannot be reached. They will not know it was your decision.
This crawler does not honour robots.txt: the snippet will not stop it. Refusing it requires server-side filtering, on its address or its user agent string.
User-agent: Perplexity-User Disallow: /
How to check its visits on your side
Look for the user agent above in your server logs: it is the only proof a crawler actually came. A robots.txt directive states what you ask for, not what happens.
The other crawlers run by Perplexity
-
PerplexityBot
Feeds cited answers.
Other crawlers with the same purpose
A decision taken here is rarely taken alone: these crawlers fall under the same call.