Amzn-User
Crawler operated by Amazon. This crawler fetches one specific page because a user just asked for it. Traffic is low and deliberate; blocking it means declining to answer someone who is looking for you.
- Operator
- Amazon
- Purpose
- User request
- Honours robots.txt
- yes
- User agent string
- Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amzn-User/0.1) Chrome/120.0.0.0 Safari/537.36
Allow it
Goes in the robots.txt served at the root of your domain. robots.txt groups do not stack: a crawler applies only the most specific group naming it, so a broader, more permissive rule will not be added to this one.
Allowing it costs a few requests a day, every one of them triggered by a person who was looking for you. It is the most intentional traffic a crawler can produce.
User-agent: Amzn-User Allow: /
Refuse it
The refusal takes effect immediately for crawlers that honour robots.txt, and is not retroactive: whatever has already been collected stays collected.
Refusing it tells that person the page cannot be reached. They will not know it was your decision.
User-agent: Amzn-User Disallow: /
How to check its visits on your side
Look for the user agent above in your server logs: it is the only proof a crawler actually came. A robots.txt directive states what you ask for, not what happens.
The other crawlers run by Amazon
-
Amazonbot
Collects content to train models.
-
Amzn-SearchBot
Feeds cited answers.
Other crawlers with the same purpose
A decision taken here is rarely taken alone: these crawlers fall under the same call.