AhrefsBot
Ahrefs's scraper.
- Operator
- Ahrefs
- Type
- Scraper
- In the user agent
AhrefsBot- robots.txt token
AhrefsBot- Follows robots.txt
- says it follows robots.txt
- Official documentation
- ahrefs.com
- Checked on
- 2026-11-11
What is AhrefsBot?
AhrefsBot is Ahrefs's scraper. It collects data for a product that is not an AI assistant.
The operator's description: Builds the link and content database behind Ahrefs and the Yep search engine.
(ahrefs.com)
How do I block or allow AhrefsBot?
With robots.txt, using the token AhrefsBot. Ahrefs says it follows robots.txt. To block the whole site:
User-agent: AhrefsBot
Disallow: /To allow it, use Allow instead of Disallow. Before blocking, weigh what it brings: look at how often it visits and what it brings before deciding.
How do I see whether AhrefsBot visits my site?
In your server or CDN access logs, look for AhrefsBot in the user agent. Google Analytics does not show these visits: agents do not run its JavaScript.
Anyone can copy the name in a user agent, so counting log lines is not enough. Knocklog reads your site's traffic on any host, confirms who is who, and shows how often AhrefsBot came, which pages it read and what that brought you.
How do I know it is really AhrefsBot?
Ahrefs publishes AhrefsBot's IP addresses at https://api.ahrefs.com/v3/public/crawler-ip-ranges. A visit that claims to be AhrefsBot from outside those ranges is spoofed.
The IP's reverse DNS, confirmed forward and back, must end in .ahrefs.com or .ahrefs.net.
Knocklog runs this check on every visit and separates the genuine agent from impostors.
Does AhrefsBot visit your site?
See in seconds how your site treats AI agents. Then connect your traffic to learn who really came.
Other Ahrefs agents
- AhrefsSiteAudit Scraper