If you have spotted “PerplexityBot” in your server logs, you may be wondering what it is, whether you should allow it, and how it affects your chances of being cited by Perplexity. This guide explains Perplexity’s crawlers in plain language, shows you how to control access, and helps you decide on the right setup for your site.
What Is PerplexityBot?
PerplexityBot is the web crawler Perplexity uses to discover and index pages so they can be surfaced and linked in Perplexity’s search results and answers. Perplexity has stated that PerplexityBot is not used to crawl content for training AI foundation models. Its role is to support search and citation.
When PerplexityBot can access your pages, they can become part of the pool of sources Perplexity draws on when answering questions. When it is blocked, your chances of being cited drop significantly.
PerplexityBot vs Perplexity-User
Perplexity documents two main user agents, and they behave differently:
| User agent | Purpose | Typical trigger |
|---|---|---|
| PerplexityBot | Crawls and indexes pages for Perplexity search results | Automated crawling |
| Perplexity-User | Fetches a page to answer a specific user request | A user asks a question that needs that page |
The distinction matters for robots.txt. PerplexityBot follows robots.txt rules. Perplexity has said that Perplexity-User, because it acts on behalf of a user’s request, generally does not follow robots.txt in the same way. If you need to restrict user-triggered fetches, you may need firewall or bot management controls rather than robots.txt alone.
Why Crawler Access Matters for Citations
Perplexity is a citation-first answer engine. Every answer is grounded in sources, and those sources are displayed prominently. That makes crawl access a foundational requirement for AI search visibility on this platform. Great content that Perplexity cannot reach is invisible to it.
For many brands, Perplexity referrals are highly engaged visitors who click through to verify information or dig deeper. Blocking the crawler means giving up that traffic and handing citations to competitors.
How to Allow PerplexityBot
To explicitly allow PerplexityBot across your site, add this to your robots.txt:
User-agent: PerplexityBot
Allow: /To allow it while keeping private sections out:
User-agent: PerplexityBot
Allow: /
Disallow: /admin/
Disallow: /cart/How to Block PerplexityBot
If you decide not to appear in Perplexity, you can block the crawler:
User-agent: PerplexityBot
Disallow: /Remember that this affects indexing by PerplexityBot. It may not stop every user-triggered fetch. If your concern is protecting premium content, combine robots.txt with paywalls, authentication, or bot management rules at your CDN.
Check for Accidental Blocks
Many sites block Perplexity without meaning to. Review these common causes:
- Wildcard rules such as
User-agent: *followed byDisallow: / - Security plugins that block AI bots by default
- CDN bot protection that challenges unfamiliar crawlers with CAPTCHAs or blocks
- Rate limiting that returns 429 errors to frequent crawlers
- Geo-blocking that rejects requests from certain regions
Check your live robots.txt in a browser, then check your firewall logs. An AI SEO audit will flag these issues alongside other technical problems.
How to Verify PerplexityBot Visits
User agent strings are easy to fake, so do not trust log entries blindly.
Check the IP Addresses
Perplexity publishes IP address ranges for its crawlers. Compare the IPs in your logs with the official list. Requests claiming to be PerplexityBot from other addresses may be imposters.
Look at Response Codes
Filter your logs for PerplexityBot and review the status codes. You want 200 responses for important pages. Frequent 403 or 429 responses suggest your security setup is blocking or throttling the crawler.
Monitor Referral Traffic
In analytics, track sessions from perplexity.ai. If the crawler is reaching your content and your pages are useful, you should see referrals over time.
Should You Allow PerplexityBot?
Before deciding, look at your own situation honestly. Ask three questions. Do your customers use AI tools to research purchases in your category? Is your content primarily a marketing asset or the product you sell? Would you rather control how your brand is described or leave that to third-party sources? For a service firm, a SaaS company, or an online store, the answers usually point toward allowing access. For a subscription research publisher, the answers may point toward a partial block.
For most businesses focused on growth, allowing PerplexityBot is the sensible choice. It supports citations, referral traffic, and brand awareness among research-minded users. The main exceptions are publishers whose business depends on exclusive content, or sites with specific licensing restrictions. Even then, a partial block covering premium sections may work better than a full block.
Access Is Only the First Step
Allowing the crawler makes your content eligible, not guaranteed. To earn citations, your pages need clear answers, specific evidence, current information, and demonstrated expertise. Our guides to AI search citations and the AI SEO checklist explain how to turn access into visibility.
Quick Setup Checklist
- Open your live robots.txt and search for Perplexity rules
- Add an explicit allow group for PerplexityBot if you want citations
- Review CDN and security plugin settings for AI bot blocks
- Verify crawler IPs against Perplexity’s published ranges
- Track perplexity.ai referrals monthly in analytics
Frequently Asked Questions
What is PerplexityBot?
PerplexityBot is Perplexity’s web crawler. It discovers and indexes pages so they can appear and be linked in Perplexity’s search answers. Perplexity states it is not used for foundation model training.
What is the difference between PerplexityBot and Perplexity-User?
PerplexityBot crawls automatically to build Perplexity’s index. Perplexity-User fetches pages when a specific user request needs them, and Perplexity says it generally does not follow robots.txt rules.
Will blocking PerplexityBot stop Perplexity citing me?
Blocking PerplexityBot greatly reduces your chances of being indexed and cited. However, user-triggered fetches and third-party mentions may still lead to your brand appearing in some answers.
How do I allow PerplexityBot on WordPress?
Edit robots.txt through your SEO plugin or hosting file manager, add User-agent: PerplexityBot with Allow: /, save changes, then visit yourdomain.com/robots.txt to confirm it is live.
How can I verify genuine PerplexityBot traffic?
Compare the IP addresses in your server logs with the IP ranges Perplexity publishes. Requests using the PerplexityBot user agent from unlisted addresses may be spoofed and should be treated cautiously.
Does PerplexityBot slow down my website?
Normally, crawler traffic is light for most sites. If you notice heavy load, review rate limits carefully rather than blocking completely, so you keep visibility while protecting server performance.