Hide Your Website From Search Engines: A Complete Guide to Search Crawler Blocking

Hide Your Website From Search Engines: A Complete Guide to Search Crawler Blocking

By Michael Thompson

January 20, 2025 at 12:34 PM

Web crawlers index publicly accessible content to display in search results. Here's how to control your site's visibility:

Block All Search Engine Access

To prevent your entire site from appearing in search results:

  1. Go to Settings
  2. Select Crawler
  3. Check "Block search engine crawlers"

This adds a blocking rule to your robots.txt file while keeping the site accessible to direct visitors.

Hide Specific Pages

You can exclude individual pages using two methods:

Page Settings Method

  • Open Pages panel
  • Click the page
  • Go to SEO tab
  • Toggle "Hide page from search results"

Benefits:

  • Available on all plans
  • No coding needed
  • Removes page from sitemap
  • Supports demo pages

Code Injection Method

  • Add noindex tag to page header
  • Requires manual code implementation
  • Page stays in sitemap
  • Available for homepage

Important Notes:

  • Collection items/portfolio subpages can only be hidden by excluding the entire collection
  • Index page tags don't affect subpages - add tags to both
  • Tags only block external search engines, not site search
  • Blog pages have additional visibility options

For AI crawler exclusion or other access controls, adjust those settings separately in your site configuration.

Remember that changes can take time to reflect in search results after implementation. Regularly verify your visibility settings match your intended site access.

Related Articles

Previous Articles