How to Exclude AI Crawlers from your Squarespace Website
Stop AI companies from using your website to train their models
You may be familiar with good old website crawlers, mostly used by search engines to index website pages. What you may not know is that you can actually ask search engines and other companies to never index some or even all pages of your website - this way when a website crawler program loads your website, it will know to move along and ignore it. The vast majority of website owners don’t exclude crawlers from indexing their website, as search is a major source of traffic for pretty much any website out there.
With the rapid development of AI technology, we now have a whole new class of crawlers to deal with - AI crawlers. These are programs designed to scrape web pages for content, which AI companies then use to train their language models and enhance AI accuracy. But what if for some reason you didn’t want those companies to use content on your website to teach their models?
If you're using Squarespace for your website, you have the option to conveniently block any AI web crawlers from scraping content from your website with just a few clicks!
Here's how you need to do:
Enter the website editor mode and open the crawlers menu:
Either use the admin search tool to search for "crawlers"
Or navigate to Settings -> Website -> Crawlers
Modify the AI Crawler Settings:
At the bottom you will find the "Artificial Intelligence Crawlers" setting.
Turning this setting off instructs AI web crawlers not to scan or index your site, effectively preventing your content from being used for AI training.
Why would you want to disable AI crawlers on Squarespace?
You might simply not be keen on sharing your content with AI companies. Maybe you have a page on your website that you want to only share with a limited number of people, but don’t want to password protect it. Then you can disable both AI and traditional crawlers (using the same menu option we described above).
It’s important to know that this is not a 100% guarantee that your content won’t be crawled by AI bots. In essence what you can do here is just instruct those bots to ignore your website altogether - but it is up to AI companies to develop programs that honor such requests. And there is no guarantee that they 100% will.
You might also be concerned with how this might affect your site's visibility in search results. Don’t worry - as of writing of this article, changing this setting will not prevent AI search tools from showing your site in search results.
By disabling this setting, you can stop AI systems from using your content to “learn”, while still keeping your site visible in search results powered by AI.
