A setting called “Search engine blocking” is available in the System > System Features > Features tab which, when enabled, will add a "robots" meta tag to your site's page headers to prevent them from being scanned and indexed by most search engines. This effectively prevents your site from being found by the public without the direct URL. This will be most useful for implementing clients whose sites are not yet ready to be accessed by the public and for clients who use a non-Terra Dotta site for their public website. This setting is disabled by default.
These five options are available from the drop-down menu:
- NOINDEX: Disallow search engines from showing the page in their results.
- NOFOLLOW: Tells the search engine robots to not follow any links on the page.
- NOINDEX,NOFOLLOW: Disallows search engines from showing the page in their results and following any links from the page.
- INDEX,NOFOLLOW: Allows search engines to show the page in their results but disallows them from following any links from the page.
- NOINDEX,FOLLOW: Tells the search engine robots to follow the links on the page but does not index the page.