Block bots and crawlers with .htaccess and robots.txt
If you want to block search engine and crawler bots from visiting your pages you can do so by uploading the following files to the / directory on your site. Double check you aren’t rewriting over any .htaccess file before you do it.
(.htaccess link doesn’t work because it’s a no read file. So I have uploaded them as a zip)