View Full Version : How to prevent robots from accessing my site
classifiedsecrets
12-11-2006, 03:48 PM
I want to use .htaccess to prevent robots from accessing my site.
I tried using
BrowserMatchNoCase (emailsiphon|LinksWalker) goaway
deny from goaway
I get a 500 error.
how do I do it?
Thanks much in advance for your assistance.
sjlplat
12-11-2006, 07:22 PM
This might help you achieve what you're looking to do:
http://www.javascriptkit.com/howto/htaccess13.shtml
enoobz
12-12-2006, 10:36 PM
Just use this http://www.seochat.com/seo-tools/robots-generator/
Thanks.
classifiedsecrets
12-24-2006, 02:48 PM
Thanks very much for the information.
welcomesite
12-28-2006, 05:22 AM
I just created a blank robots.txt file, so I don't get any 'file not found' messages in my stats.
I don't want to mess with the robots.
I'm #1 on Google now for certain searches.
Powered by vBulletin® Version 4.1.10 Copyright © 2012 vBulletin Solutions, Inc. All rights reserved.