PDA

View Full Version : How to prevent robots from accessing my site



classifiedsecrets
12-11-2006, 03:48 PM
I want to use .htaccess to prevent robots from accessing my site.

I tried using

BrowserMatchNoCase (emailsiphon|LinksWalker) goaway
deny from goaway

I get a 500 error.

how do I do it?

Thanks much in advance for your assistance.

sjlplat
12-11-2006, 07:22 PM
This might help you achieve what you're looking to do:

http://www.javascriptkit.com/howto/htaccess13.shtml

enoobz
12-12-2006, 10:36 PM
Just use this http://www.seochat.com/seo-tools/robots-generator/

Thanks.

classifiedsecrets
12-24-2006, 02:48 PM
Thanks very much for the information.

welcomesite
12-28-2006, 05:22 AM
I just created a blank robots.txt file, so I don't get any 'file not found' messages in my stats.

I don't want to mess with the robots.
I'm #1 on Google now for certain searches.