sjlplat
07-05-2007, 01:06 AM
I've noticed some activity from a frequent, yet unknown visitor at a specific block of IP's. After a little research, I discovered the IP block in question belongs to Websense (http://www.websense.com). Websense is a web filtering software company. Evidently, they use crawlers to identify content and add it to their filters.
I really don't like the idea of having a web crawler poking around my websites, collecting my content unidentified, and then determining whether or not my content is worthy of being excluded from it's filters. So, I decided to have a little fun with it.
I've written this simple PHP script to redirect Websense crawlers to....
You guessed it - the Websense website! :eek: :D
While it's far easier to simply add a deny entry to .htaccess, this method made me feel all warm and fuzzy inside. :p
Here's the code for anyone interested:
<?php
$ip = $_SERVER['REMOTE_ADDR'];
$parts = explode('.', $ip);
$mask = $parts[count($parts) - 4] . '.' . $parts[count($parts) - 3] . '.' . $parts[count($parts) - 2];
if ($mask == '66.194.6') {
header("HTTP/1.1 301 Moved Permanently");
header("location: http://www.websense.com");
exit();
}
?>
I really don't like the idea of having a web crawler poking around my websites, collecting my content unidentified, and then determining whether or not my content is worthy of being excluded from it's filters. So, I decided to have a little fun with it.
I've written this simple PHP script to redirect Websense crawlers to....
You guessed it - the Websense website! :eek: :D
While it's far easier to simply add a deny entry to .htaccess, this method made me feel all warm and fuzzy inside. :p
Here's the code for anyone interested:
<?php
$ip = $_SERVER['REMOTE_ADDR'];
$parts = explode('.', $ip);
$mask = $parts[count($parts) - 4] . '.' . $parts[count($parts) - 3] . '.' . $parts[count($parts) - 2];
if ($mask == '66.194.6') {
header("HTTP/1.1 301 Moved Permanently");
header("location: http://www.websense.com");
exit();
}
?>