Google Spider

[Admiral_Ron]

I am not sure if I am in the right section or not....

I was wondering if anyone would know of a way to block inbound search engines? I have a web site I run out of my house, just for family, but it is getting tons of "Google Spider" and "Websense" hits. Does anyone know if you could block these type of inbound conneciton without hindering port 80

[dmorris]

You can do so with a "robots.txt" on your web server.
Just do a google search and you can read about the parameters.

Also you can just block those IPs with the firewall if you don't want to touch your webserver, but they have bots all over the place...

[Admiral_Ron]

Thank you for the info....

[sky-knight]

Too bad it doesn't work....

Robots.txt is ignored by Google most of the time and a lot of the time by MSN. Unfortunately unless you want to code your page to deny access from the browser they present in the host header all you can do is firewall the IP's off.