Is there a way to combat this junk? - Crawler, Spider, and User Agent ID forum at WebmasterWorld - WebmasterWorld

Forum Moderators: open

Message Too Old, No Replies

Is there a way to combat this junk?

Mozilla/4.0 (compatible ; MSIE 6.0; Windows NT 5.1)

SEOPTI

1:13 am on Jan 22, 2007 (gmt 0)

WebmasterWorld Senior Member

10+ Year Member

This is hitting my sites like crazy with different IPs and steals hundreds of pages. Each time I see it, I block it with a 403.

wilderness

2:00 am on Jan 22, 2007 (gmt 0)

WebmasterWorld Senior Member

10+ Year Member

Top Contributors Of The Month

Jim's tranq work great, however this line will solve your problem

SetEnvIf User-Agent "compatible ;" keep_out