Msg#: 3114262 posted 4:35 pm on Oct 9, 2006 (gmt 0)
What would the best way to block both of these agents? They just sweep through the site ignoring robots.txt, and i have been adding their ip address to be blocked. I want to deny all user agents in .htaccess, so would the user agent Java catch them all or do I have to be specific for each agent? An example would be?