Welcome to WebmasterWorld Guest from 18.104.22.168
Is it safe to assume that any 'legitimate' bot would have a user agent string?
If so, is there a way (via httpd.conf) to deny access to bots that are missing their user agent string?
I have found a few sites that can check individualy, but not in bulk.
One interesting thing - Yahoo Japan's bot has been visiting with no user agent..Almost banned them!
Yahoo's 'ycar' hosts make requests with no UA, but we haven't figured out what 'ycar' is yet... It's still an open question over in our spider and user-agent ID forum.