Page is a not externally linkable
madmatt69 - 3:47 pm on Jan 21, 2010 (gmt 0)
Is it safe to assume that any 'legitimate' bot would have a user agent string? If so, is there a way (via httpd.conf) to deny access to bots that are missing their user agent string? Thanks!
Hi all recently my site stats show a bot that just sucked up over a gig of bandwidth. In my 'awstats' report it just defines it as:
Unknown robot (identified by empty user agent string)