homepage Welcome to WebmasterWorld Guest from 54.166.84.82
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

    
Unidentified Bots
Any reason not to ban them?
carfac

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3727460 posted 6:17 pm on Aug 20, 2008 (gmt 0)

Hi:

I see a lot of "whatevers" (bots probably, real humans, I doubt) reading my robots.txt... with no user agent.

Any reason not to just write a new rule in and just instantly ban anyone who reads robots.txt with no UA?

 

carfac

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3727460 posted 7:49 pm on Aug 20, 2008 (gmt 0)

Sorry- wrong area. Will repost in the correct forum.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved