homepage Welcome to WebmasterWorld Guest from 54.204.94.228
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Hits by Mozzilla/3.0- Not too polite
carfac




msg:405534
 3:29 pm on Sep 7, 2002 (gmt 0)

Hi:

Just got hit by Mozzilla/3.0, and it doers NOT read robots.txt.

Note the two "zz"'s

Might be a Japanese search engine, I do not know for sure. Couple hundred hits in 3 seconds, until it fell into my Spider Trap! That cut it off!

Should be blockable just by UA Mozzilla, or IP of 218.45.232.200

Here is a log entry:

218.45.232.200 - - [07/Sep/2002:01:49:51 -0600] "GET /xx/more5.html HTTP/1.0" 200 42666 "-" "Mozzilla/3.0"
218.45.232.200 - - [07/Sep/2002:01:49:51 -0600] "GET /xx/more4.html HTTP/1.0" 200 44041 "-" "Mozzilla/3.0"
218.45.232.200 - - [07/Sep/2002:01:49:51 -0600] "GET /xx/more3.html HTTP/1.0" 200 42617 "-" "Mozzilla/3.0"

Just a heads up for ya'll!

dave

 

Pushycat




msg:405535
 6:06 pm on Sep 7, 2002 (gmt 0)

Thanks for the heads up.

For those of us using IIS I added it as a "website stripper" to my browscap.ini file which will be available for download on Sunday evening as usual.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved