homepage Welcome to WebmasterWorld Guest from 54.211.138.180
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Naverbot seems to have changed
Possibel variant Taking big bites Now
carfac




msg:404253
 4:40 pm on Mar 2, 2003 (gmt 0)

Seems to grab every second or two... nice to do it at night, but I think 1400 hits in a little over an hour means you ain't coming back!

218.145.25.112 - - [01/Mar/2003:01:36:34 -0700] "GET /my/page.cgi?film=2636 HTTP/1.0" 200 13438 "-" "dloader(NaverRobot)/1.0"

Seems to be a variant of the Naverbot from Japan/Korea... I have not see it with the dloader in front of the name.

I went ahead and banned dloader and naverbot seperately, and I banned the IP for good measure!

As jdMorgan says, EAT MY 403's!

dave

 

Brett_Tabke




msg:404254
 5:53 am on Mar 10, 2003 (gmt 0)

Does that thing honor robots.txt?

Finder




msg:404255
 1:29 am on Mar 12, 2003 (gmt 0)

It always has for me.

nativenewyorker




msg:404256
 5:54 pm on Mar 12, 2003 (gmt 0)

carfac,

I had requests from 218.145.25.11 and 61.78.61.168 for dloader(NaverRobot)/1.0 this morning.

Ted

wilderness




msg:404257
 7:11 pm on Mar 12, 2003 (gmt 0)

<snip>from 218.145.25.11 and 61.78.61.168 for dloader</snip>

Both Korean IP's paid me a visit Monday AM EST. Although they received denies "403" on a double whammy. One for dloader and the other for their IP.

Perhaps I'm just more fortuante than most as I have nothing to gain and nothing to lose from far east traffic. As a result I have almost all of it denied.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved