homepage Welcome to WebmasterWorld Guest from 54.196.62.23
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Naverbot seems to have changed
Possibel variant Taking big bites Now
carfac

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 1711 posted 4:40 pm on Mar 2, 2003 (gmt 0)

Seems to grab every second or two... nice to do it at night, but I think 1400 hits in a little over an hour means you ain't coming back!

218.145.25.112 - - [01/Mar/2003:01:36:34 -0700] "GET /my/page.cgi?film=2636 HTTP/1.0" 200 13438 "-" "dloader(NaverRobot)/1.0"

Seems to be a variant of the Naverbot from Japan/Korea... I have not see it with the dloader in front of the name.

I went ahead and banned dloader and naverbot seperately, and I banned the IP for good measure!

As jdMorgan says, EAT MY 403's!

dave

 

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 1711 posted 5:53 am on Mar 10, 2003 (gmt 0)

Does that thing honor robots.txt?

Finder

10+ Year Member



 
Msg#: 1711 posted 1:29 am on Mar 12, 2003 (gmt 0)

It always has for me.

nativenewyorker

10+ Year Member



 
Msg#: 1711 posted 5:54 pm on Mar 12, 2003 (gmt 0)

carfac,

I had requests from 218.145.25.11 and 61.78.61.168 for dloader(NaverRobot)/1.0 this morning.

Ted

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 1711 posted 7:11 pm on Mar 12, 2003 (gmt 0)

<snip>from 218.145.25.11 and 61.78.61.168 for dloader</snip>

Both Korean IP's paid me a visit Monday AM EST. Although they received denies "403" on a double whammy. One for dloader and the other for their IP.

Perhaps I'm just more fortuante than most as I have nothing to gain and nothing to lose from far east traffic. As a result I have almost all of it denied.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved