homepage Welcome to WebmasterWorld Guest from 54.226.252.142
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Baidu Hong Kong
lucy24




msg:4595114
 10:16 pm on Jul 21, 2013 (gmt 0)

hot off the presses:

Started seeing this new robot a couple days ago.

185.10.104.194 - - [date] "GET /ebooks/perez/PerezEsp.html HTTP/1.0" 200 12948 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1)"

Looking at the IP you would guess it's a brand-new Ukrainian robot, and you would be half right. In fact it's Baidu Hong Kong

185.10.104.0/22

Now, it's possible they asked one of their sister Baidus for a copy of robots.txt and therefore didn't need to ask on their own behalf, but...

Naah.

If you guessed from all those directory slashes that the requested file is not directly linked from the front page (which, in any case, they didn't ask for), you would be right.

If you guessed that the content of this particular page is in the public domain, you would also be right-- but the same does not apply to the robot's subsequent requests.

Let's stick with the first guess: Shoot to kill.

:: now back to wondering what the ### the PiplBot wants with my favicon ::

 

keyplyr




msg:4595428
 6:10 pm on Jul 22, 2013 (gmt 0)



Yes new, thanks

Baidu Hong Kong
185.10.104.0 - 185.10.107.255
185.10.104.0/22

BLOCKED!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved