homepage Welcome to WebmasterWorld Guest from 184.73.52.98
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
"\"Mozilla/5.0"
keyplyr




msg:4440262
 11:54 pm on Apr 12, 2012 (gmt 0)



Hits robots.txt (only) 12 to 15 times every day from 1.202.218.8

Chinanet
1.202.0.0 - 1.203.255.255
1.202.0.0/15

 

wilderness




msg:4440275
 12:38 am on Apr 13, 2012 (gmt 0)

keyplr are you growing senile ;)

There are multiple threads on this thing, including one in the Apache forum, where the "then noob" wished to convert them to catholicism.

keyplyr




msg:4440298
 1:51 am on Apr 13, 2012 (gmt 0)


Senile... possibly :)

However, the trusty WW search utility did not return any results with that exact UA and I never read the Apache forum.

Anyway, since you seem well informed, prey tell.

wilderness




msg:4440300
 2:00 am on Apr 13, 2012 (gmt 0)

"1.202." [google.com]

keyplyr




msg:4440305
 2:15 am on Apr 13, 2012 (gmt 0)

"1.202." [google.com]


That search is for the IP, not the UA. However, besides my own post, I just see people attesting they block Chinanet. Well so do I... and all of China for that matter. I already knew I blocked it - LOL.

My OP was just to document the UA. I know where it comes from. At least I know where the WHOIS says it comes from.

Point is, China has thousands of server farms, colos, dedi servers, etc just like the rest of the world, except they all get funneled into several China ISPs similar to the way parts of Russia are. Theoretically, IPv6 will help us to identify in a more specific way, but that remains to be seen. If all goes as planned, I will probably take down my wide blocks and be more surgical.

wilderness




msg:4440327
 3:21 am on Apr 13, 2012 (gmt 0)

Theoretically, IPv6 will help us to identify in a more specific way, but that remains to be seen.


I hope so.

It would sure be nice to be able to break down some the large US provider ranges that are not available via sub-nets.

lucy24




msg:4440375
 9:13 am on Apr 13, 2012 (gmt 0)

Funny you should say that. I wasted a lot of time this evening trying to make sense of the monstrosity that is 38. The blasted thing's got humans and robots so closely intertwined there's no separating them, and random bits of Canada subleasing from what should be a US range, and the whole thing is chopped into such teeny little pieces you might as well be in APNIC, and...

Oh, and UAs with escaped quotation marks drive me bonkers because my log-wrangling function can't deal with them so the whole thing gets left out in the cold.

iamzippy




msg:4440390
 10:11 am on Apr 13, 2012 (gmt 0)

... "-" "\"Mozilla/5.0" "-" has been showing here here since 10/Feb/2012. It seems to have settled into an orbit of 90-180 minutes.

As for the escaped quote, I've also found fully-quoted Opera and even a PHP tag with a quoted system call argument. Turns out my log-wrangler had been choking for a while, too.

So I figured out a regex to capture even escaped quotes within the UA field delimiters. I'll be happy to share...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved