homepage Welcome to WebmasterWorld Guest from 54.145.183.126
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
"\"Mozilla/5.0"
keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4440260 posted 11:54 pm on Apr 12, 2012 (gmt 0)



Hits robots.txt (only) 12 to 15 times every day from 1.202.218.8

Chinanet
1.202.0.0 - 1.203.255.255
1.202.0.0/15

 

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4440260 posted 12:38 am on Apr 13, 2012 (gmt 0)

keyplr are you growing senile ;)

There are multiple threads on this thing, including one in the Apache forum, where the "then noob" wished to convert them to catholicism.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4440260 posted 1:51 am on Apr 13, 2012 (gmt 0)


Senile... possibly :)

However, the trusty WW search utility did not return any results with that exact UA and I never read the Apache forum.

Anyway, since you seem well informed, prey tell.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4440260 posted 2:00 am on Apr 13, 2012 (gmt 0)

"1.202." [google.com]

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4440260 posted 2:15 am on Apr 13, 2012 (gmt 0)

"1.202." [google.com]


That search is for the IP, not the UA. However, besides my own post, I just see people attesting they block Chinanet. Well so do I... and all of China for that matter. I already knew I blocked it - LOL.

My OP was just to document the UA. I know where it comes from. At least I know where the WHOIS says it comes from.

Point is, China has thousands of server farms, colos, dedi servers, etc just like the rest of the world, except they all get funneled into several China ISPs similar to the way parts of Russia are. Theoretically, IPv6 will help us to identify in a more specific way, but that remains to be seen. If all goes as planned, I will probably take down my wide blocks and be more surgical.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4440260 posted 3:21 am on Apr 13, 2012 (gmt 0)

Theoretically, IPv6 will help us to identify in a more specific way, but that remains to be seen.


I hope so.

It would sure be nice to be able to break down some the large US provider ranges that are not available via sub-nets.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4440260 posted 9:13 am on Apr 13, 2012 (gmt 0)

Funny you should say that. I wasted a lot of time this evening trying to make sense of the monstrosity that is 38. The blasted thing's got humans and robots so closely intertwined there's no separating them, and random bits of Canada subleasing from what should be a US range, and the whole thing is chopped into such teeny little pieces you might as well be in APNIC, and...

Oh, and UAs with escaped quotation marks drive me bonkers because my log-wrangling function can't deal with them so the whole thing gets left out in the cold.

iamzippy

5+ Year Member



 
Msg#: 4440260 posted 10:11 am on Apr 13, 2012 (gmt 0)

... "-" "\"Mozilla/5.0" "-" has been showing here here since 10/Feb/2012. It seems to have settled into an orbit of 90-180 minutes.

As for the escaped quote, I've also found fully-quoted Opera and even a PHP tag with a quoted system call argument. Turns out my log-wrangler had been choking for a while, too.

So I figured out a regex to capture even escaped quotes within the UA field delimiters. I'll be happy to share...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved