homepage Welcome to WebmasterWorld Guest from 54.196.197.153
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Google-HTTP-Java-Client/1.17.0-rc (gzip)
Anyone seeing this?
physics




msg:4647623
 10:56 pm on Feb 21, 2014 (gmt 0)

Wondering if it's possibly mobile-browsing related, or just a bot.

Hits,Approved,TLD,Host,IP,UA
9641,0,amazonaws.com,ec2-54-241-198-78.us-west-1.compute.amazonaws.com,54.241.198.78,Google-HTTP-Java-Client/1.17.0-rc (gzip)

 

incrediBILL




msg:4647687
 5:14 am on Feb 22, 2014 (gmt 0)

It's a library Google wrote:
https://code.google.com/p/google-http-java-client/

If it's used on AWS, it's a scraper most likely but it's always possible it's Kindle related which is hard to tell as nobody over there using AWS seems bright enough to alter the default user agents to tell us why they're crawling so by that fact alone, and the sheer volume of AWS garbage (we have multiple threads on the topic) most of us just block AWS in it's entirety and leave it at that.

Angonasec




msg:4647689
 5:26 am on Feb 22, 2014 (gmt 0)

We block all 54.

physics




msg:4648324
 4:17 pm on Feb 23, 2014 (gmt 0)

Wow, you block the entire a class?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved