homepage Welcome to WebmasterWorld Guest from 54.226.43.155
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Mythic Beasts Test Netblock
keyplyr




msg:4517610
 8:55 pm on Nov 9, 2012 (gmt 0)

UA: Apache-HttpClient/4.1.3 (java 1.5)
robots.txt: no

Blocked by UA, but thought I'd post the range since it looks like a range used for computer science projects.

Mythic Beasts Test Netblock (colocated customers)Cambridge, UK
93.93.135.0 - 93.93.135.255
93.93.135.0/24

 

dstiles




msg:4517623
 9:42 pm on Nov 9, 2012 (gmt 0)

I get mail through mythic beasts - they operate legit mailing lists for (eg) genealogy.

Having said that, I blocked the range 93.93.128.0 - 93.93.135.255 in May 2010.

lucy24




msg:4517641
 10:48 pm on Nov 9, 2012 (gmt 0)

Hey, I've met Mythic Beasts. Only I've got them labeled as the full range
93.93.128.0/21
(128-135)

Notes say "93.93.130.137 = 62.200.22.2"
where the second range is /16 Citrix. Whoever they are.

:: detour to raw logs ::

Wow, long time ago. 65 total on 9 February, starting with google uk. They look human, but in retrospect they were awfully systematic. Some people do explore, but they don't explore that many different directories.

62.200.22.2 - - [09/Feb/2012:03:31:00 -0800] ... "Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"

followed a few hours later by

93.93.130.137 - - [09/Feb/2012:06:35:18 -0800] ... "Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"

Sporadic returns, every hour or so. Then a break of 18 hours -- compatible with UK time -- before continuing where they left off for another 60 pickups on 10 February.

I know Firefox has been progressing by leaps and bounds, but how long ago was 7.0.1? Other firefoxes from the same day seem to be mostly 9 and 10. Does Linux have different numbering? Is the UA some kind of laptop?

dstiles




msg:4517841
 9:06 pm on Nov 10, 2012 (gmt 0)

Current firefox is 16.02 and I think 7 was about 18 months or so ago - dumb updates are a real nuisance!

I block FF up to 13 except for 12 because Windows 2000 can't use anything higher (which means my wife!). Drastic, I admit, but a LOT of scrapers are now using FF.

wilderness




msg:4517846
 9:51 pm on Nov 10, 2012 (gmt 0)

I'm still using 7.01 and I'm not scraping ;)

dstiles




msg:4518096
 7:57 pm on Nov 11, 2012 (gmt 0)

But you are more exposed to malware. :)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved