homepage Welcome to WebmasterWorld Guest from 54.161.246.212
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
CMS Crawler
keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 10:01 pm on Jan 29, 2012 (gmt 0)

CMS Crawler identifies which publishing solutions that different sites use. Every day we visit hundreds of thousands of websites and try to determine which tool is used. We are looking for everything from simple file-based systems to the largest ECM, CMS and e-commerce solutions.

rDNS: sdt.infobyte.se
Infobyte, Sweden
83.233.186.24 - 83.233.186.31
83.233.186.24/29

Came in as:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3
and requested robots.txt and another top level page.

Then switched UAs to:
Mozilla/4.0 (CMS Crawler: http://www.cmscrawler.com)


Old Thread: [webmasterworld.com...]

 

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4412172 posted 9:17 pm on Jan 30, 2012 (gmt 0)

IP is part of the Swedish Breband group. I have the range 83.233.16 blocked with the comment... "ignore name! seems to be broadband but has loads of server-type rDNS and advertises web hosting etc"

I block al Firefox 1 and 2 and am currently considering blocking 3 - I've left that for the moment in case some older ubuntu services are still using it, but I think it's about to become history.

Thanks for the heads-up on cmscrawler - hadn't got that one.

Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4412172 posted 10:14 pm on Jan 30, 2012 (gmt 0)

I block al Firefox 1 and 2 and am currently considering blocking 3

RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*Gecko/200 [NC,OR]

Works for me:)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 11:03 pm on Jan 30, 2012 (gmt 0)


Is this a valid Firefox build?

Mozilla/5.0 (Windows NT 5.1; rv:10.0) Gecko/20100101 Firefox/10.0

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 1:39 am on Jan 31, 2012 (gmt 0)

None is the current version.

The new Firefox 10 Beta [mozilla.org]

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 2:37 am on Jan 31, 2012 (gmt 0)

None is the current version.

huh?

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 3:58 am on Jan 31, 2012 (gmt 0)

Nine.

Common typo as the "o" is the next key right of "i".

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 4:31 am on Jan 31, 2012 (gmt 0)

Oh... yeah, I'm aware of that - I'm using it :)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 4:56 am on Jan 31, 2012 (gmt 0)

Well this:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*Gecko/200 [NC,OR]

Blocks *some* Firefox/3 builds.

Example: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0

Quote from a Linux user friend:
if you block Firefox 3 you _will_ be blocking many Linux users. I run a "bleeding edge" testing release of Debian. Debian stable is still based on Firefox 3 and probably will be for at least another year. The current stable versions of Ubuntu is also based on Firefox 3, and the next stable version is not expected for several months. Once it comes out, companies will typically wait 1-6 months to upgrade. There are linux users who value stability over new features. Firefox 4 is still less than a year old. Generally, stable releases of Linux will persist 3-5 years.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 5:03 am on Jan 31, 2012 (gmt 0)

if you block Firefox 3 you _will_ be blocking many Linux users.


Wonder how they really feel you block ALL-Linux ;)

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4412172 posted 5:14 am on Jan 31, 2012 (gmt 0)

Psst! g1! Non-final .* at large!

:)

Do there exist any UAs that contain the string "Gecko/200" but do not begin with "Mozilla"?

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 7:38 am on Jan 31, 2012 (gmt 0)

Do there exist any UAs that contain the string "Gecko/200" but do not begin with "Mozilla"?


I've 685 references to "Gecko", certainly too many to go through.
Looked at about 50 and they ALL began with Mozilla.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4412172 posted 9:18 am on Jan 31, 2012 (gmt 0)

:: detour for some spot-checking of my own ::

I don't see any human UAs that don't start with "Mozilla" except notably Opera. And of course some mobiles. Even Linux normally starts with "Mozilla".

"UA contains 'Gecko/200' but does not contain 'Linux'" is doable.

Google Preview contains "Linux" and "Gecko"-- if anyone cares-- but its gecko is unnumbered.

Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4412172 posted 2:39 pm on Jan 31, 2012 (gmt 0)

Further veering off the thread's topic but wanting to double-check --

Mozilla/5.0 (Windows NT 5.1; rv:10.0) Gecko/20100101 Firefox/10.0

Are y'all sure 2010-dated Ff 10.x UAs are NOT legit? Like that one, plus the following?

Mozilla/5.0 (Windows NT 6.0; WOW64; rv:10.0) Gecko/20100101 Firefox/10.0
Mozilla/5.0 (Windows NT 6.0; rv:10.0) Gecko/20100101 Firefox/10.0
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20100101 Firefox/10.0
Mozilla/5.0 (Windows NT 6.1; rv:10.0) Gecko/20100101 Firefox/10.0

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0) Gecko/20100101 Firefox/10.0

(Total hits through this month for ALL of the above are under 100 so that's iffy as-is.)

Staffa

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4412172 posted 4:09 pm on Jan 31, 2012 (gmt 0)

I'm pretty sure 2010 for v.10 is too early.

I use Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13

At 12/03 v. was 3.n and not yet 10

Staffa

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4412172 posted 6:12 pm on Jan 31, 2012 (gmt 0)

Update
It seems that Gecko/date has nothing to do with the Ff version.

I just had a genuine visitor with UA

Mozilla/5.0 (Windows NT 6.0; rv:9.0.1) Gecko/20100101 Firefox/9.0.1

While the Gecko date is older than Ff/3.n the rv:9.n.n is the Gecko corresponding to the Ff/9 according to the Mozilla site. Firefox 9 was released for Windows on December 20, 2011.

My Ff 3.n has rv:1.9 which also corresponds to the Ff version according to the same source.

"Firefox 10, based on Gecko 10.0, will ship in early 2012" (Mozilla 25/01/2012)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 7:54 pm on Jan 31, 2012 (gmt 0)


"Firefox 10, based on Gecko 10.0, will ship in early 2012" (Mozilla 25/01/2012)

Except you can get Firefox 10 beta right now, and for the last couple months.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4412172 posted 9:28 pm on Jan 31, 2012 (gmt 0)

As you say, FF 10 is in Beta and some people are using it.

I was using FF 3.6 on Ubuntu 10.04 Lucid until about five months ago, when I added a line to the repository to get the latest. It updated in rapid order and I'm now using 9.0.1. But the official Ubuntu release is still 3.6 (as far as I know). Stupid, since at least two of the upgrades were for serious security reasons.

I feel the Ubuntu progenitors are not paying attention: they've screwed up a few things in the past coule of years. But... FF 3.6 is still valid for some linux machines. On Windows - it's way out of date and should be suspect.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 10:20 pm on Jan 31, 2012 (gmt 0)

I just upgraded to 10

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4412172 posted 10:37 pm on Jan 31, 2012 (gmt 0)

I just upgrade from 3.6 to 7 (I hate changes in software (have to many crucial software's) and the possibility of instability of my machines)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved