CMS Crawler

[keyplyr]

CMS Crawler identifies which publishing solutions that different sites use. Every day we visit hundreds of thousands of websites and try to determine which tool is used. We are looking for everything from simple file-based systems to the largest ECM, CMS and e-commerce solutions.

rDNS: sdt.infobyte.se
Infobyte, Sweden
83.233.186.24 - 83.233.186.31
83.233.186.24/29

Came in as:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3
and requested robots.txt and another top level page.

Then switched UAs to:
Mozilla/4.0 (CMS Crawler: http://www.cmscrawler.com)


Old Thread: [webmasterworld.com...]

[dstiles]

IP is part of the Swedish Breband group. I have the range 83.233.16 blocked with the comment... "ignore name! seems to be broadband but has loads of server-type rDNS and advertises web hosting etc"

I block al Firefox 1 and 2 and am currently considering blocking 3 - I've left that for the moment in case some older ubuntu services are still using it, but I think it's about to become history.

Thanks for the heads-up on cmscrawler - hadn't got that one.

[Pfui]

I block al Firefox 1 and 2 and am currently considering blocking 3

RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*Gecko/200 [NC,OR]

Works for me:)

[keyplyr]

Is this a valid Firefox build?

Mozilla/5.0 (Windows NT 5.1; rv:10.0) Gecko/20100101 Firefox/10.0

[wilderness]

None is the current version.

The new Firefox 10 Beta [mozilla.org]

[keyplyr]

None is the current version.

huh?

[wilderness]

Nine.

Common typo as the "o" is the next key right of "i".

[keyplyr]

Oh... yeah, I'm aware of that - I'm using it :)

[keyplyr]

Well this:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*Gecko/200 [NC,OR]

Blocks *some* Firefox/3 builds.

Example: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0

Quote from a Linux user friend:

if you block Firefox 3 you _will_ be blocking many Linux users. I run a "bleeding edge" testing release of Debian. Debian stable is still based on Firefox 3 and probably will be for at least another year. The current stable versions of Ubuntu is also based on Firefox 3, and the next stable version is not expected for several months. Once it comes out, companies will typically wait 1-6 months to upgrade. There are linux users who value stability over new features. Firefox 4 is still less than a year old. Generally, stable releases of Linux will persist 3-5 years.

[wilderness]

if you block Firefox 3 you _will_ be blocking many Linux users.

Wonder how they really feel you block ALL-Linux ;)

[lucy24]

Psst! g1! Non-final .* at large!

:)

Do there exist any UAs that contain the string "Gecko/200" but do not begin with "Mozilla"?

[wilderness]

Do there exist any UAs that contain the string "Gecko/200" but do not begin with "Mozilla"?

I've 685 references to "Gecko", certainly too many to go through.
Looked at about 50 and they ALL began with Mozilla.

[lucy24]

:: detour for some spot-checking of my own ::

I don't see any human UAs that don't start with "Mozilla" except notably Opera. And of course some mobiles. Even Linux normally starts with "Mozilla".

"UA contains 'Gecko/200' but does not contain 'Linux'" is doable.

Google Preview contains "Linux" and "Gecko"-- if anyone cares-- but its gecko is unnumbered.

[Pfui]

Further veering off the thread's topic but wanting to double-check --

Mozilla/5.0 (Windows NT 5.1; rv:10.0) Gecko/20100101 Firefox/10.0

Are y'all sure 2010-dated Ff 10.x UAs are NOT legit? Like that one, plus the following?

Mozilla/5.0 (Windows NT 6.0; WOW64; rv:10.0) Gecko/20100101 Firefox/10.0
Mozilla/5.0 (Windows NT 6.0; rv:10.0) Gecko/20100101 Firefox/10.0
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20100101 Firefox/10.0
Mozilla/5.0 (Windows NT 6.1; rv:10.0) Gecko/20100101 Firefox/10.0

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0) Gecko/20100101 Firefox/10.0

(Total hits through this month for ALL of the above are under 100 so that's iffy as-is.)

[Staffa]

I'm pretty sure 2010 for v.10 is too early.

I use Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13

At 12/03 v. was 3.n and not yet 10

[Staffa]

Update
It seems that Gecko/date has nothing to do with the Ff version.

I just had a genuine visitor with UA

Mozilla/5.0 (Windows NT 6.0; rv:9.0.1) Gecko/20100101 Firefox/9.0.1

While the Gecko date is older than Ff/3.n the rv:9.n.n is the Gecko corresponding to the Ff/9 according to the Mozilla site. Firefox 9 was released for Windows on December 20, 2011.

My Ff 3.n has rv:1.9 which also corresponds to the Ff version according to the same source.

"Firefox 10, based on Gecko 10.0, will ship in early 2012" (Mozilla 25/01/2012)

[keyplyr]

"Firefox 10, based on Gecko 10.0, will ship in early 2012" (Mozilla 25/01/2012)

Except you can get Firefox 10 beta right now, and for the last couple months.

[dstiles]

As you say, FF 10 is in Beta and some people are using it.

I was using FF 3.6 on Ubuntu 10.04 Lucid until about five months ago, when I added a line to the repository to get the latest. It updated in rapid order and I'm now using 9.0.1. But the official Ubuntu release is still 3.6 (as far as I know). Stupid, since at least two of the upgrades were for serious security reasons.

I feel the Ubuntu progenitors are not paying attention: they've screwed up a few things in the past coule of years. But... FF 3.6 is still valid for some linux machines. On Windows - it's way out of date and should be suspect.

[keyplyr]

I just upgraded to 10

[wilderness]

I just upgrade from 3.6 to 7 (I hate changes in software (have to many crucial software's) and the possibility of instability of my machines)