homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Code, Content, and Presentation / Content Management
Forum Library, Charter, Moderators: ergophobe

Content Management Forum

Please recomend a CMS for a site with over 175,000 pages

 12:13 pm on Nov 2, 2009 (gmt 0)

Hi all,

looking for recomendations on a CMS that can handle 175,000 + pages for a site.

Anyone have experience with this?




 4:52 pm on Nov 2, 2009 (gmt 0)


Pretty much any CMS can handle a large number of pages. That's just database size. Depending on the ways you serve up those pages, it might be a lower load than a site with only a few thousand pages.

For example, if you only have admins logging in and don't have "who's online" type of features, and you use a static HTML cache of the pages, this could be a very low overhead site.

Even if you're not caching, but you're basically doing simply lookups on indexed fields in the database, page retrieval could still be very fast regardless of database size.

Whereas a site with lots of features that customize per user, that show who's online, list recent comments, allow users to slice and dice data and so on, you'll have a way higher server load.

I think Wordpress.com adds about 200,000 new pages every day (see [en.wordpress.com...] Matt Mullenweg says that Wordpress can serve over 20,000,000 page views per month on a low-end dedicated server (a "$100/mo server").

Drupal numbers it's pages sequentially if you aren't using friendly URLs, and on their site, a recent page was #621,132


 5:13 pm on Nov 2, 2009 (gmt 0)

thanks for the insight ..

much appreciated.

So in theory all of these CMS should be able to manage a LARGE site.

No whos online features are required, its a straight forward promotional site, with lots of quality content. Its all dynamically created, a complete mess with up to 50+ variables in some urls !

Caching would have to be a must to lower server loads (site is on a couple of servers sharing the load at the moment.

Out of interest, what would you recomend I go investigate first, obviously I will have to look at a lot of cms´s but whats your gut feeling?

Where to start the search?


 6:44 pm on Nov 2, 2009 (gmt 0)

You know, Dries, the founder and creator of Drupal has what he calls "The Ockham's Razor Principle of Content Management Systems" which goes like this:

Given two functionally equivalent content management systems, the simplest one should be selected.

Source [buytaert.net]

I think that's a smart way to look at it. So first you need to define your requirements.

1. A lot of pages. [I think that cuts out few to zero CMS]

2. ?

Have a look at CMSMatrix.org and OpensourceCMS.com to see what's out there and try out some demos.

Following Dries advice, you would use the simplest CMS that meets your needs.

The question I would ask is if you have 50 variables in the URL, how will this integrate with any CMS? It sounds like you already have one. Somehow you'll need to get that data from one to the other.

So I'm not really sure what you're after. And just so we're clear here, I've never run anything with near that many pages. My biggest is about 12,000 pages, with the equivalent of about 20,000 page of printed material. I run that on system I built myself because

1. I had very specific search and data entry requirements

2. I built it long enough ago that I found all the open source CMS systems lacking. I might do it on Drupal these days, but I'd have to think about that a lot.


 9:23 am on Nov 3, 2009 (gmt 0)

Ok thanks ergophobe,

your done what I do to my clients, make them think through the fundamentals first (dam it, I am kinda feeling a little dumb now! )

So here goes:

1.1 Must be able to publish via an xml feed from a Siebel 8.0 CRM.

1.2 That requires some SOAP development, so a CMS that faciliates SOAP integration is good.

2. Multi editorial levels is not important. The basics found in any CMS will do.

3. Multi lingual content.

4. Faciliate clean code, ergo W3C usability guidelines, (thanks pageoneresults, [webmasterworld.com...]

5. Large community of widget coders.

enough to start with.


 9:49 pm on Nov 3, 2009 (gmt 0)

sorry, I don't know Siebel CRM... I would head over to CMSMatrix and OpenSourceCMS and see if you can't find something there.

Otherwise, I would start with some of the big ones (Drupal, Joomla, DotNetNuke, Expression Engine, etc) and

1. Search in your favorite SE "CMSNAME Siebel" and CMSName SOAP"
2. Do site searches on the main sites, as in "site:drupal.org Siebel"

See what that turns up.


 9:51 pm on Nov 3, 2009 (gmt 0)

BTW, that latter search yields


Not too promising, but I only looked at that one page form among the SERPs


 11:01 am on Nov 4, 2009 (gmt 0)

Once again,

thanks for the responses, they are much appreciated.

It was that rather worrying lack of content in Drupal, that got me a little worried! I saw that link adn basically came running to WebmasterWorld for a little extra guidance.

I will happily post any further findings as they come up.



 5:13 pm on Nov 4, 2009 (gmt 0)

I looked around a bit more jst because you piqued my curiosity. Part of the problem is that I barely know what CRM is. Never used it, never seen one in action.

As is typical of open source packages though (so drupal, Joomla, etc), they tend to only get a lot of interest for integrations with other open source packages because of licensing issues. Not always, but that's generally true.

So as such, SugarCRM and CiviCRM seem to have a lot more support. Not that that helps you.


 9:12 am on Nov 5, 2009 (gmt 0)

Saleforce is a comon entreprise CRM in the USA.

But no CRM appears to have done a decent job on its web publishing modules.

Which is why I am stuck looking for a CMS that will pull data from Siebel in an orderly manner.

Bit of a brain teaser this one!


 5:35 pm on Nov 5, 2009 (gmt 0)

Well please report back. As a general rule, if you can get well-formed data, you can build a module for the major CMS....

In fact, I have a new business idea for you... just because the base CMS is a free product doesn't mean the plugin has to be. I wonder how much one of the crack Drupal programmers would charge to whip that out. Drupal has a more robust API, but actually Joomla has a much stronger tradition of "for pay" add-ins.

In any case, I wonder how many copies you would have to sell to recoup your investment. Then you would have both the software you need and a little side business.


 7:07 pm on Nov 5, 2009 (gmt 0)

Unless you specifically have a need for an "admin section" there should be no problem for anyone with coding skills to whip something up for you.

On my Drupal site I call our CRM and get dropdown list data for our forms. When the form is filled out I package the data up and send it back to our CRM for importing.

The whole process is oblivious to Drupal except for the fact that I have to get the data into and out of Drupal forms using FAPI - it's just PHP, CURL, and some XPATH...and it could have been done on Drupal, Joomla or any other CMS.


 12:50 am on Nov 6, 2009 (gmt 0)

I searched for zope siebel integration, and stumbled on somebody (not me) who had this on their resume:
Assisted conversion from Siebel Customer Relation Management (CRM) system to large in-house system utilizing Python, Zope, LDAP and other open source technologies

- so I guess you could maybe look in that direction...

Zope has a BSD-type license which doesn't get in the way of integration with closed source / commercially licensed apps.


 6:25 am on Nov 18, 2009 (gmt 0)

The following 2 messages were cut out to new thread by ergophobe. New thread at: content_management/4027427.htm [webmasterworld.com]
9:06 am on Nov. 18, 2009 (PT -8)


 8:52 am on Nov 18, 2009 (gmt 0)

I am still at this.

Have found someone in Barcelona that has successufully integrated salesforce with joomla.

will report here when I finally get to talk to him.


 5:14 pm on Nov 18, 2009 (gmt 0)

Sounds like a telegram from a CIA bureau chief!

Let us know once you've contacted the Barcelona asset ;-)


 5:25 pm on Nov 18, 2009 (gmt 0)

jajajaja, I am actually based in the opposite part of Spain, but have clientes in Barcelona.

Same thing hapens to us Europeans when the yanks start talking about there geografic zones :)


 5:47 pm on Nov 18, 2009 (gmt 0)

Still, it has this air of convassing the whole globe and finally identifying "the asset" in Barcelona.

This is no doubt because I just watched one of the Bourne films and they're always talking about activating "the asset in Madrid" or Tangiers.


 11:58 am on Dec 1, 2009 (gmt 0)

Ok, latest news for those with an interest.

Talked to my contact in Barcelona. Apparently, there no problem with the development! Just not much demand, which is why its difficult to source via searches in your favorite SE.

Also gone to the local university, and they are interested as well.


 4:12 pm on Dec 1, 2009 (gmt 0)

Like I said - wrap this into a product. Although the "just not much demand" part might be an issue.

Glad it's getting sorted.


 8:01 pm on Dec 7, 2009 (gmt 0)

use drupal... it will help you to create billion of contents in just few days... it is user friendly and easy to use... joomla is another option.. but I'd not recommend you to use it... better you use drupal for content management... it can also help you to migrate contents from the old domain to new one...




 10:32 pm on Dec 7, 2009 (gmt 0)


Do you know a solution to integrate Siebel CRM? I couldn't find anyone who had done this.

If you can't integrate Siebel CRM without major expense, that's a dealbreaker for OddDog.


 3:22 pm on Dec 12, 2009 (gmt 0)

The following message was cut out to new thread by ergophobe. New thread at: content_management/4042833.htm [webmasterworld.com]
8:26 am on Dec. 14, 2009 (PT -8)


 3:59 pm on Feb 5, 2010 (gmt 0)

Sorry Ergophobe, no idea about Siebel CRM.




 5:48 am on Feb 6, 2010 (gmt 0)

You might want to take a few minutes to Google something like "drupal vs joomla vs wordpress". There are a lot of nice reviews on this very decision. I find that Joomla is a nice base in between the two. Drupal is very advanced, and some mention there is a slight learning curve. Joomla is easy to learn and use, and unlike Wordpress, isn't specifically geared towards blogging (although it can definitely handle that).

Global Options:
 top home search open messages active posts  

Home / Forums Index / Code, Content, and Presentation / Content Management
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved