Welcome to WebmasterWorld Guest from 54.226.133.245

Forum Moderators: ergophobe

Message Too Old, No Replies

100k pages - Which technology is best to serve this?

ASP.net or off the shelf app

     
8:15 pm on Aug 3, 2012 (gmt 0)

New User

joined:Feb 17, 2012
posts: 13
votes: 0


I am publishing a site with 100,000 pages. At first glance I thought to use Magento and disable the shopping cart parts, so I could have an SEO strong, but this seems to be very memory use intensive which raises hosting costs above the $30/mo I have in mind for this site.

If you were to publish 100k pages (revenue from ads), and you wanted a low hosting cost yet have reasonably fast page loads, what would be your tech stack? (i.e., ASP.NET or off the shelf app)?


JB
4:56 am on Aug 7, 2012 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8127
votes: 88


The number of pages on a site is perhaps the least important factor in choosing a way to manage that content.

Any CMS can handle that though if I recall correctly, traditionally WordPress has performance issues with large numbers of "pages" (but not large number of "posts"). Sometime like that.

Anyway, what's really going to matter is how the pages are put together (how many DB tables are you drawing from, are they indexed, can the queries and/or pages be cached and so on), how many concurrent users you have at *peak* load, and what you have for hardware ($30/mo can get you a decent VPS if you don't need a lot of support) and software (opcode caching or not) etc.

If I were expecting a lot of users and complex or numerous DB queries, I would want to be sure the system had good caching. This might be simple or complex. On some sites, I've just grabbed the output buffer in PHP and if there's no static version of the page, cache one. On the next request, you're serving static pages with just a quick check for whether not the cached version exists.

If you don't have a lot of dynamic content, you can get very fast page loads even if the initial page generation is processor/memory intensive.
2:52 pm on Aug 8, 2012 (gmt 0)

New User

joined:Feb 17, 2012
posts: 13
votes: 0


What tech stack would you use to publish 100,000 pages with minimal DB table queries? I am thinking of a site similar to schooldigger.com, which has as many pages.

Also, I've only used cpanel not VPS - is it difficult to learn for a novice?

JB
3:08 pm on Aug 8, 2012 (gmt 0)

Full Member

10+ Year Member

joined:Jan 2, 2005
posts:330
votes: 0


Magento guzzles memory, I'd give that a miss.
3:13 pm on Aug 8, 2012 (gmt 0)

New User

joined:Feb 17, 2012
posts: 13
votes: 0


What would you propose Jinxed? ASP.NET custom build?
3:14 pm on Aug 8, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 28, 2003
posts:1977
votes: 0


Actually, I would consider looking at WordPress. There are plenty of sites that are using WordPress and don't have any issues. Pages can be easily cached, which helps with server load.
3:18 pm on Aug 8, 2012 (gmt 0)

New User

joined:Feb 17, 2012
posts: 13
votes: 0


OK, but 100,000 pages? That is the size of my database I want to publish.
3:26 pm on Aug 8, 2012 (gmt 0)

Full Member

10+ Year Member

joined:Jan 2, 2005
posts:330
votes: 0


I think you're confusing a CMS and the language the CMS is built in.

PHP, ASP etc all do the job.

To be honest, I would look into Wordpress. I don't personally use it but it is the market leader for a reason and easy to tailor to your personal needs.
3:38 pm on Aug 8, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 28, 2003
posts:1977
votes: 0


One consideration is whether or not you'll have to hit the database each and every time a page is loaded for a user.

When you're talking server load and having to server up lots of page views, it's best to using caching--and WordPress can handle that by caching the content.
3:48 pm on Aug 8, 2012 (gmt 0)

New User

joined:Feb 17, 2012
posts: 13
votes: 0


Magento caches content very well which is why I started using it for this project. I also stated writing my own database driven site in ASP.NET, but went to Magento since it exists and has the caching, indexing, CMS features, and SEO tools. But it is very slow.
4:22 pm on Aug 8, 2012 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8127
votes: 88


JB - you keep mentioning the number of pages. If it's a simple query (which a cached page in Wordpress or Drupal should be), then 100,000 pages is a *small* number, especially if you have your DB tables properly indexed and you measure and slow queries (if possible on your hosting setup).

The number of concurrent users is a far more important number and that's where the overhead of something that is highly dynamic starts to hurt. For example, the "who's online" feature of bulletin boards and forums tends to cause huge overhead in and of itself if there's a large user base.

So when you say Magento is slow, that is not likely because you have 10 pages, 10,000 pages or a million pages. It is most likely simply because generating any page in Magento takes many queries or a few very slow queries. That's the part that will make scaling difficult.

So like bhartzer says - if you use WP and implement agressive caching (WP SuperCache for example) or Drupal (with Boost), your bottleneck will likely be on the front end - CSS and JS cruft. Then it's a matter of a theme that keeps things simple, and not adding features that add complex and slow DB queries.
4:53 pm on Aug 8, 2012 (gmt 0)

New User

joined:Feb 17, 2012
posts: 13
votes: 0


ergophobe - very awesome description. makes perfect sense now. thank you!
4:02 am on Aug 9, 2012 (gmt 0)

Moderator This Forum

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8127
votes: 88


You're welcome - doesn't solve your problem or answer your question, but hopefully it helps you focus on the questions that will matter the most.
2:44 pm on Aug 25, 2012 (gmt 0)

New User

joined:Aug 25, 2012
posts:2
votes: 0


Have you looked into WordPress with SQLite? Also important: does the content change over time a lot? You might also want to look at Sphinx Search Engine to speed up your full text searches (from Wordpress search functions):

[wordpress.org...]

Also remember that with popular posts, you will want to have query caching activated. But if all posts get visited the same amount of times, you might not want to do activate this.
2:30 am on Oct 4, 2012 (gmt 0)

New User

joined:Sept 29, 2012
posts:19
votes: 0


Any forum software can easily handle that much and more.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members