Welcome to WebmasterWorld Guest from 54.196.244.186

Forum Moderators: phranque

Message Too Old, No Replies

Super Massive sites

100,000 , 500,000, etc etc sites, except the BBC :)

     
5:14 pm on Jul 7, 2006 (gmt 0)

Preferred Member

5+ Year Member

joined:June 15, 2006
posts:500
votes: 0


Hi Guys

I 've been wondering, do these sites with gigantic page numbers use some kinda automata to generate these pages?

or

do they create a page for each user like, say a forums,

so that each page is static html rather than dynamic, generated for each request

ergo, a ,,, classifieds site could have as many pages as individual items available for sale

5:40 pm on July 7, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 2, 2004
posts:551
votes: 0


Most blogs only consist of one file that changes depending on the request.

Yes giant sites use a content management system, but so do small ones. It all looks the same to the browser though. Learn SQL and a whole bunch of opportunities will open up for you.

5:44 pm on July 7, 2006 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts:12166
votes: 51


I 've been wondering, do these sites with gigantic page numbers use some kinda automata to generate these pages?

Yes, most everything is served from a database based on the query. You can literally build 5-10 template pages capable of generating millions of pages. The only limit is how many products are in the database. If you have 5,000,000 products, you can easily generate 5,000,000 pages.

5:54 pm on July 7, 2006 (gmt 0)

Preferred Member

5+ Year Member

joined:June 15, 2006
posts:500
votes: 0


I am familiar with ASP.net, but the page is generated for each request, consequently, the content is dynamic,

But I guess you're saying that these sites actually generate a static html page which is fixed an unchanging thereafter?

5:56 pm on July 7, 2006 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts:12166
votes: 51


But I guess you're saying that these sites actually generate a static html page which is fixed an unchanging thereafter?

No static pages, that's old school stuff.

No, these pages are generated dynamically on the fly based on the query. More like a virtual page. You need to build the core pages which contain the queries and then let the spiders do their thing from there.

6:03 pm on July 7, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 20, 2003
posts:408
votes: 0


I use a database to generate thousands of static html pages which are rebuilt every week or so. The pages load faster and dont have to access the database server continually (just once per week).

For me it a nice middle ground that allows me to be a little more creative/inventive when building the pages from the database. Static pages also have a SE friendly url without the need for mod_rewrite for dynamic pages.

[edited by: PhraSEOlogy at 6:06 pm (utc) on July 7, 2006]

6:05 pm on July 7, 2006 (gmt 0)

Preferred Member

5+ Year Member

joined:June 15, 2006
posts:500
votes: 0


Agghhh,, I have this picture in my head off the google spider.,,,,, pushing its way thru this kinda site,,, and actually generating the site by "stepping" on the links, triggering the query on the link,,,

Then the 100,000 pages vamoose as soon as the spider is gone,

or perhaps, the way the spider works causes the individual pages to exist when it follows an internal path way,,, till it exits the website

6:11 pm on July 7, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 20, 2003
posts:408
votes: 0


vite_rts,

Think of dynamic pages as a "bridge" to the real data (or page content) in the database. The data exists irrespective of type of page (static/dynamic). Dynamic pages just take out the process of having to convert the data to a static page.

6:35 pm on July 7, 2006 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts:12166
votes: 51


Then the 100,000 pages vamoose as soon as the spider is gone.

Actually the pages do not "physically" exist. They are dynamic. Once the spider indexes those 100,000 pages, they've now generated those pages for their index. They don't exist from a physical standpoint.