Dreamquick - 11:20 am on Sep 29, 2003 (gmt 0)

Typically there are two routes to producing thousands of pages, both require an underlying database to populate them.

1) Take database content and use an off-line application to generate the pages, essentially using a template and then inserting the variables plus enough unique content to avoid being penalised for having v. similar pages.

2) Create a couple of dynamic pages (I worked out that you only need two actual pages to do this with the right back-end tech - mod_rewrite etc) which then expose a virtual file & directory structure by querying the database on-the-fly. Here you'd have one standard index page and one dynamic page acting as the template, capable of pulling the content from the database on demand to create X different view of itself.

Really a lot of this depends on your datasource - if you're amazon then you have a lot of distinct products with lots of content attached to them which leads to very little problem with duplicates, on the other hand if you were offering an SE-friendly store finder covering 1,000's of locations then you might have to give it a little thought to avoid creating pages which are very similar for instance what's different between your "London Widget Shop" and your "Manchester Widget Shop" aside from their location?

<added>
I know a couple of thousand might sound like a lot but it's not really if you are running perfectly valid permuations ... for instance if I have 10 products I want to target to 100 towns across the UK then I might be looking at creating a thousand possible pages from two reasonbly small lists.

It's also worth bearing in mind that the site structure you use will largely be dictated by how you intend to market the site - if it's going to be a normal site which will gain links naturally then the structure could be a bit nicer than one which is designed purely to be SE fodder.
</added>

- Tony

[edited by: Dreamquick at 11:41 am (utc) on Sep. 29, 2003]