Getting dynamic pages indexed

What is the best way to make sure my dynamic pages are indexed?

[sichen1234]

Hello. I would like to know what the best way to get my dynamic pages indexed by Google, Yahoo, and MSN are. Here are some options that I thought of:

1. Getting rid of all the _id tags in the URLs, so product_id=1000 becomes product=100
2. Change the "?" to "/" so www.example.com/store?product_id=1000 becomes www.example.com/store/~product=1000
3. Using the apache mod rewrite to change the URLs, so that I get URLs like: www.example.com/store/product/1000
4. Creating a static HTML map that points to all the dynamic URLs on my site
5. Creating a static HTML *version* of every dynamic page on my site

Someone has suggested each of the 5 steps before. As you can tell, each step becomes a little more work, and #5 will require some real programming. Does anyone know which of these are necessary?

[edited by: pageoneresults at 5:03 pm (utc) on July 5, 2004]
[edit reason] Examplified and Delinked URI References (mysite.com) [/edit]

[Birdman]

My opinion is that option three(3) is the best. Option two(2) would be my second choice.

The .htaccess is fairly simple too. Just go through your cart script and find the parts where URLs are written and alter the code to have it write them in the new form(/store/1000).

Then, your .htaccess would be like this:

RewriteEngine on
RewriteRule ^store/(.*)$ store.php?product_id=$1 [L]

Assuming the PHP script is named /store.php

[ControlEngineer]

One thing you make sure of is that after you are listed a search on "widgets" doesn't lead to a page about your snowshovels. I get frustrated sometimes with searches when I think I see exactly what I am looking for in the result, but I go to the site and it is something totaly different. That happens most often with news or information sites when one day widgetworldnews.com/article12345 is about the new blue widgets but later article12345 is about a new widget box company.

Having a large number of static pages, perhaps with 10 products per page and links to the dynamic pages, would help. It shouldn't be too difficult to do with an access or even excel data base and program to generate the pages for upload.

[sichen1234]

If I were to do the rewrite, then should I change the links inside my store also to do:

[mystore.com...]

So that the search engines can find it? Meaning, should it be that way from the home page all the way down?

Thanks,

Si

[Birdman]

Yes, you need to change all links on the site so that there are not two different links that go to the same content.

In other words, make sure there are no dynamic links(store?product_id=1000) left on the site. That will further confuse the crawlers.

Just in case you are not clear about the mod_rewrite solution. The mod_rewrite rules DO NOT make the URLs SE friendly, they rewrite friendly URLs back into their original form(/store.php?...). This way the scripts will work as usual.

[sichen1234]

Hi there. Thanks for all the suggestions. I'll take a shot at these.

Would you also say that it is no longer necessary to generate and store static HTML copies of your dynamic pages on the server?

Or is it a "can't hurt" kind of a thing to do?

(I know others have recommended this from time to time)

Thanks!

[sichen1234]

Hi there. Thanks for all the help so far.

One thing I noticed is that mod rewrite changes the URL shown on the browser, so if you start with
[mysite.com...]

The browser will eventually display:
[mysite.com...]

Is there anyway to control mod rewrite so that the URL shown in browser does not change?

Thanks!

Si Chen