Making database driven pages Google-friendly

ISAPI filter to disguise query strings a good way?

[millie]

Hi everybody,

Really hope you can help. I'm having problems with Google indexing dynamic product pages on a couple of our e-commerce sites.

As a worst-case scenario the URLs I'm asking Google to index are like this:
www.mysite.com/products_detail.asp?productheadingID=531

I have text links from the main page to important dynamic pages. I have a site map accessible from all pages of the site. The sites have some link credibility and all individual pages are optimised. Robots.txt files have been checked for idiot errors.

Google just will not go beyond the static pages of these sites. Have played the waiting game for many months now.

So my questions:

Would it help to implement an ISAPI filter to disguise the query string and convert the URL to .htm?

EG: from www.mysite.com/products_detail.asp?productheadingID=531

To: www.mysite.com/SEFS/productheadingID.531/SEFE/products_detail.htm

Is there much to gain from this?
Is there any harm it can do?
Is there a better way?
Would you do it?!

Thanks in advance,
Millie

[sullen]

Don't know about the isapi filters, but on my sites Google has been spidering dynamic pages with variables in the querystring just fine lately.

Before you go to all this effort, are you absolutely sure there is nothing else about the pages which would prevent them from being spidered?

What about changing the variable name from productheadingID to productCode? (I never use the letters ID in variable names just in case, though can't guarantee it'll work.) How often does the rest of the site get spidered?

[Iguana]

I use page.asp?formid=6 and Google has these indexed currently.

They have at times dropped out for a month (in the days of the monthly update) and I guessed this was because of a slow response from the pages when Googlebot came calling

[chadmg]

If you search around in these forums you'll find that google does not like when you put ID in the querystring. It thinks you're relaying the Session ID and displaying visit specific pages. To be safe you should at least take out the ID. There's nothing wrong with projectheading=531.

Other search engines aren't as good with querystrings so it's even better to fake the address with an ISAPI filter, .htaccess, or some other code. I didn't have access to ISAPI filters on my shared windows server so I have my 404 error page load the correct page. It's not too hard to do.

[millie]

Thanks for your help everyone!

I'll definitely look into the session ID thing in more detail - it looks like I might have been badly advised on this. Am relieved I posted the question.

Googlebot visits daily - average of 7 times a day but with only an average of 20 page views per visit. This would make sense from a point of view of it poking about the static pages but going no further.

PR is 5.

Thanks all! M

[millie]

Googlebot visiting 7 times a day sounds pretty unlikely doesn't it?! Think I might need to work out what my stats are reporting on here.
M

[Lorel]

I have a similar question re a site that has Ampersands (&) in the url for the shopping cart, on site search engine (for the site map) and also the classified section. Those URLs wouldn't validate on WC3 and I was told to include an amp; instead of the & symbol.

However now I'm concerned that the search engines may not read those URLs either because I ran across a major search engine submit site that said not to use the ASCII code in the URL.

Any suggestions?

[dirkz]

to disguise the query string and convert the URL to .htm

It highly depends on the PR of the linking page(s) whether Googlebot will index URLs with query strings. Even when disguised, keep in mind that shortness could be a factor.

www.mysite.com/SEFS/productheadingID.531/SEFE/products_detail.htm

This is still very long and complicated. Maybe you can get some inspiration from this forum, most pages are very likely "rewritten" (I think that's the equivalent for what you call ISAPI filter on apache) to get the static html appearance. Make your URLs as short as possible, but still containing the keywords for SEs and users.

Those URLs wouldn't validate on WC3 and I was told to include an amp; instead of the & symbol.

&s in the URL are very common, maybe the person giving you the "advice" of using & meant something utterly different or lives in world far from here.

[BroadLea]

I have a 3-year-old Miva Merchant site with URLs like:

[mysite.com...]

and finally, about a month ago, they were getting spidered AND appearing in SERPS.