ISAPI rewrite - tantalizingly unsuccessful, so far...

Struggles with rewriting query-string urls to get Google in.

[SteveJohnston]

Hi Everyone,

We are working hard on a Microsoft Commerce Server e-commerce site which has until very recently refused access to Googlebot. Our homework has informed us that it has been the long urls with query strings that are the culprit.

As a result we have successfully implemented an ISAPI rewritten site map page with nice, neat and short urls with no hint of a query string. This appears to have had the desired effect, in that Googlbot has been all over the site in recent days.

The strange thing, and it may not be related, is that the web logs report visits to the original long urls.

Also, the site is enjoying almost daily revisits by Google, which is reflected in the frequent changes in the Google cache status of the site map page we have modified regularly during this period.

The killer is that whilst the site map changes appear to change very frequently there is no sigh of any of the product pages whatsoever.

So, my questions, finally, are:

1. Am I just being impatient.
2. Does Google smell a rat.
3. Because the logs are reporting the urls in their original form, is there a possiblity that Google sees this too?

Any help/ideas greatfully received.

Cheers,

Steve.

[Rhadamanthus]

When you hit a URL-remapped page, does the address bar in your browser change to the new URL, or does it stay with the "mapped" URL? For instance, if I map "foo.html" to "bar.html", and then type in "foo.html" in my browser, will it show "foo.html" or "bar.html" after it loads the page? You can setup ISAPI_Rewrite to do it either way, although I don't remember the exact syntax off the top of my head. If it changes the URL in the address bar, then it's sending out the real URL to the outside world, and this may be what's causing your problem.

[pageoneresults]

Hello SteveJohnston, Welcome to WebmasterWorld!

Have you checked the Server Header status of the rewritten URLs? Make sure they are not returning a 404.

Also, are you generating page specific titles and meta data? If not, you should definitely look into making sure at least your title elements are specific to each page.

I haven't used the ISAPI tool, but I can vouch for the IISRewrite tool that utilizes the ini file for rewriting URLs. Ever since we started using this a while back, our database pages are getting crawled and indexed regularly. And, they usually appear in the top ten for their targeted keyword phrases.

Check your URLs here to make sure they are returning the proper header information...

Server Header Checker [searchengineworld.com]

[SteveJohnston]

Thanks for the input.

[Rhadamanthus: When you hit a URL-remapped page, does the address bar in your browser change to the new URL, or does it stay with the "mapped" URL? ]

No it stays with the mapped URL.

[pageoneresults: Have you checked the Server Header status of the rewritten URLs? Make sure they are not returning a 404.]

They are not returning a 404, but in the headers the location is identified as the .asp+querystring link. So although Google is getting to these pages perhaps it decides not to index because of this?

[pageoneresults: Also, are you generating page specific titles and meta data? If not, you should definitely look into making sure at least your title elements are specific to each page.]

No we are not. We will, but doubt that this is the showstopper.

We are going to work some more on the header information and also investigate the IISRewrite tool you mention.

Cheers people.

Steve