(Previously posted here [webmasterworld.com])
Recently however, we identified another potential duplicate content issue in the following:
http://mydomain.ext/myfolder/
and
http://www.mydomain.ext/myfolder/default.asp
We needed to redirect [mydomain.ext...] to [mydomain.ext...] anywhere it might occured on the site.
Unfortunately, this problem was a little more tricky. At first, it seemed that ASP servervariables were unable to determine the difference between the two requested URLs. Calling request.servervariables("URL") (or "PATH-INFO") would return "/myfolder/default.asp" in both cases as, when requesting a folder root, IIS would automatically add the default document name onto the "URL" result. With both results being the same, we had thought we would be unable to 301 redirect these requests.
The answer lay in a 'quirk/feature' of the default documents list in IIS. We dicovered (purposefully or accidentally) that IIS has set all the default documents with capital letters e.g.:
Default.asp
Index.htm
etc..
Looking back at the request.servervariables("URL") results the difference became clear:
Requesting http://mydomain.ext/myfolder/ resulted in "/myfolder/default.asp"
Requesting http://mydomain.ext/myfolder/ resulted in "/myfolder/Default.asp"
So long as we made a case-sensitive comparison of the "URL", we could determine when default.asp was included in the URL and 301 to the root of the folder instead with the following update to our existing code: