Our website has VERY strict url handling for what will be returned with a 200 status code. This is to remove duplicate content issues. We enforce protocol (http vs https), sub-domain (we dont use www), url case, file names (obviously), and ending slashes.
We recently launched and all bots and users handle this fine except Yahoo Slurp. For some reason it sometimes (not always) adds an extra slash to the request which is the 404-ed.
http://example.com/my-url - Right http://example.com/my-url/ - Slurp request
There are no links pointing to the page with an ending slash. I am curious as to if this will hurt us (or perhaps help, if it is testing this)? Any ideas?
I could have the handler 301 to the non slash URL but it could result in a 301 to a 404. Is that seen as an issue?
[edited by: martinibuster at 10:30 pm (utc) on Jan. 11, 2008] [edit reason] Fixed URL. Example.com is for examples. [/edit]