'folder' style urls

rewrites using folder style urls

[nick279]

I've just launched a new e-commerce using rewritten 'folder' style (.htaccess) urls

e.g.

/cat/
/cat/product-name/

(The urls feature a trailing slash)

Thing is (yahoo in particular) shows the urls without a trailing slash, and I was wondering if it's going to cause any confusion and I should switch to a rewrite with a more popular extension

e.g. /cat/product.html

Been indexed for around 2 months and done some deep linking to these pages. I'm now getting a couple of errors like 404s without the trailing slash appearing in GWT

Any thoughts?

[jdMorgan]

This is why a "page" URL should never end in a trailing slash...

You might consider redirecting slashless page URLs to the slashed version, so as to force Yahoo to use the right URL (it may still display incorrectly, but they'll be able to get to the right URL).

Alternatively, since the site is new, you might consider dumping the problematic trailing slashes from your page URLs, and redirecting the other way. If this site is destined for a long and bright future, fix it now. If it's only intended to live a couple of years, just leave the slashes.

Jim

[nick279]

Think I'll redirect to the slashed version, argh :)

Took me a considerable amount of time to implement this system as is, and we are getting some ok rankings, particularly for category pages, but not so great on product pages (two 'folders' deep)

It's definitley a long term project, I just can't weigh up whether to make the change or not as it's an extensive exercise.

Future projects will definitley take the /category/product.html route

[jdMorgan]

/category/product is better -- No need to include ".anything"

[nick279]

Great,

Thanks for advice

[venti]

Pretty funny, the opposite happens to us. We don't end the urls with slashes and Yahoo adds them. We simply have a 301 to redirect on this and it does well (they will get a 301 and then immediately go to the new one).

[jdMorgan]

Yes venti, but the good news is that you are following HTTP standards (no slash on "file" URLs, slash on "directory" URLs. So you are clearly in the right here, while Yahoo is wrong. Therefore, you can expect fewere troubles with search engines on the whole, and hopefully Yahoo will stop their silly "unlinked directory fishing" and follow standards in the future.

Jim

[nick279]

Also I use a little tip to help keep .htaccess a bit simpler

Use this if you have upper/lower case matching requirements

If working on php/mysql sites, and are for example rewriting /folder1/ to page.php?id=folder1

You can do a slight modification to your query

SELECT * FROM `TABLE` WHERE [b]BINARY[/b] `URL` = '$_GET['folder1']

This will only return a result if the case matches the value of the get field

e.g. 'FoldeR1' or 'Folder1' would not be matched and takes the requirement out of the htaccess file

The best method would be to code everything in lowercase from the start, but if you happen to take over a site like I did recently, you may find this useful

n.b. example only, always secure your parameters when performing rewrites

[g1smd]

Look closely at the Yahoo SERPs again.

On the page, they show the URL with the / missing.

Look at the page source and see that the target of the link DOES have the "/" on the end - usually.

[jdMorgan]

Yahoo usually omits the slash in the visual presentation of the link, but includes it if it was part of the spidered URL.

My above reference to "unlinked directory fishing" refers to Slurp's very annoying habit of trying to 'discover' directory indexes by removing the "page name" from a link on your site and requesting that page's directory.

They do this even when no links to that directory URL exist anywhere on the Web, and it results in a 403-Forbidden on any Apache server where Options -Indexes is set and an index file does not exist in that directory.

Basically, I have a big problem with search engines that try to spider un-linked, made-up URLs.

Jim

[g1smd]

What about when they do that for root?

All links go to domain.com/default.asp and they will attempt to access domain.com/ to see what is there.

I don't have a problem with that. I make sure the URL returns the right response; usually content, but sometimes 403 or 404.