Basic web directory structure for SEO... need help

[impact]

Hello,

I had no idea about this before I was told that google and other major search engines usually find it difficult to crawl beyond 2 sub directories

i.e www.domain.com/subdir-1/subdir-2/filename.html

How ever in my site I have pages like this

www.company.com/company_name/products/product_type/product_filename.html

This design of the site is about 5 months old and now I am slowly beginning to get my visitors from google search based on my product key words.

The question is should I move my url to some thing

www.company.com/products/product_type/product_filename.html

or just leave it as it is and pay more attention to making my site better.

The other thing is not all the pages located here
www.company.com/company_name/aboutus.html or similar are not in google index.

Thank you,

[Robert Charlton]

Hi impact - While not directly in response to your question, this thread below answers some questions relating to directory structure vs number of clicks from home, and should get you thinking that the directory structure probably isn't your core problem....

Google and Physical Siloing
http://www.webmasterworld.com/google/4218584.htm [webmasterworld.com]

Your directory structure does seem to have some superfluous levels in it. I'm not sure why some of them are there, and whether you've divided your navigation up into too many subcategories, or whether you need those levels because you have a large site.

In the context of the rest, these two levels, if they do parallel your linking structure, may be too much...

company_name/products/

Is "company_name/" actually manufacturer or brand?

Not sure of the distinction between "products" and "product_type", but I can imagine where main product categories and then product subcategories, given enough products, would be completely appropriate.

Your navigation structure and organization, clicks from home, number of links on a page, flow of link juice throughout the site, the inbound linking at various levels, the appropriateness of the structure, the number of products and categories, etc, etc are what matter. It's not simple... and it's not just a matter of counting directory levels.

Read that discussion (and the discussions I link to in that thread) and then follow up with more questions.

[impact]

Hello and thank your reply.

I have gone through the link that you have provided. Its bit complicated to understand in the first glance.

Let us assume my company name is Tea Fertilizer Private Limited and my domain name is tea.com. There are only 9 PAGES in my site.

This where my site pages are hosted
www.tea.com/teafertilizer/aboutus.html
www.tea.com/teafertilizer/index.html
www.tea.com/teafertilizer/contactus.html ... and so on

This is where my product pages are hosted:
www.tea.com/teafertilizer/products/organicfertilizer/teaorganic.html

There are no pages inside "product" & "organicfertilizer" directory. I have included it for SEO as the product we make is an organic fertilizer.

I just checked bing.com and there, in the search result my my entire product url is displayed, I am wondering should I or should I not remove directories at this point of time.

Thank you,

[jdMorgan]

[...] these two levels, if they do parallel your linking structure, may be too much...

There is a "hint" there that should be emphasized: URLs and filepaths are not the same thing, and need not even resemble each other if URL rewriting is used. Therefore the quoted "if [those directory structure] levels do parallel your linking structure" qualifier has some additional meaning that should be noted...

You could just as well flatten the URL structure (while keeping the directory structure the same or not, as you choose) to www.example.com/organic-tea-fertilizer-product.html or www.example.com/products/organic-tea-fertilizer (note that there is no "file" extension necessarily required on a URL).

And to add, the number of "directory levels" is almost irrelevant, if kept within reason. What matters is not the "file directory depth," but rather, the "linked-URL click depth" -- How many links must be followed to reach the page under consideration. And again, note the distinction between the URL locating-system used "out there on the Web" and the directory-and-filepath locating-system used "here, inside the server." These are two different "addressing" methods for use in two separate and distinct naming spaces, and associated only by the action of the Web server.

As a simple example to make clear what can be done, you could easily 'map' the URL www.example.com/products/organic-tea-fertilizer to the existing filepath /teafertilizer/products/organicfertilizer/teaorganic.html
using a URL rewriting rule like

RewriteRule ^products/([a-z]+)-([a-z]+)-([a-z]+)$ /$2$3/products/$1$3/$2$1.html [L] 

in .htaccess on an Apache server, or a similar rule using ISAPI Rewrite on IIS.

The "numbered dollar signs" back-reference (refer back to) the text captured by each of the parenthesizd sub-patterns in the RewriteRule's regular-expressions pattern. So in this specific example, $1 is "organic", $2 is "tea", and $3 is "fertilizer".

Having "broken free" of thinking that the directory-and-filepath and the URL have to be the same, it would of course be possible to use a more easily-manageable file-naming structure as well as a "flatter" URL, but this example is simply intended to show that the URLs and filepaths can be different, and that the existing filepaths could be retained even with a completely-rearranged URL.

As for the filesystem, I'd suggest a filepath convention like /products/organic-tea-fertilizer.html, keeping all product pages in the "/products" subdirectory just to keep things a bit organized as your product inventory expands over time. The URLs should "look good" to searchers and search engines (get rid of excessive keyword repeats), while the directory and file naming convention should make sense from a 'business plan' standpoint and support efficient site maintenance.

Jim

[HuskyPup]

There are only 9 PAGES in my site.

Do you intend staying at 9 pages or expand it greatly?

My structure is always the same, I devised it in the 90s so that no page was ever more than 2 clicks from any page on a site:

Across the top

example.com/
example.com/about.html
example.com/contact.html
example.com/faqs.html

Lefthandside navigation

example.com/country/products/index.html
example.com/country/products/indepthproduct.html

Does that help?

[impact]

@ jdMorgan,

Thank you so much for the detail explanation.

Prior to this url structure I did some thing like this. At that time the url structure was
www.tea.com/teafertilizer/products/organic-fertilizer/teaorganic.html

"organic-fertilizer" looked bit complicated to me, so I changed it to the present structure and used the Permanent Redirect (301) option available in cPanel in Linux shared hosting server.

We also use a 301 redirect to redirect www.tea.com to www.tea.com/teafertilizer/index.html

As of now we have only 9 pages at www.tea.com/teafertilizer/ ... The number of pages will increase only if we make more products. Which is about 1 page increase every 6-8 months.

We do have a help section and a blog but those are hosted in a sub domains. We have blog posting ONCE IN EVERY WEEK.
help.tea.com which is same as www.tea.com/help
blog.tea.com which is redirected to a blog platform outside our server.

The main reason for having such product url is that to append the new product all i have to do is the following url change

present url structure
www.tea.com/teafertilizer/products/organicfertilizer/teaorganic.html

proposed new url structure
www.tea.com/teafertilizer/products/biopesticides/teabiopesticide.html

As of now our URL is the SAME AS DIRECTORY structure.

My understanding from your post is that you are suggesting to let the DIRECTORY structure remain the same but for the web viewing, we should change it to the following using .httacess

www.tea.com/index.html
www.tea.com/contactus.html
www.tea.com/aboutus.html
www.tea.com/teafertilizer.com .. etc

QUESTION.
1 . If i change the present url (our directory and url structure are the same)
www.tea.com/teafertilizer/products/organicfertilizer/teafertilizer.html

to using .htaccess
www.tea.com/teafertilizer.html
Will my ranking in my search engines remain the same?

2. How about using a 301 permanent redirect to change the url instate of .htaccess?

Thank you,

[edited by: tedster at 2:04 am (utc) on Oct 28, 2010]
[edit reason] make example urls visible [/edit]

[aakk9999]

We also use a 301 redirect to redirect www.tea.com to www.tea.com/teafertilizer/index.html

Why are you doing this? You should let your www.tea.com return the content of what is now on www.tea.com/teafertilizer/index.html

My understanding from your post is that you are suggesting to let the DIRECTORY structure remain the same but for the web viewing, we should change it to the following using .httacess

www.tea.com/index.html

This is not what HuskyPup suggested. The home page content should be returned at domain root, not at index.html. You should permanent redirect index.html to home page root instead.

Also, for any changes you are doing, you must make sure that your existing redirects point to the new page location in one hit so that there are no chain redirects.

As to your questions:

1. You may experience temporary drop but if your new redirects are technically sound (in one hit, no chain redirects, change internal URLs to refers to refer to the new URL etc), you should recover reasonably soon. But the changes must be done correctly and not introduce new problems.

2. Well, .htaccess is performing 301 permanent redirects. I guess what you are asking is - could you use cPanel 301 instead of .htaccess

301 is 301 and if done properly then it does not matter what you use. Not sure if there would be any implications with regards to performance one versus another, however, if you have multiple rules you have to apply then .htaccess is the way to go, in which case you must not forget to remove your 301s set already in cPanel and make sure that all your rules are captured in .htaccess properly. .htaccess should also cater for www versus non-www redirect.

[impact]

@ aakk9999

Thank you for replying.

" Why are you doing this? You should let your www.tea.com return the content of what is now on www.tea.com/teafertilizer/index.html "

The reason why we did this in the first place was that, while www.tea.com is our domain name and TEA is our trademark but teafertilizer is the name of the company. So we thought for all "Tea Fertilizer" we will have a better ranking.

Thank you,

[Robert Charlton]

impact - When I looked at that directory structure, I thought it might have been intended to handle 1000-2000 products or more. Seriously.

For a site of the size you're describing, I'd go with a completely flat structure... no subdirectories at all.

IMO, keywords in a pathname have an insignificant effect on rankings for anything competitive. The highlighted keywords in the search results may help with click-throughs, but otherwise those keywords aren't going to be very helpful. At worst, they can look spammy. I certainly wouldn't add extra directory levels to get them in.

A company will generally rank well for its own name in Google because a large number of inbound links to the home page is likely to use the name as anchor text. It helps to include the company name in the home page title, as well as in onpage html text content.

[impact]

@ Robert

"impact - When I looked at that directory structure, I thought it might have been intended to handle 1000-2000 products or more. Seriously. "

Yea, may be we were over cautious that we ended up being to perfect for such a small site. LOL

[Robert Charlton]

...too perfect for such a small site.

And perfection not addressing the necessary issues... what a waste. ;)

In this structure you outlined... company_name/products/product_type

I thought perhaps that...

- "company_name" was to accomodate maybe each of 10 different brands, so you'd have 10 different company_name directories, all linking from home...

- "products" was to accomodate maybe each of a dozen or so different main product categories, so you'd have perhaps dozen or so different products subdirectories...

- and that "product_type" was then going to be another dozen or so product_type subcategories for each product main category.

In this kind of structure, I would have probably put brands further down the line... more probably would have included them as a main category or subcategory by itself, depending on how similar all your product brands were.

But, most important, I wouldn't have had multiple "utility" links, as I call them, for 'about us', 'contact us', etc etc, with only one link for products or services... whatever it is that you're selling.

See my comments about structure in this thread for more thoughts on that...

do backlinks to my internal pages still positively affect my homepage?
http://www.webmasterworld.com/link_development/4214750.htm [webmasterworld.com]