homepage Welcome to WebmasterWorld Guest from 54.161.220.160
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / WebmasterWorld / Accessibility and Usability
Forum Library, Charter, Moderators: ergophobe

Accessibility and Usability Forum

    
site architecture - Internal Linking
Best practice for internal linking for alpha letters?
SEOold

5+ Year Member



 
Msg#: 3365670 posted 6:59 pm on Jun 12, 2007 (gmt 0)

What is the best way to do the following?

My site is over 100K pages and itís broken down by homepage, categories and product page.
From the category page we are linking to all the pages but some pages with a site as big as mine get buried 3 to 5 levels deep, which makes it difficult for a user to even find some of the products.

I have alpha pages that will resolve this situation by bringing all or most of the pages three levels. EX. A ¶ B ¶ C ¶ D ¶ and so on.

If a user clicks on letter ĎAí they will see all the products that begin with this letter. Now every alpha page has its own title tag and description tag.

What Iím finding is that these pages are falling in supplemental index which makes sense because there are no backlinks and the title tag is only unique by two words. EX. 'A' Widgets Directory - Widgets starting with Letter 'A'. I also donít care if these pages rank or not because the main purpose is for internal navigation for users and also so SE can find my other pages that are buried and index them.

What would be the best option in working with alpha letters as I have to implement similar with another site of mine?

1. Add Meta tag noindex, follow Ė and if so would the bots really follow and index the end result.
2. Continue with the way that Iím doing?
3. Other suggestions?

Thanks in advance.
If posted under wrong category. Sorry. I was not able to find a site architecture category to post on.

 

pageoneresults

WebmasterWorld Senior Member pageoneresults us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3365670 posted 7:08 pm on Jun 12, 2007 (gmt 0)

Widgets starting with Letter 'A'. I also donít care if these pages rank or not because the main purpose is for internal navigation for users and also so SE can find my other pages that are buried and index them.

If that's the case, I'd drop a robots meta element in there and prevent them from indexing...

<meta name="robots" content="none">

What would be the best option in working with alpha letters as I have to implement similar with another site of mine?

Sometimes Webmasters want those alpha pages indexed and will set up filters to generate that content. How you link to that content determines its "rank" in the overall flow. Think about "click paths".

In your case, since you don't care if the SE's find that stuff, just block it from being indexed. Let the juice flow where it is needed. Take that bot and throw a harness on it and direct it to where it needs to go and also tell it where it needs to stay out.

1. Add Meta tag noindex, follow Ė and if so would the bots really follow and index the end result.

I wouldn't even chance it. I personally just block the bot from indexing using the above metadata element. Its a poor man's way of doing it but I can say from experience that it works.

I can confirm the above. I just did a search for my test pages and sure enough, they do not appear in Google, Yahoo! or MSN. I'm afraid to check in Ask. ;)

pageoneresults

WebmasterWorld Senior Member pageoneresults us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3365670 posted 7:19 pm on Jun 12, 2007 (gmt 0)

<meta name="robots" content="none">

Another useful tip using the above robots meta element is you can literally control the flow through your site. I'll reiterate, it is a poor man's way of doing it but it works and it follows protocol which is the important thing for me.

For example, let's say you are launching a new site. You have a lot of pages that are going live. Many of those are deep level pages that don't really need to be indexed, "at this moment". Just block them from getting indexed and push the click path back one or two clicks. Its a lot easier to bring visibility to pages that are 1, 2 and 3 clicks from entrance. Its those pages at 4, 5 and 6 that are tough.

For the user, provide multiple access points to get to that content. You can use linked menus, dropdowns, etc. The alpha listings are usually a sort feature for the user. Make it easily accessible to the user and if you block the bots from indexing it, you've covered your bases. And, you may have prevented some duplicate content issues in the process. ;)

SEOold

5+ Year Member



 
Msg#: 3365670 posted 7:33 pm on Jun 12, 2007 (gmt 0)

If that's the case, I'd drop a robots meta element in there and prevent them from indexing...

<meta name="robots" content="none">

Would this tag still alow bots to follow the links on the page?

What would be the best option in working with alpha letters as I have to implement similar with another site of mine?

Sometimes Webmasters want those alpha pages indexed and will set up filters to generate that content. How you link to that content determines its "rank" in the overall flow. Think about "click paths".

Currently that is how its set up, but they also fall in supplemental so I was tryign to avoid the situation. The content of the alpha letters changes constantly if new pages are added.

In your case, since you don't care if the SE's find that stuff, just block it from being indexed. Let the juice flow where it is needed. Take that bot and throw a harness on it and direct it to where it needs to go and also tell it where it needs to stay out.

Not sure what would be best the 1st option above or the second one.

1. Add Meta tag noindex, follow Ė and if so would the bots really follow and index the end result.


I wouldn't even chance it. I personally just block the bot from indexing using the above metadata element. Its a poor man's way of doing it but I can say from experience that it works.

Pageone can you clarify this I'm not sure if I follow you on this.

[edited by: pageoneresults at 7:36 pm (utc) on June 12, 2007]
[edit reason] Fixed formatting issues. [/edit]

pageoneresults

WebmasterWorld Senior Member pageoneresults us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3365670 posted 8:00 pm on Jun 12, 2007 (gmt 0)

Would this tag still alow bots to follow the links on the page?

No it wouldn't. The original one you provided would...

<meta name="robots" content="noindex, follow">

Currently that is how its set up, but they also fall in supplemental so I was trying to avoid the situation. The content of the alpha letters changes constantly if new pages are added.

I'm going to say its the changing content that is causing them to go into the supplemental. Just because they are in supplemental doesn't mean that they are penalized or banned, etc. Since the content is changing, I'm going to guess that Googlebot can't really figure out the "true meaning" of the page which causes it to get relegated to the supplemental index.

Not sure what would be best the 1st option above or the second one.

Do you want the bot to follow those links? If so, then your original code example of noindex, follow would be correct.

But, what value is there in having those links followed but not indexed?

SEOold

5+ Year Member



 
Msg#: 3365670 posted 8:20 pm on Jun 12, 2007 (gmt 0)


Not sure what would be best the 1st option above or the second one.

Do you want the bot to follow those links? If so, then your original code example of noindex, follow would be correct.

But, what value is there in having those links followed but not indexed?

I would like the bots to follow the link that are listed on the alpha pages but the actual alpha pages not indexed. The product pages I do want indexed.

To reword my question is there any advantage to having the alpha pages indexed? How many people would actually search for 'Widgets Starting with Letter A'. Is there any advantage that I'm over looking here. I'm actually practicing this but I was wondering if this is a step that is not even needed becasue who would search for this. Also if the pages are falling in supplemental results then the pages are not even crawled as often as the regular index.

pageoneresults

WebmasterWorld Senior Member pageoneresults us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3365670 posted 8:29 pm on Jun 12, 2007 (gmt 0)

I would like the bots to follow the link that are listed on the alpha pages but the actual alpha pages not indexed. The product pages I do want indexed.

Then your original thought was correct...

<meta name="robots" content="noindex, follow">

To reword my question is there any advantage to having the alpha pages indexed?

There are always advantages to having pages indexed. But, in your case, with the content on those pages changing regularly, there may not be an advantage. There could be a negative consequence which I've run into many times with content that changes on a page. The reference that was indexed is no longer available on the resulting page. If you think about it, that would fall under the "bait and switch" category.

How many people would actually search for 'Widgets Starting with Letter A'.

That's not what I'd be looking at. I'd be looking at the possible combination of phrases that could be searched based on the content on those index pages.

Also if the pages are falling in supplemental results then the pages are not even crawled as often as the regular index.

I wouldn't get too hung up on the Supplemental results. Think of it as a holding area while Googlebot continues to make its way through your site and sort things out. I think it is a given these days that any large site is going to have a certain percentage of its pages in the Supplemental index.

SEOold

5+ Year Member



 
Msg#: 3365670 posted 9:18 pm on Jun 12, 2007 (gmt 0)

Pageone didn't get a chance yet, thanks for all the feedback!

To reword my question is there any advantage to having the alpha pages indexed?

There are always advantages to having pages indexed. But, in your case, with the content on those pages changing regularly, there may not be an advantage. There could be a negative consequence which I've run into many times with content that changes on a page. The reference that was indexed is no longer available on the resulting page. If you think about it, that would fall under the "bait and switch" category.

To clarify what I meant by the content changes constanlty what I meant was that it is added on to the list. The old product links always stay up and new products get added on the list if they start with letter A. Do you still feel this way?

How many people would actually search for 'Widgets Starting with Letter A'.

That's not what I'd be looking at. I'd be looking at the possible combination of phrases that could be searched based on the content on those index pages.

Also if the pages are falling in supplemental results then the pages are not even crawled as often as the regular index.

I wouldn't get too hung up on the Supplemental results. Think of it as a holding area while Googlebot continues to make its way through your site and sort things out. I think it is a given these days that any large site is going to have a certain percentage of its pages in the Supplemental index.

Not hung up on supplemental. I'm aware of how it works and the issues with large sites and their percentage in supplemental index. But if the page does fall in supplemental and as you know its not crawled as frequently as other pages although their have been talks about Google will try to crawl more often then usual (but who knows). If I add new links on the letter then that link is not crawled until that page is crawled. Know what I mean.

I guess I still don't know if I should keep doing what I'm already doing or block the alpha pages from being indexed.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Accessibility and Usability
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved