I have 3 urls for every page in vbulletin. How do I get rid of that?

[tech0925]

First of all I am brand new to this forum and looking forward to talking with everyone.

I am using seomoz which I am sure most of you are familiar with. I am also using vbulletin 4.0.3 for this forum. Seomoz pointed out that I have 3 different links for every page. As you know google penalizes you for this for having the same page content and title. I think it may be threaded, hybrid, and something else. Just guessing though. I also noticed that google is indexing the archive files. Should I block these? How do I get rid of this and only have one link per page? Also, which one should I keep?

This is just blowing my mind and any help would be so appreciated. I noticed another thread similar to this but I believe it was a little different as I couldn't get any help from it. If I missed a thread with this answer I apologize in advance.

Thank You!

[phranque]

welcome to WebmasterWorld, tech0925!

i'm not sure if vbulletin has fixed the specific canonicalization issues discussed here previously but here's a couple of threads that should give you some ideas.
Vbulletin - avoiding duplicate content in Google:
http://www.webmasterworld.com/google/3133336.htm [webmasterworld.com]
Huge decrease in number of forum pages indexed:
http://www.webmasterworld.com/forum30/33094.htm [webmasterworld.com]

in general, your solution will consist of some combination of:
- determining the canonical url for each page
- internally referring only to canonical urls
- 301 redirecting any non-canonical requests to the canonical url
- use a meta robots noindex element or "X-Robots-Tag: noindex" HTTP response header on non-canonical urls if none of the above works for your application, but this won't allow consolidation of link equity
- in some cases you might use the link rel canonical element, but this is Google-proprietary and can be dangerous if misapplied and therefore is less than ideal
- in worst case scenarios you might consider using robots.txt to exclude urls from being crawled

there's a vBSEO plugin available for a license fee that may have all or part of your solution, but i don't know any of the specifics.

[g1smd]

I am guessing this is the design fault I pointed out more than 4 years ago and which still isn't fixed.

I suspect it is the URLs with &goto=next and &goto=previous on the end.

They play havoc with vBulletin site indexing.

Use robots.txt to disallow them.

There's a very long thread, many years ago, with a lot of detail of what to block.

[phranque]

thanks for mentioning that, g1smd.
if i remember correctly, the short story is that the "next"/"previous" urls are non-canonical AND it's impossible to know what the canonical url would have been at the time that url was discovered.

[g1smd]

It should be very easy fix for VB to implement.

Imagine you are viewing

/showthread.php?topic=3333

at present. Instead of returning links to

/showthread.php?topic=3333&goto=next

and to

/showthread.php?topic=3333&goto=previous

vBulletin should simply return links pointing to

/showthread.php?topic=4444

and to

/showthread.php?topic=2222

.

Imagine topic 4444 is the very next topic from 3333 and is indexed as

/showthread.php?topic=3333&goto=next

.

The problem comes if topic 3333 and/or topic 4444 is/are posted to and they are no longer adjacent threads in the forum thread listings.

From that point on, clicking a link in the SERPs pointing to

/showthread.php?topic=3333&goto=next

may well take you to topic 8888, which will not contain the content you were expecting to see.

The fact that the content at these non-canonical URLs keeps changing and is often a duplicate of URLs already indexed, must be a major negative indexing factor for every vBulletin site.

With the URLs for every piece of content constantly changing, it is quite possible that many of the pages within the site are not indexed at all, while others are indexed under multiple URLs... many of which no longer point to the same content.

This is a very serious design flaw in vBulletin and I am beyond amazed that it hasn't been fixed many years ago.


Actually, I see the parameters are nowadays called

&view=old

and

&view=new

. Whatever, this still causes the same indexing problems. Additionally, there's an extension that can perform a 302 redirect to the correct URL. This is even more dangerous.

[tech0925]

Thanks for all the information everyone! I installed vbseo on my forum late last night and that seemed to fix the problem as far as I can tell. It now redirects those extra urls to one short version. It took care of some other things to.

I am just waiting on the next crawl from seomoz to see how much it fixed. Thanks again!

[g1smd]

Use the Live HTTP Headers extension for Firefox to check out those redirects.

They should return 301 status, not 302.

[tech0925]

Thanks for that info. I downloaded that and it said 301 moved permanently. =) Thank you everyone for your help!

Hey guys, I also started a new thread for the sitemap issues that I have been having. Do any of you know how to fix that too? Here is the link [webmasterworld.com...]

[phranque]

your sitemap validation issues are due to invalid url references on the site.
the technically correct solution is to use the canonical url in the pagination and internal navigation.
fixing the link with a rewrite is a bandage to stem the bleeding - a worst-case-scenario solution.