Forum Moderators: Robert Charlton & goodroi
We recently launched a VBulletin Forum and seeing some indexing problems from "site:" operator and from Google Webmaster Tools.
Here is an example of the problem:
/forums/showthread.php?p=294
/forums/showthread.php?t=289
One is a t= and the other is a p=
Both are the same exact page with two different links. If we go to one, it starts at the top of the thread. If we go to the other link, it jumps to one of the posts.
GAH!
1) Don't like seeing two different links for the same page in site: operator
2) Don't like seeing "Duplicate Meta Description" or "Duplicate Title" in Webmaster Tools.
So far not much as been indexed as I have been watching it like a hawk since we went live. I would like to put a stop to this quickly.
Question:
Is it possible to put dynamic links into robots.txt to stop this?
If so, how should it look? Looks like there will be a big list of things we will want to put in the robots.txt if dynamic links are allowed.
If anyone knows a nice robots.txt file that cleans up VBulletin, that would be great. Otherwise, we might have to just robots.txt the entire forum to get the duplicate content under control now before it's a huge mess.
Thanks in advance for any help with this.
Kris
That said, you're asking a specific SEO question which IMO is rarely answered adequately by the forum/CMS makers out there.
You're in full control of any URL problems created by your software. You can fix any of them via an htaccess file on most hosts (like redirecting p= to t= or vice versa) and then correcting the links created by the software.
A robots exclusion file is essentially destroying value, not redistributing it, and so is not a good solution in many cases.
[added info]
You COULD block one version in robots.txt like this:
Disallow: /*?t=
But I wouldn't do that, I would read vBeulletin's forum and think about vbSEO. Hope that helps. PS, that would work on ALL SE's.