Forum Moderators: Robert Charlton & goodroi
I wish to avoid and duplicate penalties and make sure only one url points to each post. Any feedback on how you have managed this would be greatly appreciated.
As well, is there a static option or a rewrite option which makes the dynamic urls redirect possibly to static to avoid the issue entirely?
[webmasterworld.com...]
Particularly follow the posts from g1smd, he's done a lot of work in this area.
[edited by: tedster at 3:46 pm (utc) on Nov. 5, 2006]
Alternatively use robots.txt to exclude those URLs, but it is far better to do by using the meta tag.
Each thread has at least 10 URLs that can access it. Work on getting only ONE URL version indexed for all threads.
In particular, pay attention to the "previous" and "next" links and get those fixed. There are two alternative ways to do this. One is to modify those links so that they go through a 301 redirect that points to the correct thread number. That bit requires some extra scripting. The other is to calculate the correct thread number and include that in the link, instead of appending the &goto= parameter to the existing thread number, which creates duplicate content.
One is to modify those links so that they go through a 301 redirect that points to the correct thread number. That bit requires some extra scripting.
This is the one that interests me the most as it creates a different static page url with unique information.
Any ideas G on how do go about it, besides researching the apache forum like a madman!
If:
&goto parameter detected in the request
then:
calculate real thread number for that thread
and do:
send header "301 moved" along with the new URL to request
The forum script has to do it, because only the forum script can work out what the new canonical thread number will actually be.
That is, a link on a page says www.domain.com/showthread.php&t=12345&goto=nextnewest
When the link is clicked, the script sends back:
301 Moved; new URL is www.domain.com/showthread.php&t=45678
which the browser then automatically requests, as will bots.
Only drawback is its not open source. I waited a year before buying it because of this... now I wish I had installed it a year ago!
I'm not too sure if vbseo fixes all the duplicate content issues. I still see different URLs for the same content in internal links... but they may of course lead to a 301 redirect to the canonical form (preferred) or they might just have "nofollow" on them, which would hinder the bot in indexing the site.
I've managed to get it down to 1 URL per thread using nofollow and a good robots.txt. The problem I now face is supplementals for threads that are buried as much as 70 pages down on the forumdisplay pages. Is there anyway to resolve this effectively?
I would instead rely on noindex on the pages having URLs of the form that you don't want indexed. Google will still parse those pages and follow links to the rest of the site.
As long as the thread indexes and the links to previous and next pages can be followed with a canonical format for the URLs in them then Google should index the site just fine.
Don't impede the bot from traversing the site. Instead let it roam free and then tell it what not to index. Different methodology. Vastly different results.
My previous and next buttons are also fine. They point to showthread.php?t=#*$!xx which is the proper URL.
As to adding the meta tags to other ones (e.g when forumdisplay is ordered, or a thread is set to show as linear), well that's a lot more tricky and something I'm trying to work on.
I still have numerous supplementals of pages that are buried many pages deep in forumdisplay, and I'm really not sure how to fix that.
All that I need now is for google to get deep into those forumdisplay pages to index very old threads again to pickup the new descriptions and titles.
Work on everything that you can, and Google will eventually pick up on it.
I've made some more small modifications so that when links to member.php?find=lastposter only show to members. To guests it's just a word, no link. Hopefully this will serve a double purpose of reducing the number of links on each page (hence increasing the weight to the other, important links) and getting rid of needless redirects/nofollows.
Just wanted to say thanks very much g1smd. I've also posted a few times on the vbulletin.com forums for them to implement some changes and hopefully just give each piece of content 1 URL in future versions.
[edited by: Nick0r at 11:28 pm (utc) on Nov. 5, 2006]
Whatever you can do to give the bot easy access to all of your threads and thread indexes, while stopping them indexing all of the "junk" ("you are not logged in") pages, will help a lot.
[edited by: Nick0r at 11:53 pm (utc) on Nov. 5, 2006]
They have stated to me that their upgraded software only allows one indexed url per post and that all other fetched urls pointed to the same url.
I think more than one person was listening to g1smd :P
I'd rather sort out default vbulletin, rather than have to use a piece of software that performs a ton of changes to my forum.
One URL per post would be a disaster. It should be one URL per thread.
I still see some problems with vbseo: not just the use of underscores in URLs, but also the way that 301 redirects are harnessed to point to "relative" URLs (relative from thread to thread that is, like nextoldest, nextnewest, and most recent, etc).
I'd rather just nudge the vbulletin developers along with fixing the default.
Rumour has it in version 4 of vbulletin (however many months or years that may be away), they will do away with pointless things like showthread.php?p= and work on the URL canonicalization of the best forum software on the market.
New sites I've created with attention to dupe content have yet to run into any issues at all, so this is definitely the way to go.
We're not talking with wannabe SEOs in the vbulletin community now. We're talking the real deal.
This thread proves there are MAJOR issues with vbulletin, duplicate content, and URL structure.
I've personally battled this for many many months and I've just got it under grips if google is kind enough to reindex every thread on my site.
Suggestions have been made the vb team, lets pray they implement them soon.
[edited by: Nick0r at 4:14 am (utc) on Nov. 26, 2006]
--? -- I show examples and evidence that seem to demonstrate there isnt a dup content issue with vBulletin
[edited by: tedster at 8:15 am (utc) on Nov. 26, 2006]
I discovered issues with the original config file that created double slashes in the URLs, and even though the forum is now set up to use cookies, URLs with session IDs have been indexed by the SE spiders and those appear to be difficult to get rid of.
I've eliminated all other duplicate URLs from my internal linking structure, and the forum pages are now the only ones left (that I know of). So, we'll see if this makes any difference. Not having the forum indexed is fine by me if Google will restore the previous traffic levels to the rest of the site. What good is a forum if it pulls the rest of the site down, anyway?
Interesting.
And completely wrong.
Words fail me.
