| 8:21 pm on Oct 24, 2006 (gmt 0)|
Here's one of several threads on the topic.
Particularly follow the posts from g1smd, he's done a lot of work in this area.
[edited by: tedster at 3:46 pm (utc) on Nov. 5, 2006]
| 8:35 pm on Oct 24, 2006 (gmt 0)|
Get a meta robots noindex tag on ALL of the pages that would present an "Error. You are not logged in" message to bots. That includes any stuff like newthread.php, newreply.php, sendpm.php, login.php, and just about anything else that is not a thread or is not an index of threads.
Alternatively use robots.txt to exclude those URLs, but it is far better to do by using the meta tag.
Each thread has at least 10 URLs that can access it. Work on getting only ONE URL version indexed for all threads.
In particular, pay attention to the "previous" and "next" links and get those fixed. There are two alternative ways to do this. One is to modify those links so that they go through a 301 redirect that points to the correct thread number. That bit requires some extra scripting. The other is to calculate the correct thread number and include that in the link, instead of appending the &goto= parameter to the existing thread number, which creates duplicate content.
| 8:54 pm on Oct 24, 2006 (gmt 0)|
Thanks for all of the info.
|One is to modify those links so that they go through a 301 redirect that points to the correct thread number. That bit requires some extra scripting. |
This is the one that interests me the most as it creates a different static page url with unique information.
Any ideas G on how do go about it, besides researching the apache forum like a madman!
| 9:00 pm on Oct 24, 2006 (gmt 0)|
No you need the script to do something like:
&goto parameter detected in the request
calculate real thread number for that thread
send header "301 moved" along with the new URL to request
The forum script has to do it, because only the forum script can work out what the new canonical thread number will actually be.
That is, a link on a page says www.domain.com/showthread.php&t=12345&goto=nextnewest
When the link is clicked, the script sends back:
301 Moved; new URL is www.domain.com/showthread.php&t=45678
which the browser then automatically requests, as will bots.
| 12:27 am on Oct 25, 2006 (gmt 0)|
vbseo will accomplish this and much more with a single include in one php file and use of their .htaccess.
Only drawback is its not open source. I waited a year before buying it because of this... now I wish I had installed it a year ago!
| 12:32 am on Oct 25, 2006 (gmt 0)|
Hmm, but vbseo seems to be based more on pumping keywords in to URLs than anything else. That was something a few believed for a while sometime back, but I don't feel that it is a generally effective strategy today.
I'm not too sure if vbseo fixes all the duplicate content issues. I still see different URLs for the same content in internal links... but they may of course lead to a 301 redirect to the canonical form (preferred) or they might just have "nofollow" on them, which would hinder the bot in indexing the site.
| 11:33 am on Nov 3, 2006 (gmt 0)|
I've managed to get it down to 1 URL per thread using nofollow and a good robots.txt. The problem I now face is supplementals for threads that are buried as much as 70 pages down on the forumdisplay pages. Is there anyway to resolve this effectively?
| 3:50 pm on Nov 3, 2006 (gmt 0)|
I would not rely on "nofollow" for this. It impedes the bots from traversing your site, and those nofollow links are saying you are not voting for that page.
I would instead rely on noindex on the pages having URLs of the form that you don't want indexed. Google will still parse those pages and follow links to the rest of the site.
As long as the thread indexes and the links to previous and next pages can be followed with a canonical format for the URLs in them then Google should index the site just fine.
Don't impede the bot from traversing the site. Instead let it roam free and then tell it what not to index. Different methodology. Vastly different results.
| 6:42 pm on Nov 3, 2006 (gmt 0)|
noindex,follow is added to all "error, please login" pages by default. Should I now take off the nofollow from reply buttons etc?
My previous and next buttons are also fine. They point to showthread.php?t=#*$!xx which is the proper URL.
As to adding the meta tags to other ones (e.g when forumdisplay is ordered, or a thread is set to show as linear), well that's a lot more tricky and something I'm trying to work on.
I still have numerous supplementals of pages that are buried many pages deep in forumdisplay, and I'm really not sure how to fix that.
| 3:31 pm on Nov 5, 2006 (gmt 0)|
Ahhh, just discovered something. The pages marked supplemental are those that haven't been crawled in a very long time. And when they were crawled all that time ago, they had terrible meta description and title tags. They showed very little uniqueness which lead to them being supplemental.
All that I need now is for google to get deep into those forumdisplay pages to index very old threads again to pickup the new descriptions and titles.
| 7:02 pm on Nov 5, 2006 (gmt 0)|
That may well take several months or more.
Work on everything that you can, and Google will eventually pick up on it.
| 11:27 pm on Nov 5, 2006 (gmt 0)|
OK I look forward to it.
I've made some more small modifications so that when links to member.php?find=lastposter only show to members. To guests it's just a word, no link. Hopefully this will serve a double purpose of reducing the number of links on each page (hence increasing the weight to the other, important links) and getting rid of needless redirects/nofollows.
Just wanted to say thanks very much g1smd. I've also posted a few times on the vbulletin.com forums for them to implement some changes and hopefully just give each piece of content 1 URL in future versions.
[edited by: Nick0r at 11:28 pm (utc) on Nov. 5, 2006]
| 11:31 pm on Nov 5, 2006 (gmt 0)|
That also sounds like a reasonable idea.
Whatever you can do to give the bot easy access to all of your threads and thread indexes, while stopping them indexing all of the "junk" ("you are not logged in") pages, will help a lot.
| 11:52 pm on Nov 5, 2006 (gmt 0)|
g1smd, I found out yesterday after some digging in my templates, that in vbulletin 3.6 by default, meta noindex,follow is put on all 'error you are not logged in pages'.
[edited by: Nick0r at 11:53 pm (utc) on Nov. 5, 2006]
| 12:05 am on Nov 6, 2006 (gmt 0)|
Heh, someone was listening after all. :-)
| 5:52 am on Nov 6, 2006 (gmt 0)|
I emailed the company referred to earlier regarding the optimizing of the dupe issues.
They have stated to me that their upgraded software only allows one indexed url per post and that all other fetched urls pointed to the same url.
I think more than one person was listening to g1smd :P
| 10:04 am on Nov 6, 2006 (gmt 0)|
If you're talking about vbseo, then yes. But that's not really a good idea for long-standing vbulletin forums, suddenly changing 10's of 1000's of URLs is a risk to say the least, one that I'm not prepared to take.
I'd rather sort out default vbulletin, rather than have to use a piece of software that performs a ton of changes to my forum.
| 7:33 pm on Nov 6, 2006 (gmt 0)|
>> one indexed url per post and that all other fetched urls pointed to the same url <<
One URL per post would be a disaster. It should be one URL per thread.
I still see some problems with vbseo: not just the use of underscores in URLs, but also the way that 301 redirects are harnessed to point to "relative" URLs (relative from thread to thread that is, like nextoldest, nextnewest, and most recent, etc).
| 7:57 pm on Nov 6, 2006 (gmt 0)|
Yea vbseo isn't ideal.
I'd rather just nudge the vbulletin developers along with fixing the default.
Rumour has it in version 4 of vbulletin (however many months or years that may be away), they will do away with pointless things like showthread.php?p= and work on the URL canonicalization of the best forum software on the market.
| 9:57 pm on Nov 6, 2006 (gmt 0)|
My pickle is a bit different as I am looking for the correct way to start with the installation and indexing in preventative way to never let the problem start at the beginning.
New sites I've created with attention to dupe content have yet to run into any issues at all, so this is definitely the way to go.
| 11:30 pm on Nov 6, 2006 (gmt 0)|
I wrote several posts here both a few months ago, and about a year (or more) ago, outlining the main problems with vBulletin.
| 2:32 pm on Nov 7, 2006 (gmt 0)|
Cain, if that's your case. Definitely go with vbseo. On a new forum I started I've used it and haven't had any problems so far.
| 12:46 am on Nov 26, 2006 (gmt 0)|
You dont need vBseo or anything else for that matter. I posted a few articles on Duplicate Content on "vBulletin FAQ" - basically showing that there are no duplicate content issues with vBulletin whatsoever. If you want the proof look up the thread there.
| 4:14 am on Nov 26, 2006 (gmt 0)|
joey have you even read ANY of this thread?
We're not talking with wannabe SEOs in the vbulletin community now. We're talking the real deal.
This thread proves there are MAJOR issues with vbulletin, duplicate content, and URL structure.
I've personally battled this for many many months and I've just got it under grips if google is kind enough to reindex every thread on my site.
Suggestions have been made the vb team, lets pray they implement them soon.
[edited by: Nick0r at 4:14 am (utc) on Nov. 26, 2006]
| 5:30 am on Nov 26, 2006 (gmt 0)|
Did you read my article on vBfaq?
--? -- I show examples and evidence that seem to demonstrate there isnt a dup content issue with vBulletin
| 5:36 am on Nov 26, 2006 (gmt 0)|
Actually there is a dupe issue with VB out of the box with shownext posts and multiple urls that access the exact same content.
| 6:12 am on Nov 26, 2006 (gmt 0)|
I disagree - In my article I even show the same thread indexed and ranked in the various formats.
[edited by: tedster at 8:15 am (utc) on Nov. 26, 2006]
| 5:27 pm on Nov 26, 2006 (gmt 0)|
I just disallowed my /forum/ in robots.txt due to duplicate content issues. My site has suffered a 90% loss of traffic from Google since late 2004, which I now believe might be due to duplicate content from a forum I added about 2 months before the drop in traffic.
I've eliminated all other duplicate URLs from my internal linking structure, and the forum pages are now the only ones left (that I know of). So, we'll see if this makes any difference. Not having the forum indexed is fine by me if Google will restore the previous traffic levels to the rest of the site. What good is a forum if it pulls the rest of the site down, anyway?
| 5:59 pm on Nov 26, 2006 (gmt 0)|
>> showing that there are no duplicate content issues with vBulletin whatsoever <<
And completely wrong.
Words fail me.
| This 48 message thread spans 2 pages: 48 (  2 ) > > |