homepage Welcome to WebmasterWorld Guest from 174.129.163.183
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Visit PubCon.com
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 48 message thread spans 2 pages: 48 ( [1] 2 > >     
Vbulletin - avoiding duplicate content in Google
need some tips before installation
CainIV




msg:3133338
 5:24 pm on Oct 24, 2006 (gmt 0)

Hi guys, in lieu of duplicate content and canonicalization, I would like some feedback from those who have experience with Vbulletin installations on their sites.

I wish to avoid and duplicate penalties and make sure only one url points to each post. Any feedback on how you have managed this would be greatly appreciated.

As well, is there a static option or a rewrite option which makes the dynamic urls redirect possibly to static to avoid the issue entirely?

 

tedster




msg:3133562
 8:21 pm on Oct 24, 2006 (gmt 0)

Here's one of several threads on the topic.

[webmasterworld.com...]

Particularly follow the posts from g1smd, he's done a lot of work in this area.

[edited by: tedster at 3:46 pm (utc) on Nov. 5, 2006]

g1smd




msg:3133578
 8:35 pm on Oct 24, 2006 (gmt 0)

Get a meta robots noindex tag on ALL of the pages that would present an "Error. You are not logged in" message to bots. That includes any stuff like newthread.php, newreply.php, sendpm.php, login.php, and just about anything else that is not a thread or is not an index of threads.

Alternatively use robots.txt to exclude those URLs, but it is far better to do by using the meta tag.

Each thread has at least 10 URLs that can access it. Work on getting only ONE URL version indexed for all threads.

In particular, pay attention to the "previous" and "next" links and get those fixed. There are two alternative ways to do this. One is to modify those links so that they go through a 301 redirect that points to the correct thread number. That bit requires some extra scripting. The other is to calculate the correct thread number and include that in the link, instead of appending the &goto= parameter to the existing thread number, which creates duplicate content.

CainIV




msg:3133601
 8:54 pm on Oct 24, 2006 (gmt 0)

Thanks for all of the info.

One is to modify those links so that they go through a 301 redirect that points to the correct thread number. That bit requires some extra scripting.

This is the one that interests me the most as it creates a different static page url with unique information.

Any ideas G on how do go about it, besides researching the apache forum like a madman!

g1smd




msg:3133606
 9:00 pm on Oct 24, 2006 (gmt 0)

No you need the script to do something like:

If:
&goto parameter detected in the request
then:
calculate real thread number for that thread
and do:
send header "301 moved" along with the new URL to request

The forum script has to do it, because only the forum script can work out what the new canonical thread number will actually be.

That is, a link on a page says www.domain.com/showthread.php&t=12345&goto=nextnewest
When the link is clicked, the script sends back:
301 Moved; new URL is www.domain.com/showthread.php&t=45678
which the browser then automatically requests, as will bots.

motorhaven




msg:3133789
 12:27 am on Oct 25, 2006 (gmt 0)

vbseo will accomplish this and much more with a single include in one php file and use of their .htaccess.

Only drawback is its not open source. I waited a year before buying it because of this... now I wish I had installed it a year ago!

g1smd




msg:3133792
 12:32 am on Oct 25, 2006 (gmt 0)

Hmm, but vbseo seems to be based more on pumping keywords in to URLs than anything else. That was something a few believed for a while sometime back, but I don't feel that it is a generally effective strategy today.

I'm not too sure if vbseo fixes all the duplicate content issues. I still see different URLs for the same content in internal links... but they may of course lead to a 301 redirect to the canonical form (preferred) or they might just have "nofollow" on them, which would hinder the bot in indexing the site.

Nick0r




msg:3144811
 11:33 am on Nov 3, 2006 (gmt 0)

g1smd,

I've managed to get it down to 1 URL per thread using nofollow and a good robots.txt. The problem I now face is supplementals for threads that are buried as much as 70 pages down on the forumdisplay pages. Is there anyway to resolve this effectively?

g1smd




msg:3145077
 3:50 pm on Nov 3, 2006 (gmt 0)

I would not rely on "nofollow" for this. It impedes the bots from traversing your site, and those nofollow links are saying you are not voting for that page.

I would instead rely on noindex on the pages having URLs of the form that you don't want indexed. Google will still parse those pages and follow links to the rest of the site.

As long as the thread indexes and the links to previous and next pages can be followed with a canonical format for the URLs in them then Google should index the site just fine.

Don't impede the bot from traversing the site. Instead let it roam free and then tell it what not to index. Different methodology. Vastly different results.

Nick0r




msg:3145279
 6:42 pm on Nov 3, 2006 (gmt 0)

noindex,follow is added to all "error, please login" pages by default. Should I now take off the nofollow from reply buttons etc?

My previous and next buttons are also fine. They point to showthread.php?t=#*$!xx which is the proper URL.

As to adding the meta tags to other ones (e.g when forumdisplay is ordered, or a thread is set to show as linear), well that's a lot more tricky and something I'm trying to work on.

I still have numerous supplementals of pages that are buried many pages deep in forumdisplay, and I'm really not sure how to fix that.

Nick0r




msg:3146568
 3:31 pm on Nov 5, 2006 (gmt 0)

Ahhh, just discovered something. The pages marked supplemental are those that haven't been crawled in a very long time. And when they were crawled all that time ago, they had terrible meta description and title tags. They showed very little uniqueness which lead to them being supplemental.

All that I need now is for google to get deep into those forumdisplay pages to index very old threads again to pickup the new descriptions and titles.

g1smd




msg:3146699
 7:02 pm on Nov 5, 2006 (gmt 0)

That may well take several months or more.

Work on everything that you can, and Google will eventually pick up on it.

Nick0r




msg:3146847
 11:27 pm on Nov 5, 2006 (gmt 0)

OK I look forward to it.

I've made some more small modifications so that when links to member.php?find=lastposter only show to members. To guests it's just a word, no link. Hopefully this will serve a double purpose of reducing the number of links on each page (hence increasing the weight to the other, important links) and getting rid of needless redirects/nofollows.

Just wanted to say thanks very much g1smd. I've also posted a few times on the vbulletin.com forums for them to implement some changes and hopefully just give each piece of content 1 URL in future versions.

[edited by: Nick0r at 11:28 pm (utc) on Nov. 5, 2006]

g1smd




msg:3146848
 11:31 pm on Nov 5, 2006 (gmt 0)

That also sounds like a reasonable idea.

Whatever you can do to give the bot easy access to all of your threads and thread indexes, while stopping them indexing all of the "junk" ("you are not logged in") pages, will help a lot.

Nick0r




msg:3146853
 11:52 pm on Nov 5, 2006 (gmt 0)

g1smd, I found out yesterday after some digging in my templates, that in vbulletin 3.6 by default, meta noindex,follow is put on all 'error you are not logged in pages'.

[edited by: Nick0r at 11:53 pm (utc) on Nov. 5, 2006]

g1smd




msg:3146863
 12:05 am on Nov 6, 2006 (gmt 0)

Heh, someone was listening after all. :-)

CainIV




msg:3147039
 5:52 am on Nov 6, 2006 (gmt 0)

I emailed the company referred to earlier regarding the optimizing of the dupe issues.

They have stated to me that their upgraded software only allows one indexed url per post and that all other fetched urls pointed to the same url.

I think more than one person was listening to g1smd :P

Nick0r




msg:3147183
 10:04 am on Nov 6, 2006 (gmt 0)

If you're talking about vbseo, then yes. But that's not really a good idea for long-standing vbulletin forums, suddenly changing 10's of 1000's of URLs is a risk to say the least, one that I'm not prepared to take.

I'd rather sort out default vbulletin, rather than have to use a piece of software that performs a ton of changes to my forum.

g1smd




msg:3147775
 7:33 pm on Nov 6, 2006 (gmt 0)

>> one indexed url per post and that all other fetched urls pointed to the same url <<

One URL per post would be a disaster. It should be one URL per thread.

I still see some problems with vbseo: not just the use of underscores in URLs, but also the way that 301 redirects are harnessed to point to "relative" URLs (relative from thread to thread that is, like nextoldest, nextnewest, and most recent, etc).

Nick0r




msg:3147803
 7:57 pm on Nov 6, 2006 (gmt 0)

Yea vbseo isn't ideal.

I'd rather just nudge the vbulletin developers along with fixing the default.

Rumour has it in version 4 of vbulletin (however many months or years that may be away), they will do away with pointless things like showthread.php?p= and work on the URL canonicalization of the best forum software on the market.

CainIV




msg:3147949
 9:57 pm on Nov 6, 2006 (gmt 0)

My pickle is a bit different as I am looking for the correct way to start with the installation and indexing in preventative way to never let the problem start at the beginning.

New sites I've created with attention to dupe content have yet to run into any issues at all, so this is definitely the way to go.

g1smd




msg:3148042
 11:30 pm on Nov 6, 2006 (gmt 0)

I wrote several posts here both a few months ago, and about a year (or more) ago, outlining the main problems with vBulletin.

Nick0r




msg:3148689
 2:32 pm on Nov 7, 2006 (gmt 0)

Cain, if that's your case. Definitely go with vbseo. On a new forum I started I've used it and haven't had any problems so far.

joeychgo




msg:3168214
 12:46 am on Nov 26, 2006 (gmt 0)

You dont need vBseo or anything else for that matter. I posted a few articles on Duplicate Content on "vBulletin FAQ" - basically showing that there are no duplicate content issues with vBulletin whatsoever. If you want the proof look up the thread there.

Nick0r




msg:3168306
 4:14 am on Nov 26, 2006 (gmt 0)

joey have you even read ANY of this thread?

We're not talking with wannabe SEOs in the vbulletin community now. We're talking the real deal.

This thread proves there are MAJOR issues with vbulletin, duplicate content, and URL structure.

I've personally battled this for many many months and I've just got it under grips if google is kind enough to reindex every thread on my site.

Suggestions have been made the vb team, lets pray they implement them soon.

[edited by: Nick0r at 4:14 am (utc) on Nov. 26, 2006]

joeychgo




msg:3168315
 5:30 am on Nov 26, 2006 (gmt 0)

Did you read my article on vBfaq?

--? -- I show examples and evidence that seem to demonstrate there isnt a dup content issue with vBulletin

CainIV




msg:3168318
 5:36 am on Nov 26, 2006 (gmt 0)

Actually there is a dupe issue with VB out of the box with shownext posts and multiple urls that access the exact same content.

joeychgo




msg:3168323
 6:12 am on Nov 26, 2006 (gmt 0)

I disagree - In my article I even show the same thread indexed and ranked in the various formats.

[edited by: tedster at 8:15 am (utc) on Nov. 26, 2006]

AndyA




msg:3168578
 5:27 pm on Nov 26, 2006 (gmt 0)

I just disallowed my /forum/ in robots.txt due to duplicate content issues. My site has suffered a 90% loss of traffic from Google since late 2004, which I now believe might be due to duplicate content from a forum I added about 2 months before the drop in traffic.

I discovered issues with the original config file that created double slashes in the URLs, and even though the forum is now set up to use cookies, URLs with session IDs have been indexed by the SE spiders and those appear to be difficult to get rid of.

I've eliminated all other duplicate URLs from my internal linking structure, and the forum pages are now the only ones left (that I know of). So, we'll see if this makes any difference. Not having the forum indexed is fine by me if Google will restore the previous traffic levels to the rest of the site. What good is a forum if it pulls the rest of the site down, anyway?

g1smd




msg:3168600
 5:59 pm on Nov 26, 2006 (gmt 0)

>> showing that there are no duplicate content issues with vBulletin whatsoever <<

Interesting.

And completely wrong.

Words fail me.

This 48 message thread spans 2 pages: 48 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved