Welcome to WebmasterWorld Guest from 188.8.131.52
Have you noticed similar experiences with you site or forum? (by the way we use vbulletin).
Since your traffic sank, though, that's probably not the only factor at work. Can you tell anything about what kind of pages were dropped?
I'm sure it's the main factor. I agree, vbulletin creates a lot of duplicate urls, because we have 90.000 post, and 8000 threads and that generated (two weeks ago)350.000 pages in google!.
My guess it's that new big daddy algo is filtering duplicate content, as a collateral effect, unfortunately, our traffic is 50% of what we has two weeks ago
Not only do they present duplicate content, but as each thread is bumped the same URL does not point to the same thread any more.
Think about it.
That is a major design flaw with vbulletin, one that they seem in no hurry to fix. There are thousands of poorly indexed forums out there running that software (and other software with similar major flaws).
The correct response for those links should be to next issue a 301 redirect to the correct thread number. That would mean that the URLs with the &goto parameter would never be indexed, so it would not matter that they pointed to ever changing content. Their presence would still allow easy site indexing without confusing the bots.
Ideally you want just the thread or topic lists and the individual threads indexed, and nothing else (maybe member profile pages, but that invites "forum profile link drop spammers" even more).
You can also help things along by using the robots.txt file in a sensible manner:
Here's one I made earlier:
# Vbulletin Robots File.
# Allow: /forum/forumdisplay.php?f=
# Allow: /forum/forumdisplay.php
# Allow: /forum/showthread.php?t=
# Allow: /forum/showthread.php
# Allow: /forum/profile.php
# Allow: /forum/showprofile.php
# Allow: /forum/announcement.php
# Allow: /forum/faq.php
# Allow: /forum/index.php
There are many flaws within vbulletin itself, that can only be fixed by the authors. There are certain pages that should have <meta name="robots" content="noindex"> by default, and the design should take more care to present one canonical URL for each piece of content.
You can never stop learning at this game. Here's three tips I learnt this week.
1. Change the topic header to H1 tags.
phpbb just displays the topic header in a td but changing it to H1 should help a lot
2. Place rel="nofollow" on ALL href's in all templates.
This will stop the SE's wandering about.
3. If you can, serve a different template based on the UA.
If you use something like the eXtreme Styles mod you can set it to serve a different template to bots. Regular visitors read the properly formatted page with links here and there and SE's will read a plain unformatted page rich in content, low on links and with zero graphics.
I don't think this is a penalty situation as the content is exactly the same for SE and viewer.
and that is without introducing URLs that include the page parameter, for threads that are more than one page long, and the pp parameter for changing the default number of posts per page; either or both of which can be added to most of the URLs above too.
It is important to keep as many of those out as possible other than the basic /forum/showthread.php?t=54321 version.
Another big problem is the "next" and "previous" links that cause massive duplicate content issues because they allow a thread like
/forum/showthread.php?t=54321 to be indexed as
/forum/showthread.php?t=34567&goto=nextnewest and as
Additionally if any of the three threads is bumped, the "next" and "previous" links that are indexed no longer point to the same thread, because they contain the thread number of the thread that they were ON (along with the goto parameter), not the real thread number of the thread that they actually pointed to.
This is a major programming error by the people that designed the forum software. The link should either contain the true thread number of the thread that it points to, or else clicking the "next" and "previous" links should go via a 301 redirect to a URL that includes the real true canonical thread number of the target thread.
[edited by: tedster at 1:57 am (utc) on Mar. 31, 2006]
[edit reason] splice from another spot [/edit]