Moving Forums to Separate Domain

[peego]

Hello,

We're planning to move our forums to a separate domain by itself. Can I get some suggestions on the best way to move our forums to a separate domain?
- Should I get current forum content removed from google 1st, then afterwards, load it up to the new domain? What's the best course of action to do this?
- How do I remove all the current forum contents from google?

The old forum will be closed down/removed, and put up on a new domain. But I don't want to have duplicated content issues or get in trouble with google.

Thanks for any suggestions and recommendations.

[toplisek]

How do I remove all the current forum contents from google? Check admin files for forum.
Which form do you use?

Set htaccess redirection of domain. This will help Google to see new content and works...

[cr1t1calh1t]

peego- I am curious about why you are moving the forum to another domain; are you moving it because of 'thin' pages, spammers, or Panda related concerns?

I've been trying to figure out what to do with the forum on my site, and have considered deleting it or moving it to an entirely new domain as well. I haven't felt much effect from Panda until the October surprise, and have been going through my site with fine tooth comb and the forum looks like a big problem, but I don't know where to start...

For years I fought a losing battle against spam - and finally just closed down new registrations - so the forum is now sitting dormant. While it was up, it never really got any traction - and all the forum type action focused itself in the comments on the content pages. Because of this - I'm certain that the forum is just a huge 'thin' wasteland that's dragging down the rest of the site.

Content wise, and traffic wise, the forum is not a big part of my site, but I am reluctant to make such a drastic change as deleting it or moving it, for a number of reasons:
1. because of all of the dynamic URLs that vBulletin creates, and the fact that I have navigation links on all of the forum pages to everywhere else on my site. This would be a massive disruption to my internal link profile.
2. lots of spam links directed to my forum. I don't want to associate with these links, but I am still concerned about what kind of disruption, if any, the disappearance of thousands of inbound links to my domain would have.
3. technically, I'm unsure of the best way to do move the forum and handle the all the redirection I'd have to do
4. after reading your post peego - now I'm worried about the duplicate content issues if I moved the forum and how to avoid them.

peego - I'm not trying to hijack your thread, but I wanted to ask you about the reasoning behind your decision to move, and put my situation out there and hopefully get some insight into my own predicament as well.

[peego]

CR1t1,

I'm in a somewhat similar situation as you. Our site is a pretty old side, about 10yrs now. Got demolished by panda feb, and then april again. It held stead after the april hit and the past few weeks has been steadily trending lower again by about another 15-20%... I read an interesting comment in one of the other threads here yesterday, where one member said something along the lines of: "inactive forums are a big meal for a panda". Forgot which thread it was. So that got me started thinking to move my forum or not. It's a decent sized forum, but has become inactive the past year or two, and the number of pages it contains is far, far greater than the content pages we have on the main site. So perhaps the forums might be a liability in terms of how panda sees the site overall? That's what got me thinking of moving it to a separate domain.

Fortunately, we don't have a whole lot of links from the forums pointing to the main site, but its the other way around. Our main site pages all have a link that points to the forum index page (in the top navigation). I don't know how moving the forum will impact the overall site, but with how bad things are already, I figure what's the worst could happen?

My main concern/question is what's the best way of moving our forum to a new domain?

Should I noindex and disallow gbot from our forums, wait for it's all removed from the google index? and then move it over to new domain and let it get spidered? Should I point all my current links on our main site content pages to point to the new domain where the forum will be? Or simply remove all those links? Can anyone please give some suggestions?

[tangor]

I'm in the category of "if it ain't broke, don't fix it"... the SE's have been crawling those forums for some time... why confuse them? Why make them want to do Panda in the first place? (Panda, contrary to all reports, is about REDUCING THE NUMBER OF PAGES G HAS TO INDEX (and did that by applying all kinds of filters against duplicate content, ie. used to appear on a site, then a sub directory, then in a sub domain, then on scraper sites, then golly ye gollies multiplied over and over until we have pulled out our last hair)... so why move it?

What every newbie webmaster needs to know is how to do it right first... and that just isn't going to happen... so ugly uri, lame-0 scripting, etc... unless it is REALLY WRONG, should be left alone. Concentrate on doing the REST OF YOUR LIFE the correct way, as you learn... then LITTLE BY LITTLE go back and clean up a bit.

Some things you can't fix... and Panda is not one of those... can MITIGATE, but can't fix it because Google, Bing, Yahoo, Alta Vista and all the rest (which are still running) have NEVER FORGOTTEN a uri they met... and had memories like the proverbial elephant.

Which results in humongous redirects FOREVER... or just leave it alone and move on.

(When we are young and stupid we make young and stupid mistakes because we didn't find Webmasterworld FIRST and learn the correct way. But, like all young and stupids, we do grow into adults, so that's the part we need to work on... the part that GOES FORWARD.)

[peego]

What part of my post makes you assume that I've led the previous year of life the "wrong way". What part of my post gives you the impression that I'm young and stupid? What part of my post gives you the impression that I'm not an adult and need grow into an adult?

Please, no need to be so condescending. If you have nothing to contribute, then move along.

I'd appreciate any suggestions anyone can offer.

[tangor]

What part of my post suggests condescension? The opening line "if it ain't broke, don't fix it" is appropriate... unless you actually have something broken. Do you? All of your begs of assistance indicate all the reasons why one shouldn't do it. :)

That said, I DID miss the statement "put on a new domain", so what's the worry? It's a new domain and, unless you continue the old domain to redirect, is self-resolving (404)

(final edit) The "young and stupid" is not directed at age... but as new in anything which so many webmaster and newbies are... and if I offended in preaching the choir, I apologize.

[potentialgeek]

You may be better off just noindexing the thin content and not moving anything.

[Pjman]

I Have been considering blocking google bot via robots.txt, just for my forum. It does consist of 1/3 of all my sites pages and is pretty inactive.

Has anybody no indexed or blocked their forum from being indexed and seen an improvement in rankings?

I got hit by panda on panda 1.0 (feb) I have seen little to no improvement even after removing or no indexing all thin content on the entire site.

[Donna]

noindex and u will be ok

[cr1t1calh1t]

@peego - thanks for the insight into why your doing this - we are in a similar situation. good luck on bouncing back.

@potentialgeek - thanks for the noindex idea. I think that's exactly the solution I've been looking for. Simple to implement, but achieves the same radical results (hopefully...) with the least pain.

I just noindex,follow'd the entire forum, except for the main forum index page, which I set to index,follow

if you're using vbulletin (my version is 3.8.1), you may find the code I used helpful:

<if condition="in_array(THIS_SCRIPT, array('index'))">
<meta name="robots" content="index, follow">
<else />
<meta name="robots" content="noindex, follow">
</if>

I just inserted this code in the headinclude portion of the default template, found here:
Styles & Templates>Style Manager>Default Style>headinclude

[peego]

Thanks to all for the suggestions. I've put noindex on the forum.

Just wondering if I should also block gbot from the forums via robots.txt as well? Would it matter?

[ken_b]

Just wondering if I should also block gbot from the forums via robots.txt as well? Would it matter?

Yes it matters. If you do that, the bots won't be able to see the noindex on the page.

[Pjman]

Has anyone heard of Pre-Panda recoveries specifically or partial from reworking or noindexing forums? I haven't yet, but it's my last idea on how to fix the problem.


@cr1t1calh1t

Thanks for the VB code. That's the same version I'm using.

[Donna]

You will need to wait to see some results once you noindex. Content your left over will be re-scored I guess and re-ranked, you might end up with variety of different keywords . For pandalized site ETA on recovery is still unknown but imo it takes minimum of 2 panda refreshes to conclude any positive movements .

[phranque]

<if condition="in_array(THIS_SCRIPT, array('index'))">
<meta name="robots" content="index, follow">
<else />
<meta name="robots" content="noindex, follow">
</if>

http://googlewebmastercentral.blogspot.com/2007/03/using-robots-meta-tag.html [googlewebmastercentral.blogspot.com]:

Unnecessary content values
By default, Googlebot will index a page and follow links to it. So there's no need to tag pages with content values of INDEX or FOLLOW.

[cr1t1calh1t]

@Donna your point is well taken, but I'm not really sure if my problems are 100% because of panda...

when panda was first rolled out, my traffic was in flux for 2-4 weeks. My traffic dropped, but the drop and other ranking and traffic changes didn't convince me that I had been 'pandalized', but rather was just caught up in the big shifts caused by others getting pandalized. So, I just sat back and started collecting additional data. It came back even stronger than pre-panda around the time of the next iteration.

In the meantime, I'm pretty sure I may have broken my htaccess, introducing a boatload of duplicate content, as well as breaking my comments system, resulting in all comments rendering in javascript, becoming invisible to googlebot and disappearing from the index. Joomla is a nightmare...

This time, around the oct. 14 iteration - my traffic and adsense revenue went into flux again, with a bigger drop - and I did exactly the opposite, since I discovered the dupe content and comments problems. I did a complete site overhaul and have been making some very dramatic changes on sitewide basis... All those major changes, along with the panda cycle - there's no telling when I'll be able to make any conclusions

That said, I'll check back into this thread when I've got something to report re: noindexing the forum.

[phranque]

welcome to WebmasterWorld, cr1t1calh1t!

[Pjman]

Noindexing the forum is starting to make a great deal of sense now.

After thinking back to one commonality that I heard from everyone that had recovered from the 1st Panda. "I worked and corrected all the errors I found in Google WMT."

In my WMT 98% of the errors are from the forum. Mostly duplicate meta, title tags, or missing threads/posts that I removed due to spam.

Fingers crossed.

[peego]

Most of my WMT errors are in our forum as well.

[cr1t1calh1t]

@phranque - hey, thanks - glad to be here. Good lookin' out on the unnecessary content values. I'd be interested to read about negative effects observed from using INDEX or FOLLOW. I probably have all of my pages tagged unnecessarily with 'INDEX,FOLLOW', so will start removing those to see if I can correlate any effect to it. With all the changes I've been making lately, its doubtful...

I've been watching GWT for my noindexed forum pages to drop out, which I know is almost like waiting in July for Christmas to come, but I realized that I had whole ranges of forum directories blocked via robots.txt, and that I needed to allow the googlebot to get in there to see that noindex was set.

So, just a reminder to anyone who is noindexing a whole forum, or large ranges of a site (or even a single page), make sure you double check you haven't blocked bots from visiting those pages via robots.txt.

[phranque]

I'd be interested to read about negative effects observed from using INDEX or FOLLOW. I probably have all of my pages tagged unnecessarily with 'INDEX,FOLLOW', so will start removing those to see if I can correlate any effect to it.

i doubt there is any direct or significant negative affect.
at most it may be seen as a signal of poor quality and adds to code bloat.
i was merely pointing out that it is unnecessary.

the disallow/noindex is a common problem - often implemented by those who see it as a belt-and-suspenders solution.

[kidder]

I wonder what would happen to the traffic on this forum if you noindexed it :) Am I missing something here? Is this not chasing the alog? Are you looking after your users?

[cr1t1calh1t]

My forum is dead anyway.... It never gained any traction with users and was overrun by spammers. To thwart the spammers I incrementally tightened the screws on the forum, until I eventually disabled all posting and all new user registrations.

In some sense, I am chasing the algo - because I am trying to counter the possibility that google sees the forum as just a stagnant no-man's-land of 'thin content'.

As for looking after my users - that has always been my priority. The forum-like social interaction happens in the comment section. I promoted the comment section and forum equally, and the users sent me a message loud and clear - they wanted to interact in the comments, and the comment sections on my site are thriving.

[tedster]

Then it sounds like the forum should be 410 Gone, right?

[Pjman]

Any updates?

I implemented a non index to my inactive forum (Pandalized site) 3 weeks ago. 75% of it is still indexed by Google. No real ranking changes yet.

[Planet13]

Off Topic:

I am surprised that forums are still having such problems with spam. I would assume that most forums require a valid email address that the user has to validate before being able to post a topic or create a profile page.

Also, I would imagine that most forum software would be able to reject posts / profiles that have links in them.

It seems like most of the forums I go to have these safeguards built in.

[enigma1]

They do validate emails but once someone gains access he can post (maintain the session and automate it) and the automatic security thereafter is minimal. For the most part, you need good forum moderation meaning manual intervention.

Since you are having a database for the forums, you can create a map of the threads and then issue 301 redirects from the threads of the old forum to the new one. Assuming you want to keep the old content along with some rank from the old pages.

[thudufushi]

Why would you wish to noindex a forum instead of simply disallowing it in robots.txt, what is the perceived advantage?

[cr1t1calh1t]

@thudufushi - in my situation the forum was already indexed, and I wanted to get it removed. If I used robots.txt to block it then the bot would not access those pages and see the noindex directive.

My rationale for not blocking it in robots.txt, even after it has all been noindexed is that don't want to lose all of the internal links from those pages, since they are such a high percentage of the total internal links for this site.

I don't know what effect it would have to block the googlebot from seeing over 95% of your internal links, after they were already discovered and a part of the rankings for your site - but my guess is that it would be negative, and quite possibly to the extreme.

cr1t1cal