Deciding what belongs, what redirects, and what is 404

[berrysharpie]

Hello,

We have a 13 year old site that used to rank quite well, we were competitive enough, but now is struggling. It contained of a lot of content, photo galleries, a store, etc. We've been building the content slowly for 13 years.

What we did a few months ago was break down the site into subdomains which some have helped, some have not.

We went from:

www.domain.com to

www.domain.com
subdomain1.domain.com
subdomain2.domain.com
subdomain3.domain.com
subdomain4.domain.com
subdomain5.domain.com

While say, subdomain 1 - 4 are in fact benefiting from being moved, subdomain 5 and the main site www.domain.com are bouncing and struggling. Of course, subdomain 5 and the main site are the ones we really need functioning. Subdoman 5 is the store and www.domain.com is the bulk of the content.

We are now starting to believe that the content from the main site and the store were actually living a little more harmoniously than we had originally realized. Without the store, the size of the site has dropped rather abruptly but the products sold were also nice matches to the topic at hand on the main site.

So, what we are now considering, is moving the store BACK to the main site.

This is where our questions come in:

1) Do we wait it out as it has only been a few months or since the other subdomains have responded to the change but subdomain 5 has not, just go ahead and try to fix it now.

2) If we do go ahead and move it back, since it is on a subdomain and it isn't ranking, can we go ahead and issue 404's rather than 301's and make a new start on subdomain 5. The reason I ask this one is because sometimes when you do 301's, I believe you are just dragging along something the bot might have been confused about or not liked.

3) If we do go with 404's rather than 301's, what about the original 301's we issued to move the store from the main site to the subdomain.

4) If we do go with 404's, we would end out with thousands of 404's. 13 years of changes and redirects. If having thousands of 404's to finally flush the pages, is not a ad thing, that might be a nice option to finally let go of year old to 13 year old junk. Over time, pages change so much that old 301's no longer match what they point to so I figure it could hurt over time. A big flush might be nice while we are already kind of down for the count.

I hope this all makes sense and I can clarify if I am not being clear about something. Any thoughts or advice would be greatly appreciated. It would be nice to get this site purring again like it used to. It's our baby and we can't just let it go down without a fight.

Thank you,

Kris

[berrysharpie]

After a lot of thought today, I think we are committing to moving the store back to the main site. There really wasn't much need to move it to a subdomain in the first place and the other parts of the site that we did move to subdomains are now doing better so that is a plus.

Now it's all coming down to the store and the main site to focus on. So our big decision now is the 301 vs 404. We have discussed and read oodles amount of information between the two and the most logical does seem to be 301, as always. We don't want the bots to think we have a massively broken site with thousands of 404's.

So, how do you go about getting a fresh start on an area without issuing 404's. If we have to do 301's in order to send the pages somewhere, do they HAVE to be to our new store or can we flush them another way?

One of the reasons we changed our store in the first place is because we moved from one partner to another. We believed that the old partner was playing games with our feed and basically making a mess of things for us.

We don't want to 301 old pages to new pages if, for any reason, they had some sort of ding on them. We can't prove that they did or didn't but I guess for our own clear peace of mind, we would really like to not allow them to be part of our new store.

Whatever we read about 301's, it sure seems as though you carry anything bad from the old to the new through this process. Or are we wrong?

This is such a messy process but I just don't know how you get rid of old junk if you don't want to redirect it to things that don't fully match anymore or if you feel something might have been wrong with the old links.

So I guess that is where we are at now. 301's but how. Could we redirect somewhere else on a different subdomain to basically suck up all the old links. That seems wrong as well. We don't think it's right to send everything to the index page of the site because that is potentially thousands of pages all going to one page. So how do you get rid of a large number of pages that don't have a direct match because they are old.

Thanks again if anyone has any thoughts.

Kris

[g1smd]

It's quite possible that you haven't waited long enough after the move, for Google to have fully picked up all of the changes that you have made. I usually allow at least six months. Your WMT reports will give many clues.

Reversing changes is likely to confuse the bot unless the technical implementation is absolutely spot on. There's no room for error this time.

If you do move again, then a 301 redirect is in order - both for traffic and for redirecting any benefit from newly gained links that are currently pointing to the new subdomain.

If store pages return to their old URLs then reindexing will likely be quite quick and it will take a little while for any link benefit for old links still pointing at those old URLs to kick in.

Make this move the one where you tighten up your URL structure and the technical implementation. Take the opportunity to go extensionless on your URLs (if you haven't already) to make URL rewriting simpler.

[berrysharpie]

g1smd,

Thank you for the reply and thoughts.

It definitely is possible we are being impatient. Let me try to explain the pattern of how the new subdomains faired in the index and maybe that will help determine also if we are being too quick.

Subdomains 1,2,3 all launched on August 15th. Subdomains 4,5 launched 1 week later.

Within the first week of each being launched, WMT showed powerful indexing and everything looked very smooth. In fact, within the first 2 weeks, all subdomains even showed healthy signs in the search results.

On around the 3rd week for each being introduced, we definitely saw some bouncing which will fully expected.

The difference? Subdomains 1,2,3 & 4 all settled in within a month and the bouncing has reduced dramatically. Everything looks healthy. Those links for all those domains had EXACT direct 301's from old links to the new subdomain links.

Subdomain 5, didn't bounce. The links showed up very nicely like the others for a couple of weeks and then vanished completely. I have checked some anchors on them and many still look nicely anchored but no signs of them in the index.

I normally wouldn't expect such quick results and feel so impatient except we were very suprised to see Subdomains 1,2,3 and 4 settling in so quickly.

The big difference about subdomain 5 is that the 301's from the old links to the new are:

1) NOT exact link redirects. That was simply impossible to do. We changed from one partner to another that carried different gear with different names to similar gear, etc. It just wasn't going to happen. We were lucky to get maybe a 30-40% hit on a match on the product and it still wasn't going to be a link match. That was a similar product or exact product match with a different way the name was displayed or something of the like.

2) As mentioned in my second follow up post, we were fairly suspicious that our previous partner was screwing around with our feed. Things like misspellings, using keywords too much. Not A LOT but more than we usually tolerate. So, after much complaining and nothing being done, we left. We will not allow nonsense on our site like that.

So, it is possible we have been impatient but with the other subdomains taking so quickly, we feared that maybe the store links were in fact hit somehow because of the old store and then we dragged that hit along through the 301's.

Is that possible?

If it is, we don't want to sit around and wait on links that could have taken a hit right from the start. That is why we are trying to figure out if we should take another jolt, create a new store and dissolve the new subdomain one quickly before it takes hold too much.

We have learned in the past that if something does go wrong, sometimes it's better to respond quickly no matter how drastic it might seem and that has in fact cured the problem and we were able to move on. Sometimes, you just need to be patient. We just aren't sure which case this one is when the other subdomains seem healthy and this subdomain is struggling and it's possibly related to the old 301's.

IF we do send the links BACK to the original domain, they will once again be new links. The only way to use the old links would be to switch back to our old partner which we will not do under any circumstance. And IF we go go backwards and if the new subdomains have dragged along some sort of ding from our old store, would it really be wise to 301 the subdomain to the new store?

I know this sounds very messy and it's a lot to read. I do very much appreciate you even reading through all this mess. We want to do the right thing, whether it be wait or fix it now.

Thank you again,

Kris

[lucy24]

Uhm... Hate to bring this up, but what about humans? On the off chance that someone, somewhere has a bookmark to the subdomain, or follows an outdated link, where will they be sent?

I don't think it's 404 vs. 301 anyway. It's 410 vs. 301. "It used to exist but doesn't any more" vs. "We used to be here and now we're there."

[g1smd]

The problem with subdomain 5 is seemingly rather obvious then. It's the combination of moving domain and changing the page URLs at the same time (this often doesn't go well) as well as not having full coverage with the redirects.

I don't know how you should proceed (move or not move), but having moved a page from URL A to URL B, if you now move again you will have double the work to do. The existing A=>B redirect will need to be altered A=>C and a completely new B=>C redirect will need to be created.

You absolutely must not create an A=>B=>C redirection chain.

[edited by: g1smd at 11:11 pm (utc) on Oct 21, 2012]

[berrysharpie]

Lucy24, Excellent point and that is something we were just discussing right now. We do believe 301 from the subdomain to a new attempt at the store is most likely the smart way for the bots and more importantly, people. After some more research, it also doesn't seem as though possible "dings", which we still haven't even proven we had on the old store, are not passed through. So, all in all, it looks like a full blown 301 is in order and the easiest solution.

The links from the subdomain5 to a new store would be direct hits. So that is a major plus. What is left are the really old store pages and, well, since they have been around for a long time, it's still better to go ahead and redirect them to the best possible match. If there isn't a match, a redirect to the front page of the store.

So what we might mostly be looking at here is that a large reduction of pages was possibly just too much on the main site AND the store.

Our biggest mistake probably right now was that we should have introduced one subdomain at a time and slowly taken our time at it when we saw the main site was not destabilizing.

[berrysharpie]

g1smd,

That's definitely what we figuring out right now. If we go ahead and do this, we have to plan that out. Luckily, going from subdomain to new links would be easy for us and direct hits. It's those nasty ones from the original store that got redirected but I believe we have some ideas in place for a fairly easy tweak on that.

I'm glad that the problem is obvious now though because it's less likely a ding, but more likely just a gigantic mess. It was absolutely impossible to do exact match redirects. We had to do our utmost best to match on keywords from one to the other. It did amazingly well but of course many were going to go awry.

So I guess it is coming down to two possibilities...

1) It's not been long enough to do anything about it because of the magnitude of change,

2) We deflated the size of the site and so we will now continue to sink because we lost so much of our weighting. If it's this problem, we want to move things back from subdomains to the main site. I wish we had never done this during our switch from one partner to another. We should have just focused on that and never considered subdomains. We thought it might be kind of nicer and more organized.

At least we know the other subdomains are ok. What a mess...

[berrysharpie]

g1smd,

We won't do a chain. Never.

[berrysharpie]

Thanks for the replies, we have decided to use a little patience. The first point made was we might just not have waited long enough. 2 months is not.

We don't believe we have been hit by Panda or Penguin. We also do see some positive signs from anchors. Lastly, the fact that the subdomain 5 did enter the index originally and ranked well for a brief time doesn't suggest that because they have vanished now, won't mean they can't return.

We've also spent the last several hours with a close eye on the pages to check for potential spam, mistakes or anything else that could have caused the problem and they really do look clean.

So based on all that, we will go with the first suggestion that we haven't waited long enough.

Thanks for the suggestions.

Kris

[berrysharpie]

Just wanted to update on what's been going on. Really no major difference except this:

When I now do a site operator search followed by the title of the page I am interested in, I get a dead hit.

site:mysite.com title of page

That would seem like a healthy sign that there isn't likely any poison that got carried from the old URL's through the 301 redirects? Maybe?

We are still fearing that moving too many pages from the main domain to multiple subdomains was a mistake. We've been told by a few sources now that we probably should not have moved the store into a subdomain while all the other things we moved were a good idea. So it is possible in the long run we might have to still move that store back to the main domain. If so, and there is no poison on those URL's, maybe we can do a very clean 301 again. Someday.

So I guess the main question right now is if doing that site operator search followed by the title or close keywords gets the page I was looking for, is that a healthy sign?

Thanks