Forum Moderators: phranque
The 301 is working like a dream for visitors (Thx for the code Jim!). Visitors are redirected to the exact page rather than dumping them on the home page.
Looks like G has ignored the 301 from an indexing point of view. Here's the prognosis:
1. On the 24th the .com disappeared from Google's index (site:www.domain.com returned nothing).
2. Today (25th), the same outdated .com results were showing once again in G's index - from about 8 months ago. Pages that no longer exist.
3. On the 23rd, G hardly crawled the domain with its bots/spiders. The 24th & 25th are seeing similar levels of crawling as was experienced previously on the .co.uk prior to switchover. Hence a suspicion that G's treating the .com as if it were the .co.uk.
4. G's index still returns all current pages on the .com domain in its index as if they were still listed under .co.uk (site:www.domain.co.uk). This confirms for me that G is treating the .com as if it were the .co.uk
5. Searching for "keyphrase" for UK only on G.co.uk returns the same result as before - the .co.uk around position 21. When you look at G's cached version you see that it's today's page. So this confirms that G thinks the .com is the .co.uk.
I had thought that G would replace the .co.uk domain in it's index with the .com. Desperately seeking advice.
Merry Xmas,
Coburn
That would pretty much explain things, if
just = within last two weeks or thereabouts
I don't know the ins and outs, but I do know that following a 301 is a far simpler task for G (and the other major players too) than getting the change fully recorded in their records.
Most recently changes are as follows for those interested in sharing our learning:
6. G index now returning some of the live .com pages with a "site:www.domain.com". The old outdated pages are listed as "Supplemental Result". Our all important home page is also listed as a Supplemental.
7. G index is now showing all pages for the .co.uk as Supplemental.
8. 24th saw recent levels of deep crawling for the .com for the first time. 25th and 26th however show very little crawling at all on the .com. (this makes me very nervous)
9. allinanchor:kw1 kw2 kw3 is now not showing the domain - we used to be tops. So no recognition of backlinks yet.
The best plan is to start with one subdomain of one domain and stick with it forever, never change the names of any pages, never delete old pages, etc. Of course, this is often impossible, but any changes like this always have some temporary negative impact, and there is practically nothing you can do about it during the time required to sort things out. I recommend taking at least a 30-day, phones-and-pagers-off vacation after making changes like this, once you are quite satisfied that the technical implementation is correct...
Don't worry about being crawled or not being crawled daily. Most sites are NOT crawled daily. Also, you have made a major change, so what we used to call the freshbot function of G's crawler is going to drop your old site and tell the deep-bot about the changes. When deep-bot next gets around to your site, it will probably make several visits going deeper each time. But it will take a while to get back where you were in the SERPs, no matter what. Give it 90 days... After that time, you may have cause for worry. Until then, do something productive with your time that improves your site for visitors. Or go on holiday. ;)
Jim
Has there been a difference in the past 6 months to a year from how it used to be, in how quickly Google is responding to 301's?
Here is what I've determined. 301's within an existing site get picked up very quickly and Google updates it's index usually within 7-10 days. There are other factors at play here such as how frequently your site is spidered.
301's from an old domain to new domain are handled completely differently. It almost seems as if there is a holding period involved. The jury is still out on my end. If the site in question (the one I worked on) does not come back into the SERPs by end of January, I'm going to start sending off emails to Google.
Right, they are. But what I'm looking for is the timeframe, whether it's being handled differently now than it used to be. That's *very* important to know.
Added:
2. Today (25th), the same outdated .com results were showing once again in G's index - from about 8 months ago. Pages that no longer exist.
I missed that the first time reading. So that places the page they're showing from right around April.
Coburn, what used to be good to do was if the site was on a dedicated IP, to put up a link to the IP number rather than the domain name - but it seems it very well could be completely different now. That's why I'm looking for a timeframe, if there is one, if 301 handling was changed.
>>almost seems as if there is a holding period involved.
Exactly what I'm wondering about - and whether it was always that way, since I don't remember it being like that.
Marcia: 216.198.218.28 - I don't believe that the IP has changed at all since before April. In the meantime I've asked our hosting company to confirm.
April=.com, May=.co.uk, Dec=.com (big sigh, talk about bad timing to switch to a new domain!)
Edward: Given the structure of our switch, how long would you speculate the changes to take?
Jim: We would love to have stuck with the .co.uk. The effects of the 'sandbox/deep freeze' were killing us though! So, do you think that switching to the .com domain will do the trick to get us out of the beloved 'sandbox/deep freeze' (prefer Marcia's 'deep freeze' name).
Edward: Given the structure of our switch, how long would you speculate the changes to take?
I wish I had an answer for that one. When we did everything for this particular client, we followed the written guidelines from the authoritative resources on how to do this (based on our interpretation of those guidelines). Here's a brief outline.
Client receives their .edu domain after years of waiting. There is a new marketing campaign scheduled for 2005 to promote the change.
We were not involved with the previous .com domain. This is a new web site with all new content for the .edu version. I am familiar with the previous .com promotional strategies.
We spent hours finding all references to the clients old .com site. We then implemented a very detailed httpd file that mapped all of those old URI references to the new .edu equivalents. That was right before the launch in 2004 October.
All implementations are correct as far as we know using a variety of online tools to check for various things. Mainly what the server headers are returning. In our case we have the proper 301, 200 sequence.
The client is still showing in Google under their previous .com. Company name searches put it in the #1 spot but with the old URI reference. There is no cached page. There is a similar pages link which at the beginning showed the .edu in the #1 spot for being similar. That is no longer the case.
It has been almost 90 days and still no love from Google. Yahoo! just recently added a large chunk of pages from the new .edu domain. Google indexed the .edu domain within a week after it was launched.
Based on my experiences so far following the protocols, I'd have to question whether or not something is broke with Google. Or, there is something special they are doing in these types of redirect situations. From what I'm seeing, there is a holding period. How long it is, I don't know. I can tell you that it has left a bad taste in my mouth and I am now questioning whether or not this type of implementation should be recommended.
301s within a site are fine. 301s from an old site to a new site may not be recommended. Everything you read online from the authoritative resources say the 301 is the definitive way to transfer all visitors from one to the other permanently.
It looks like now, because of certain abuses in the 301, 302 protocols, that there are blanket issues to deal with. If you've been reading here regularly, you'll see all the nightmare topics on sites being hijacked using a 302 method. You'll also see various references to Yahoo! not handling 301s, 302s properly. If that is the case, what is a professional developer to do?
I'm ranting now, sorry about that. It gets me a little heated under the collar. I like to follow standards. In doing so, I may be falling prey to the inability of the search engines to properly follow the written protocols.
Developments:
10. Our .com home page is showing in G's index! Dated 27th on the SERPs page, though when you click on the cached version it displays todays date being the 29th (Better than February!).
11. We're still not being deep crawled. Though from what you say Jim and what I read on a ww post yesterday: the deep bot only kicks in on the 1st and 15th, so no worries there. Am puzzled though at how G can show a number of our live pages as cached today when the logs show virtually no hits from the G-bot...?
12. The main reason for champaigne is that I think that the .com may be out of the sandbox! Remember: the whole reason for the 301 change was to our .co.uk escape G's 'sandbox'/'deep freeze'. Our .com is now showing higher in the SERP's than .co.uk used to.
New problem: We are overoptimised for our keyphrase - showing 1st globally for a allinanchor:kw1 kw2 kw3 (as well as kw1 kw3, and no. 2 for kw1 kw2). So looks like we've had a filter slapped on us (tipped off by internetheaven - thx!). We've had about 25% of our link partners differentiate their anchor text - so now we're wondering whether we should be getting links to other pages in the site (aside from the home page), whether this will get us out of the filter. Still not showing for the other pages in the site - and my understanding of the filter was that it would penalise only the overoptimised home page, not the other pages. Any advice from my betters?
Marcia - hosts confirm that the IP was changed back in September.
To be fair, no search engine handles domain changes quickly and perfectly, so I'm not picking on big G here specifically.
Just resist the temptation to submit more than a very few URLs, submit more than once, or to switch back to the old URLs before the maximum waiting period is over -- 90 days. The worst-case scenario is to switch back and forth faster than Google can comprehend the change and get settled down again. If you do that, you'll kill both domains' rankings.
I've found it helps to mentally "amortize" the pain that changing a domain can cause. Yes, it can hurt for awhile, but after that you will (hopefully) reap the benefit of better branding, better search localization, or whatever it was that made you decide to switch in the first place. So, as they say in the exercise industry and in the military, "No pain, no gain." Also, for the benefit of those who read this later, if changing domains is optional --that is, if you have a choice about the timing-- then it's best to do this during your off-season, or at least to avoid your peak season.
Jim
So I'm not sure whether the resubmission did the trick. Just happy to have the home page as fresh as...
16. Going back to our G 'deep freezing'/'sandboxing' and probable filter. For kw1 kw2 kw3 we're showing #145 out of 1.5 million. So I'm guessing that we're out of the deep freeze...anyone care to confirm or doubt? If so, we're into a filter/penalty. allinanchor: kw1 kw2 kw3 shows us at number 1 position. We had link partners vary anchor text recently, so are wondering whether there's some other filter to blame - like too many links coming in to the home page and virtually none to the rest of the site. Other pages on the site show very poorly for their optimised phrases.
This guesswork is making my head ache. Time to count sheep.
This guesswork is making my head ache. Time to count sheep.
I found out long ago that the worse thing you can do is watch this on a daily basis. It will only cause stress. Just let it go for a week and then come back and look at it. Google has been in constant flux for quite a while now so changes will happen constantly, even on page refresh. ;)
Jim: MSN beta and Yahoo has the freshest results and ranks the .com very well. So they've both handled the 301 switch of domains purrrfectly. It's G that's hit a wobbly.
17. G's being consistent! We're consistently ranking at 50-54 for our kw1 kw3 combination. Should I read anything into the fact that G's showing both the .com and the .co.uk as having zero backlinks? I know G's messed up their backlinks reputation, but zero? Before the last update, the .com was showing 60 (from way back). We've only been deep crawled once since the swapover, and that was back on the 24th. Picked up PR across the site :-) Could it be that the reason we're not performing well in the SERP's is due to G not somehow 'seeing' our backlinks? G's still showing us as tops for the 'allin' commands.
18. Performed a check for &filter=0 and we fare worse - so am relieved that we're at least not suffering from a dupe content filter.
puzzled,
Coburn
