homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Server move, googlebot lost

 4:57 pm on Mar 8, 2008 (gmt 0)

We switched from one server to another, left the site up on both boxes. Google continued to crawl the old box for a day or two and then appeared to have found the new location, did some moderate crawling and we figured everything went through AOK.. It has completely stopped crawlng the old server.

However, it appears to have stopped with the crawls on the new one as well. It only visits the index page when we update and some older urls occasionally, nowhere near the scale we were seeing before the move. The cache has not been updated since the day before the move and none of the new pages are being indexed whereas they would have been included within minutes of being posted before (with the XX minutes ago thing next to them). Old urls seem to have preserved their rankings.

The move was a straight control panel move, all urls / rewrites / structures and pages are mirrored to the last bit. The site is completely clean, we don't play around.

I didn't expect this (even though I feared it might happen).

Anyone have any ideas / experiences?



 7:02 pm on Mar 8, 2008 (gmt 0)

What is the time scale involved here? Days, weeks months? Also, you said "only visits the index page when we update" - Can we assume from this that this is a blog platform of some kind that automatically pings Google or at least creates a new feed?

Here are some other factors you can check after a move:

1. Is the new server configured to respond to if-modified-since requests?
2. Is the new server using gzip compression?
3. Are there any DNS configuration problems?
4. I assume this is a new IP address - is it shared with any other domain?

Also, for anyone reading this thread who is planning a hosting move, there's a link in the Hot Topics area [webmasterworld.com], which is always pinned to the top of this forum's index page. It goes to a nice blog post from Matt Cutts on the topic "Moving to a New Web Host".

Old urls seem to have preserved their rankings.

That's definitely a good thing, but I do appreciate your concern on the spidering change.


 11:18 pm on Mar 8, 2008 (gmt 0)

Hi tedster,

Thank you for the reply. We are talking a short time period here, it may have just end up being googlebot waiting for all it's machines to sync. Your assumption is correct, yes.

As for the factors you have pointed to, #1 is interesting, I will have to look into that. The rest are definitely not the issue here.

Thanks again!


 5:33 pm on Mar 10, 2008 (gmt 0)

Well it seems that my hope everything was just a glitch was not justified. We still rank for old terms, got rankings for some new terms on older (pre-dating the move) pages, but anything that was added to the site since the move is nowhere to be found whereas usually it would have shown up literally witin minutes (some of the new pages did get crawled, but not all and they don't show up) :( GB activity seems to have picked up a bit but I don't think it's back to pre-move levels.

Google is still showing the cache from the last day before the move and when searching for the site domain the "see more results from blahblah" link is gone. Also when searching for site name, a page from the site shows at #1 instead of the site index which was not the case before.

New nameservers and server are working properly, resolving from all over the world.

I'm on the verge of a breakdown as a really huge amount of time and quality work was devoted to the site by our whole team from producing quality content to making the site absolutely clean and user firendly.

If we did any nonos I would understand the whole thing, however since that's not the case I'm simply utterly confused and completely desperate.


 5:46 pm on Mar 10, 2008 (gmt 0)

Some added thoughts:

Does the domain name resolve directly, with no redirects involved at all? And is the new IP address dedicated and not shared?

Also, is your error handling on the new server completely correct? For example, does the server return a true 404 status in the http header and not just a custom error page with some other http status?


 6:00 pm on Mar 10, 2008 (gmt 0)

Hi tedster, I really appreciate your willingness to help.

What do you mean by directly? It's a pure DNS entry if that is what you asked, no 301,302,meta or similar redirects. Dedicted ip, as it was before, fresh one as well, hasn't been used before unless our host wasn't honest with us which I simply don't believe as they are more professional and dedicted to what they do than anyone I have dealt with before.

Well, error handling.. we use some rewrite rules so there are no true 404 errors being generated, but that was the case before the move as well.


 6:11 pm on Mar 10, 2008 (gmt 0)

Just out of curiosity, have you run a DNS Report on the new location? Any Fails or Warnings in that Report?


 6:16 pm on Mar 10, 2008 (gmt 0)

We did a ton of traceroutes from various locations worldwide and they all came out ok. Traffic has remained the same with the approximately same geographical distribution as before. Is there anything else we can check, I'm not sure what you meant by DNS report?



 6:33 pm on Mar 10, 2008 (gmt 0)

A DNS Report may reveal issues that you are not aware of. There could be a misconfiguration that is transparent right now but once you run that DNS Report, it may show up. One of the first things I do when backtracking issues like this is to verify that DNS is in working order. I've said it before here at WebmasterWorld that many of the issues most of us run into can be tracked back to something at the DNS level (when discussing this type of stuff). Traceroutes are just one part of it.


 6:34 pm on Mar 10, 2008 (gmt 0)

Thanks. Where could I run a DNS report?


 6:38 pm on Mar 10, 2008 (gmt 0)

A search on that phrase will turn up several options for you - we don't want to plug any one specific service.


 6:38 pm on Mar 10, 2008 (gmt 0)

The DNS Report offers a 21 day free trial. There are other resources out there too. That report has saved many from disaster.


 6:48 pm on Mar 10, 2008 (gmt 0)

Appreciate the tip, ill check it out!

Anyone else have any ideas as to what could be happening here?


 7:00 pm on Mar 10, 2008 (gmt 0)

Anyone else have any ideas as to what could be happening here?

Before we make any more guesses, let's confirm that everything is in order with your DNS. Again, many of these types of issues can be tracked back to something in the DNS so that would be my "first" suggested move. From the findings in that report we can then make more refined "educated guesses" as to what may be the challenge.

I would also like to point out that since we don't have a "before and after" snapshot of a DNS Report, it may be difficult to pinpoint any one particular issue. I'd be looking at the information for your Parent Nameservers, the NS Records and your SOA particularly this one "SOA MINIMUM TTL value".


 8:58 pm on Mar 10, 2008 (gmt 0)

pageoneresults, the dns settings are the exact copy of the old ones (i have the original zone file), word for word, number for number. (aside from the new ips and NSs of course). All TTL values are as they were before and I've had my host confirm that the DNS is functioning perfectly fine.


 3:42 pm on Mar 11, 2008 (gmt 0)

One problem is dupe content...whenever you move to a new server you should password protect the old server OR it will get the same stuff on both servers and kick it out. I NEVER allow the same content up on two servers at one time...I password protect old server as fast as I can.


 6:00 pm on Mar 11, 2008 (gmt 0)

I did a server move in the past and the site completely vanished from the serps. Turns out that the time/date stamp on all the content on the server refreshed and the site got penalized for duplicate content because of all the proxy hijacks and content copiers that were on the site. At least that is my theory and the theory of some people I had look into it. It was eventually resolved but it was a scary time to say the least...


 6:10 pm on Mar 11, 2008 (gmt 0)

If the server clock is off, that can also mess with your if-modified-since responses.


 6:32 pm on Mar 11, 2008 (gmt 0)

Why did you get a new Ip address? Is the server at another location were you had to get a new IP?

I know sounds trivial but just my thought as to a possible reason for this type of odd bot behavior.

I have moved many sites to different servers and never have done a password protect I all I ever did was change the dns leave both sites up till the bot activity as stopped and traffic has been rerouted through the dns and then take the other server down. Just a by the book move as he has done as well.

I have whenever at all possible kept the same ip address so really there is no dns just a short period of down time.

Never had an issue but I havent moved a site in about 2 years so there could be more to this and a by the book move is not by the book anymore.


 8:35 pm on Mar 12, 2008 (gmt 0)

I guess I will find out what the new rules are as I just moved a site to a new server and had to get another IP address as I had other sites using the one I moved this site from.

I will leave the site on my server for a couple days till I don't see any activity and then shut the IIS down.

do you have an update as to the cause or still looking?


 11:17 pm on Mar 12, 2008 (gmt 0)

Thanks everyone.

Ok GB is crawling at what seems to be pre-move frequency, cache has been updated to a post move date, but... None of our new posts are being indexed, while everything up to the move would be in the results within minutes of it being posted.. G blog search is picking it up, but G itself isn't which is scary. We weren't updating during the move because we wanted to let G find the new server and begin cralwing it before we do. Old pages are doing as well as they ever did, but nothing new is being included and that is making me very concerned :(

bwnwbn, nope no ideas yet unfortunately..


 6:29 pm on Mar 24, 2008 (gmt 0)

Have you come back or discovered the issue.

Our move was flawless no issues nothing.


 10:09 am on Apr 21, 2008 (gmt 0)

Did this ever resolve? We are about to move IP addresses for some big websites and this topic didn't fill me with confidence.


 1:48 pm on Jun 17, 2008 (gmt 0)

Around 4 hours back we moved one of our active forums from a VPS to a dedicated server now we can find the same problem on our server.
We moved from VPS due to heavy server loads.

New Server IP - AAA.BBB.#*$!.ZZZ

** Our Database was moved to new server around 6 months back

Present Status:
We have webfiles on both old server and new server (.htacess pointing only to domain name)

But I can find googlebot vanished, any suggestion/precautions we should do immedaitely to not suffer ?
We can find almost 50% traffic drop in last 5 hours when we also changed IP address of our custom nameservers used for this domain.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved