Googlebot refuses to recognize new IP address...

[bobmark]

Hi,
my hosting service recently migrated my site from one IP address to another (a change of servers to do with an upgrade).
All other robots immediately followed the url (which is unchanged) and began spidering the new location; Googlebot continues to spider the old IP address (the old site is still there temporarily). I assume this is something to do with Google's attempts to reduce doorway page spamming (i.e. they use IP address rather than just url).
I am worried about the implications of this. Will Googlebot conclude that the 2 IP addresses are dooorway pages and what will happen when all files at the old IP address are removed? Will I be dropped from Google?
Has anyone had experience with this? Is there any way to help googlebot recognize the new IP address as the only valid one (I don't see any way in robots.txt paramaters)?
Thanks,
Mark

[Sasquatch]

How long since your domain name was switched to the new IP?

I suspect that google does not update their DNS cache very often, and if they are not crawling the new site yet it just means that they have not updated their cache.

[Marcia]

It's a continuing issue:

[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]

There's been some good luck with a 301 redirect from a prior URL to the new, and I don't know how well it's working now, but an inbound link to the new IP number can't hurt.

Maybe someone has an authoritative answer for the timeframe.

[bobmark]

Thanks, sasquatch and Marcia.
The site was moved on the 12th.
I looked at the threads you referenced, Marcia but I'm a bit confused. They all seem to relate to using a 301 redirect for a different url.
My url is unchanged: www.widgets.com
it is just that www.widgets.com is now at yyy.yyy.yyy.yyy
instead of xxx.xxx.xxx.xxx on the same host.
I can see from the logs and page refresh dates that googlebot still crawls www.widgets.com at xxx.xxx.xxx.xxx.

[Rick_M]

I'm having the same problem. I was on shared hosting with company A, but I'm now using shared hosting at company B. At both places I had a shared IP address. Now, googlebot is spidering my site on Company A which is similar to my site on Company B, so it isn't hurting me too much. However, my site on Company B has more regular updates and it would be nice to have those included as well. Rather than keep accounts at both places until googlebot figures this out, I'd like to have a quicker solution.

Any suggestions for moving from one shared IP address to another? Do I want my old host to clear out it's zone file? I'm worried if googlebot doesn't find my site there, it might not find my new site for at least a month which would be very bad for me.

For now, I'm just keeping both sites up, and if I make critical additions, I update the old site as well.

[nancyb]

bobmark,

I moved my domain on the 6th of Oct. Googlebot found the new IP on the 8th and spidered deeply for the next two days. However, I also keep seeing googlebot in my old host log files and that googlebot appears to be the fresh bot. Don't know if this helps, but it has me concerned, too. I updated the old host with new pages, mainly in case a human was still going there, but am hoping two different googlebots won't talk to each other and assume there are mirror domains. Since it is the same domain I'm reasonalbly confident they will know this, but ....

I had/have unique IPs at both the old and new hosting service.

[bobmark]

Thanks rick & nancy,
I have yet to see googlebot show up at the new IP address but it still visits the old. When I realized this I updated the old too.
My understanding to this point is there is no way to force googlebot to query the host's dns lookup; it just keeps on with it's cached IP address. Every other spider who visits regularly found the new IP address from day 1 so they must check the lookup on each visit.
I suppose theoretically it will at some point decide it needs to refresh its cache and query for the IP address, but I am worried about what damage may occur to ranking/freshness or worse in the interim.

[cminblues]

I suspect that google does not update their DNS cache very often

Also my experience with google DNS cache leads to this.

cminblues

[bobmark]

Rereading yours, Nancy
"I moved my domain on the 6th of Oct. Googlebot found the new IP on the 8th and spidered deeply for the next two days"
I wonder if Googlebot queries for dns only on the monthly deep crawl (which I assume is what you got on the 8th.) so in my case I will see the daily crawls still sticking to the old cached IP address until the next big one.

[przero2]

Monthly is about what I would think ... However we noticed sometimes when Googlebot need to crawl the web in a hurry (following a delayed update previous month or other causes!), it might sacrifice the DNS update process which could lead to DNS cache (googlebot's) not updated for 2 or even 3 months ...?

So, it is safe to leave the site at old and new IP .. Once we had an issue where the server or name servers have not been changed but the domain went from a shared IP to static IP. And the domain was only accessible on the new IP address and not the old IP. It took Googlebot over 3 months to get over this and it was a good lesson learned for it so as not to repeat this again;)

[bobmark]

Thanks przero2,
So if I understand you're saying if my "old" site is deleted from the server while it is still the cached IP address, Googlebot would not query dns and look for a new IP address and Google could drop it from the index until it finds it again through re-submission or an external link.
Have I got it right?

[jdMorgan]

bobmark,

Yes, you've got it right. Google needs to address this issue to make changing hosts a more deterministic process. Imagine the havoc that would ensue if a large hosting company went out of business... Like a well-known european ISP did recently.

You also picked up correctly on the difference between "moving to a different host (IP address)" versus the "changing domain names". A redirect on your old site wouldn't help you, because you're still using the same domain name, so what could you do, redirect to yourself? Good way to lock up a server, but not much help in your case!

Good luck with this, I hope big G flushes their DNS cache soon!
Jim