homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

Do IPs matter during crawling?

WebmasterWorld Senior Member korkus2000 us a WebmasterWorld Top Contributor of All Time 10+ Year Member

Msg#: 11479 posted 11:23 pm on Apr 9, 2003 (gmt 0)

Say I reroute my domain name and there isn't a second of down time. It just changes DNS to a server that has a different IP. Does googlebot get confused by this? Does the IP really matter or is it all just based on the actual domain name?



10+ Year Member

Msg#: 11479 posted 12:14 am on Apr 10, 2003 (gmt 0)

yes it matters. Two distinct actions are performed to access your web page from your url.

First, the full domain name is translated to an ip address via dns.

Then, your web page is requested at that ip address.

During normal browsing, these two tasks are performed sequentially. Your system first requests the ip address for the url you are requesting and then, immediately, requests the web page from that ip address.

Some spiders, for performance reasons, will disconnect the the tight temporal relationship between the dns lookup (to translate the url to the ip address) and the page request to that ip address. Google does this.

That means that Google might lookup the ip address for your domain name today, but not attempt to request your web page from that ip address until this weekend. If you have modified your nameservers to serve the site from a different ip address between those two events, google will look for your page at the wrong ip address. You can miss a crawl by changing ip addresses in the period before a crawl unless you serve your site from both ip addresses during the transition.

There is a second issue to keep in mind that could also trip you up. It may take up to as much as about 48 hours for the new dns assignments to propogate across the web. That issue involves browsers as much as spiders. This means that, for as much as two days after you assign your domain to a different ip address, requests will still executed against the old ip address.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved