Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google Introduces New Locale-Aware Crawl Configurations

         

engine

6:02 pm on Jan 30, 2015 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Today we’re introducing new locale-aware crawl configurations for Googlebot for pages that we detect may adapt the content they serve based on the request's language and perceived location. These are:

Geo-distributed crawling where Googlebot would start to use IP addresses that appear to be coming from outside the USA, in addition to the current IP addresses that appear to be from the USA that Googlebot currently uses.
Language-dependent crawling where Googlebot would start to crawl with an Accept-Language HTTP header in the request.

As these new crawling configurations are enabled automatically for pages we detect to be locale-adaptive, you may notice changes in how we crawl and show your site in Google search results without you altering your CMS or server settings.

Note that these new configurations do not alter our recommendation to use separate URLs with rel=alternate hreflang annotations for each locale. link [googlewebmastercentral.blogspot.com]


Here's the full article on how Google uses different crawl settings for sites that cannot have separate URLs for each locale.

Locale-aware crawling by Googlebot [support.google.com]

lucy24

7:58 pm on Jan 30, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I think you meant to say
Googlebot would start to use IP addresses that appear to be coming from outside the USA
because I cannot possibly be the only person who automatically blocks any visitor who claims to be Googlebot but comes from somewhere other than 66.249