incrediBILL - 1:59 am on Nov 27, 2012 (gmt 0)

Is there an equivalent for getting googlebot IPs

Nope. I've asked Matt if they'd do something like that which I tentatively called "Spiders.xml" which would reside on their root domain and the crawl team appeared to have ZERO interest.

Maybe if hundreds or thousands of people requested it they would bow to peer pressure or perhaps getting other search engines to start providing it they would do it.

Heck, Google proposed the round trip rDNS validation method for Googlebot and they won't even do THAT for any of their other bots and stuff.

Google doesn't care about providing the simple tools people need to easily solve real world problems and instead will spends tons of money dealing with issues that can be abated with relative ease.

Not so smart for such bright people.