Hi everyone.... I know this was posted before but I cannot remember where... does anyone remember the top IPs for the Googlebot? I don't remember. I do remember someone mentioning that there were four main Googlebots...
Remove 22.214.171.124 from the list... I was informed that it does not belong to Google. Not sure if it once did or if it was a mistake from the beginning.
As to the rest of the IPs, I don't keep any records on how I came across them. Most of the IPs I list come from submissions from people looking at their logs, including my own logs. As you all know, search engines are constantly adding new IPs and abandoning old ones. It's likely that many of the IPs in my lists are no longer actively in use by search engines. I make no guarantee as to their completeness or accuracy. Take the addresses "as is". :)
Anybody with a little programming knowledge can identify themselves as Googlebot. They might do this to look for cloaked sites or simply to hide [webmasterworld.com] their real user agent. If the IP doesn't resolve to a Google IP block on a whois search then it probably doesn't belong to Google.