Welcome to WebmasterWorld Guest from 54.197.116.116

Forum Moderators: open

Message Too Old, No Replies

New Googlebot

What about crawl and crawler?

   
11:27 pm on Feb 28, 2003 (gmt 0)

10+ Year Member



I've made a new list of Googlebot after a study of my logs :

But I have some questions :

- Is crawlx.googlebot.com always mean that's a "Deep Crawl" and crawlerx.googlebot.com a "Fresh crawl"?

- There's 800 bots for crawlx, but in crawl7 and crawl9 it's not the same number in the IP range (29 and 21).

- Where is crawler19?

Regards

Eric (from France, sorry for my english)

[edited by: WebGuerrilla at 11:39 pm (utc) on Feb. 28, 2003]
[edit reason] TOS #13 [/edit]

2:05 am on Mar 1, 2003 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Ya, there were a whole bunch of new ips this last month.
6:42 am on Mar 1, 2003 (gmt 0)

10+ Year Member



Brett...have you heard of any outside of the normal 216.239.x.x deep crawlers and the 64.68.x.x freshbots? Ive only seen a crossover in the bots being used for image grabbing, but otherwise pretty standard distinctions.
11:07 am on Mar 1, 2003 (gmt 0)

10+ Year Member



[edited by: WebGuerrilla at 11:39 pm (utc) on Feb. 28, 2003]
[edit reason] TOS #13 [/edit]

Not allowed to use url in a post?