Welcome to WebmasterWorld Guest from 54.196.233.208

Forum Moderators: open

Message Too Old, No Replies

New Googlebot

What about crawl and crawler?

     
11:27 pm on Feb 28, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 29, 2003
posts:34
votes: 0


I've made a new list of Googlebot after a study of my logs :

But I have some questions :

- Is crawlx.googlebot.com always mean that's a "Deep Crawl" and crawlerx.googlebot.com a "Fresh crawl"?

- There's 800 bots for crawlx, but in crawl7 and crawl9 it's not the same number in the IP range (29 and 21).

- Where is crawler19?

Regards

Eric (from France, sorry for my english)

[edited by: WebGuerrilla at 11:39 pm (utc) on Feb. 28, 2003]
[edit reason] TOS #13 [/edit]

2:05 am on Mar 1, 2003 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


Ya, there were a whole bunch of new ips this last month.
6:42 am on Mar 1, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Oct 30, 2002
posts:404
votes: 0


Brett...have you heard of any outside of the normal 216.239.x.x deep crawlers and the 64.68.x.x freshbots? Ive only seen a crossover in the bots being used for image grabbing, but otherwise pretty standard distinctions.
11:07 am on Mar 1, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 29, 2003
posts:34
votes: 0


[edited by: WebGuerrilla at 11:39 pm (utc) on Feb. 28, 2003]
[edit reason] TOS #13 [/edit]

Not allowed to use url in a post?