homepage Welcome to WebmasterWorld Guest from 54.211.73.232
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
New Googlebot
What about crawl and crawler?
kendos

10+ Year Member



 
Msg#: 9744 posted 11:27 pm on Feb 28, 2003 (gmt 0)

I've made a new list of Googlebot after a study of my logs :

But I have some questions :

- Is crawlx.googlebot.com always mean that's a "Deep Crawl" and crawlerx.googlebot.com a "Fresh crawl"?

- There's 800 bots for crawlx, but in crawl7 and crawl9 it's not the same number in the IP range (29 and 21).

- Where is crawler19?

Regards

Eric (from France, sorry for my english)

[edited by: WebGuerrilla at 11:39 pm (utc) on Feb. 28, 2003]
[edit reason] TOS #13 [/edit]

 

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9744 posted 2:05 am on Mar 1, 2003 (gmt 0)

Ya, there were a whole bunch of new ips this last month.

Marval

10+ Year Member



 
Msg#: 9744 posted 6:42 am on Mar 1, 2003 (gmt 0)

Brett...have you heard of any outside of the normal 216.239.x.x deep crawlers and the 64.68.x.x freshbots? Ive only seen a crossover in the bots being used for image grabbing, but otherwise pretty standard distinctions.

kendos

10+ Year Member



 
Msg#: 9744 posted 11:07 am on Mar 1, 2003 (gmt 0)

[edited by: WebGuerrilla at 11:39 pm (utc) on Feb. 28, 2003]
[edit reason] TOS #13 [/edit]

Not allowed to use url in a post?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved