Forum Moderators: open

Message Too Old, No Replies

Can I Ask A Dumb Question?

Ms. Minty Fresh Bot

         

Hardwood Guy

11:07 am on Dec 29, 2002 (gmt 0)

10+ Year Member



Thanks for those that may be interested in this. It may sound simple to many but it doesn't for me. First off how many actual bots are out there? The question has been recently answered in some ways but I look at it this way. It's kinda like Santa Claus the way I see it. How can these bots cover so much ground and so many web pages out there? Upon looking at some of my bot traffic this morning I saw five of them and they've been there for as long as 18 minutes. Here's another tack of three more....

64.68.85.12 - crawl16.googlebot.com 00:11:29

64.68.84.142 - crawl13.googlebot.com 00:04:09

64.68.84.207 - crawl15.googlebot.com 00:00:00

Are there duplicate bots under the same fresbot number? What's the difference between one bot and the other? Anything?

Thanks for your time:)

Grumpus

1:36 pm on Dec 29, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Each IP is likely a different server running a bot program that sends data to a common place for sorting. I'd suspect that there are several "threads" for each IP. In other words, one IP has one program running which probably polls several pages at a time. (my guess is 8, but that's from olden days of machines and I'm not up on my hardware capabilities anymore).

I don't know exact totals, but there are probably 18-24 each for Freshbot and regular crawlbot. Not sure at all on the image bot.

G.