What's your spider traffic look like?

[walnut99]

I have observed that Inktomi slurp (65MB) has been eating up a lot of bandwidth, followed by MSN Bot (52MB), Google only ate up 2MB in bandwidth.

Does this means that Yahoo and MSN are gearing up to archive more web pages?

How does your stats look like?

[wilderness]

"How does your stats look like?"

I'd hate to see either my stats or my logs "IF" I allowed all these bots to spider images.

Are you allowing them to spider your images?

[walnut99]

We don't limit what the spider crawls in our website.

[mcneely]

Interesting snacking habits of GoogleBot I've noticed lately..

On a diet maybe?

But 4 months ago, GoogleBot would make a very large number of requests under one ip, but now it changes ip on every 3rd, 4th request...much like how the InktomiSlurp does..