GoogleBot increasing spider speed?

[ianster]

Last night I noticed a ton of traffic coming from Googlebot for about 5.5 hrs straight. It seems to have saturdated one of my T1s for the entire time it was here, crawling in excess of 7,500 pages/hr during this time.

I have seen them come before pretty hard, but never like this. Has anyone else noticed this?

[Marcia]

What IP numbers?

[seindal]

I have a site where googlebot has taken about 10000 pages/day. It is a fairly new affiliate site, so I don't complain the indexing, but I was surprised to see googlebot take several pages per second.

René.

[farside847]

Yes, I posted about this yesterday but evidently it was not important enough for the Admin to let go through (grumble, grumble)

Starting yesterday at 1AM my site has been spanked hard by the googlebot. To the tune of 2 hits per second continuously.

This is taking my server that is normally a happy and zippy rabbit and turning it into a grumpy and lethargic turtle.

Yesterday:
- 88402 hits in 24 hours,
- 23048 duplicate hits
- IPs:
-- 64.68.82.133
-- 64.68.82.135
-- 64.68.82.136
-- 64.68.82.137
-- 64.68.82.141
-- 64.68.82.142
-- 64.68.82.143
-- 64.68.82.163
-- 64.68.82.164
-- 64.68.82.165
-- 64.68.82.176
-- 64.68.82.178
-- 64.68.82.181
-- 64.68.82.182
-- 64.68.82.183
-- 64.68.82.185
-- 64.68.82.189
-- 64.68.82.202
-- 64.68.82.204
-- 64.68.82.208
-- 64.68.82.25
-- 64.68.82.26
-- 64.68.82.27
-- 64.68.82.30
-- 64.68.82.31
-- 64.68.82.32
-- 64.68.82.33
-- 64.68.82.34
-- 64.68.82.35
-- 64.68.82.36
-- 64.68.82.37
-- 64.68.82.38
-- 64.68.82.39
-- 64.68.82.41
-- 64.68.82.45
-- 64.68.82.46
-- 64.68.82.47
-- 64.68.82.50
-- 64.68.82.52
-- 64.68.82.54
-- 64.68.82.55
-- 64.68.82.57
-- 64.68.82.6
-- 64.68.82.65
-- 64.68.82.66
-- 64.68.82.67
-- 64.68.82.7
-- 64.68.82.77
-- 64.68.82.78
-- 64.68.82.79
-- 64.68.86.153
-- 64.68.86.161
-- 64.68.86.54
-- 64.68.86.59
-- 64.68.86.79
-- 64.68.86.9
-- 64.68.87.14
-- 64.68.87.43
-- 64.68.87.54
-- 64.68.87.78

So Far Today:
- 12061 hits in 8 hours,
- 10 duplicate hits
- IPs:
-- 64.68.86.153
-- 64.68.86.161
-- 64.68.86.54
-- 64.68.86.59
-- 64.68.86.79
-- 64.68.86.9
-- 64.68.87.14
-- 64.68.87.43
-- 64.68.87.78

I have a PR6 site with 119k pages, but I didnt think google would grab them all in one day!

[ogletree]

Anybody got a tool that looks at a log file or files nad gives some sort of googlebot report. I just went through my logs and it says I have only had about 2 hits a day since 7/19 and on the 20th I had 44.

[akognet]

Ogletree,

I sent you a stickymail about the bot-tracking tool that I use. Let me know how it works out.

[algarveb]

Hi,

My site too is being hit with the IP 64. bot.

I have sent a couple of sticky messages - but can someone also tell me please (I'm new to this:

I've also seen an IP of 203. I read a thread that said 'freshbot' used the IP 64, and that 'deepbot' used IP216 - so what is IP203?

Thanks for your patience if this is a little too basic - but an answer would be appreciated.

b

[ianster]

Marcia, I'm getting hit in the 64.68.82.0/24, 64.68.86.0/24, and 64.68.87.0/24, ranges just like farside847.

[swones]

akognet

Can you sticky me about this tool too please.

Thanks.

Simon.

[zacsg]

akognet
could you sticky me about the bot tool too?

thank you

zac