Forum Moderators: open

Message Too Old, No Replies

Slurp hitting same file thousands of times

not sure who to email about this

         

kpaul

7:54 pm on May 27, 2005 (gmt 0)

10+ Year Member



i've posted this before here:

[webmasterworld.com...]

but got no answer. basically, the IP 68.142.231.65 is requesting the same file over and over again, thousands of times per days, eating up a lot of bandwidth. it's reading a css file. the 'bot' doesn't identify itself, but the ip resolves to Inktomi.

i've tried emailing Yahoo (through web support form)

support@inktomi.com and one other place but haven't got any answers.

should i just block this IP and not worry about it? why would it be hitting a single file repeatedly like it is?

thanks!
kpaul

Span

9:01 pm on May 29, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



kpaul, do you mean that bot is requesting that same file continously since April 19? Five weeks? I would have blocked it within hours. Shoot first, ask later. Inktomi or not.

kpaul

9:53 pm on May 29, 2005 (gmt 0)

10+ Year Member



it starts up, goes for a few days, then stops for a few days, then starts up again.

it's taken a lot of bandwidth this month. it is just one IP.

do you think i'll be safe blocking that IP? i don't want to take the chance of losing the few yahoo referals i'm getting.

all my emails to them are bouncing or not getting replies, though.

it's weird that it's not showing the UA at all - may the IP is being spoofed somehow?

i think i may just go ahead and ban that IP, although it's stopped again ... for the moment.

thanks for the answer ;)

-kpaul

Span

10:15 pm on May 29, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It may be an Inktomi IP address, but I'm sure it's not a bot that has anything to do with indexing pages, so you will be safe by blocking it.
That IP is not used for spidering, it doesn't show up in Google anywhere (except right here).
Someone in their labs testing something?

kpaul

10:32 pm on May 29, 2005 (gmt 0)

10+ Year Member



that's what i thought at first because it was only taking a CSS file. then again, why would it take it once every couple seconds?

i'm gonna ban it.

thanks again.

bumpaw

9:25 pm on May 31, 2005 (gmt 0)

10+ Year Member



I see Inktomi Slurp in Awstats report but the log file shows yahoo! slurp. Are these one and the same?

yahooseeker

2:44 pm on Jun 2, 2005 (gmt 0)



can you please give us the hostname of your web page
that is hosting the included css file.

It may be helpful to see if the access logs if you have them handy.

thanks
yahooseeker team.
( yahooseeker at yahoo-inc dot com )