Welcome to WebmasterWorld Guest from 54.198.90.50

Forum Moderators: open

Message Too Old, No Replies

Inktomi Spider Has a Party

Spider visits out the roof

     
1:01 am on Nov 26, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 31, 2002
posts:43
votes: 0


Three weeks ago, a site I manage was visited by Inktomi's spiders 213 times. Two weeks ago, 977 visits. But last week, Inktomi came visiting a whopping 4,399 times.

Anybody else notice this?

10:49 pm on Nov 29, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jimbeetle is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Oct 26, 2002
posts:3292
votes: 6


Yep, Ink appears to be real busy this month. The last I checked a couple of days ago it slurped up about 2,300 pages on a 1,500 page site. It also did pretty well in October with 1,700 pages but seems to be a bit hungrier of late.
3:41 am on Dec 4, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 31, 2002
posts:43
votes: 0


Yeah -- I also noticed a huge increase in visits on other sites I manage. Well, I hope I get some good listings from it. :D
4:02 am on Dec 4, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 27, 2002
posts:1685
votes: 0


I still see slurp asking for files I've never had.

I still see slurp asking for files that I 301'd lastyear. Most of the other bots have moved on, but not slurp.

Maybe someday.......

<shrug>

Pendanticist.

6:12 pm on Dec 5, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:June 16, 2003
posts:633
votes: 0


I have a fairly new site, and it doesn't look like Ink has been making too many frequent trips to my site lately. I've only gotten 6 visits from Inktomi this month :(
6:55 pm on Dec 5, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jimbeetle is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Oct 26, 2002
posts:3292
votes: 6


I still see slurp asking for files that I 301'd lastyear.

Same here, more than 12 months since I did a major overhaul in one section and Ink still hasn't gotten the word though it has requested those file numerous times and got the proper 301.

Have more reorganization in mind but putting it off until I can slim down my htaccess a bit before adding more entries.