homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderators: Brett Tabke

Paid Inclusion Engines and Topics Forum

  posting off  
Slurp/si and Slurp/cat

 3:28 pm on Aug 30, 2000 (gmt 0)

Slurp/si and Slurp/cat are spidering like crazy. started about 6:00am PT and is following everylink on every page.
WOW is has been a long time since I have seen slurp usually it is just the submission spider.. Something must be up today. Any body else getting hit hard?


Dr. Bill



 9:11 pm on Aug 30, 2000 (gmt 0)

I have also just noticed a huge run from Slurp on several domains - full complete crawls.


 9:52 pm on Aug 30, 2000 (gmt 0)

Unfortunately I don't think this is going to be the golden goose. All the spidering is coming out of Japan (goo3**.goo.ne.jp 202.212.5.**) - when this has happen before it was to maintain the Japanese db. This pattern seems very similar to the other times.


 12:15 am on Aug 31, 2000 (gmt 0)


I am not seeing goo in the listings at all.. do they have a spider of their own?


 1:18 am on Aug 31, 2000 (gmt 0)

Dr Bill,
It will come in as Slurp, but the IP and host name are out of Japan. It still is inktomi, but it is an isolated database for the Japanese portals.


 1:24 am on Aug 31, 2000 (gmt 0)

Hi Littleman.

Is the Slurp/si or Slurp/cat or both?


 1:57 am on Aug 31, 2000 (gmt 0)

The Japanese Slurp I have seen over the last couple of days is actually Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...] - they hit one server 26 thousand times over the last two days - I really hate that they do that, its almost like a DOS. Slurp/si only came in 250 times on the same server. There have been no visits from Slurp/cat for me. So maybe something is happening on your particular sight? That would be nice, eh?


 3:06 pm on Aug 31, 2000 (gmt 0)

Ha ha littleman, 26k in two days huh? it only happens when you have large number of subdomains setup
I wonder what you're doing...


 5:31 pm on Aug 31, 2000 (gmt 0)


Sofar I have had close to 30,000 pages taken and it is still going. I hope it never stops :)


 1:24 am on Sep 2, 2000 (gmt 0)

I've been visited by the entire stable of inktomi spiders in the last 5 days. (notice the very last one).

hits id : ip domain agent
78 SLURP: wm3023.inktomi.com Slurp/cat (slurp@inktomi.com; [inktomi.com...]
53 SLURP: wm3020.inktomi.com Slurp/cat (slurp@inktomi.com; [inktomi.com...]
40 SLURP: j4024.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
40 SLURP: j4025.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
39 SLURP: j4014.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
39 SLURP: j4018.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
37 SLURP: j4012.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
37 SLURP: j4020.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
34 SLURP: j4013.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
34 SLURP: j4021.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
32 SLURP: j4019.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
32 SLURP: j4022.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
31 SLURP: j4015.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
31 SLURP: j4017.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
31 SLURP: j4023.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
31 SLURP: j3013.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
30 SLURP: j4016.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
28 SLURP: j3018.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
26 SLURP: j3003.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
26 SLURP: j3014.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
25 SLURP: j4011.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
24 SLURP: j3017.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
23 SLURP: j3009.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
23 SLURP: j3015.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
22 SLURP: j3001.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
21 SLURP: j3002.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
20 SLURP: j3012.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
19 SLURP: j3005.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
19 SLURP: j3008.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
19 SLURP: j3010.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
18 SLURP: j3019.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
16 SLURP: j3004.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
15 SLURP: j3006.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
13 SLURP: j3007.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
13 SLURP: j3011.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
13 SLURP: wm3008.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
12 SLURP: q2000.inktomisearch.com Slurp/si (slurp@inktomi.com; [inktomi.com...]
12 SLURP: j110.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
12 SLURP: j111.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
11 SLURP: j3016.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
10 SLURP: wm3019.inktomi.com Slurp/cat (slurp@inktomi.com; [inktomi.com...]
7 SLURP: wm3021.inktomi.com Slurp/si (slurp@inktomi.com; [inktomi.com...]
6 SLURP: goo212.goo.ne.jp Slurp/si (slurp@inktomi.com; [inktomi.com...]
6 SLURP: j5004.inktomisearch.com Slurp/si (slurp@inktomi.com;
6 SLURP: j100.inktomi.com Slurp/si
6 SLURP: j101.inktomi.com Slurp/si
6 SLURP: j5006.inktomi.com Slurp/si
6 SLURP: si4000.inktomi.com Slurp/si
6 SLURP: si4001.inktomi.com Slurp/si
6 SLURP: si4002.inktomi.com Slurp/si
6 SLURP: wm3018.inktomi.com Slurp/si
6 SLURP: wm3024.inktomi.com Slurp/si
6 SLURP: si3000.inktomi.com Slurp/si
6 SLURP: si3001.inktomi.com Slurp/si
6 SLURP: si3002.inktomi.com Slurp/si
6 SLURP: si3003.inktomi.com Slurp/si
5 SLURP: j301.inktomi.com Slurp.so/1.0
4 SLURP: j300.inktomi.com Slurp.so/1.0
4 SLURP: q2005.inktomi.com Slurp/si
3 SLURP: wm3021.inktomi.com Slurp/cat (slurp@inktomi.com; [inktomi.com...]
2 SLURP: crusade.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
2 SLURP: y2300.inktomi.com Slurp/3.0 (slurp@inktomi.com; [inktomi.com...]
SLURP: Slurp/si (slurp@inktomi.com; [inktomi.com...]
SLURP: y400.inktomi.com Slurp/2.0-KiteWeekly (slurp@inktomi.com; [inktomi.com...]
SLURP: j521.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
SLURP: Slurp/si (slurp@inktomi.com; [inktomi.com...]
y2404.inktomi.com Slurp/2.0-GreatWhiteCrawl (slurp@inktomi.com; [inktomi.com...]


 1:01 pm on Sep 14, 2000 (gmt 0)

Any one else seen "the great white crawl"?


 1:12 pm on Sep 14, 2000 (gmt 0)


I have yet to see Great White :( Would like him/her to visit me.


 10:35 pm on Sep 16, 2000 (gmt 0)

Inktomi has been poundingmy sites as well, not as much as others are saying, however my site isn't really that big either. Out of those different spiders, which one are the good ones?


 10:38 pm on Sep 16, 2000 (gmt 0)

Do you have a IP Address for that great white??


 10:59 pm on Sep 16, 2000 (gmt 0)

Has any one seen this

Tue Sep 12 07:43:25 2000 -- ¦Mozilla/4.72 [en] (X11; U; NetBSD 1.4.2 i386; Nav)¦¦

When I do a look up on the IP address it comes back as j6000.inktomi.com what's up with that. ??


 12:13 am on Sep 17, 2000 (gmt 0)


That's the Ink spider that retrieves new submissions, been around for a while now, you must have submitted pages recently.


 7:40 pm on Sep 20, 2000 (gmt 0)

Please clarify for me if Slurp.so is the permanent database for Ink? I'm hoping so, hit very hard on September 17 by both Slurp.so and Slurp.

And could you tell me more about the great white crawl?


 8:00 pm on Sep 20, 2000 (gmt 0)

websurfer - You have an interesting format for IP logging. Are you running a home grown logging script? Is that the raw perl local(time) function? Just curious.


 6:55 am on Sep 21, 2000 (gmt 0)

I got hit by j6000.inktomi.com, slurp/si and slurp/so on the same day (19th) for a site which dropped out of the DB a week ago and I deliberately have not resubmitted. I wonder what will happen now?

Global Options:
 top home search open messages active posts  

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved