Welcome to WebmasterWorld Guest from 54.158.31.149

Forum Moderators: open

Message Too Old, No Replies

Slurp/si and Slurp/cat

WOW

     
3:28 pm on Aug 30, 2000 (gmt 0)

Full Member

10+ Year Member

joined:July 22, 2000
posts:329
votes: 0


Slurp/si and Slurp/cat are spidering like crazy. started about 6:00am PT and is following everylink on every page.
WOW is has been a long time since I have seen slurp usually it is just the submission spider.. Something must be up today. Any body else getting hit hard?

Regards

Dr. Bill

9:11 pm on Aug 30, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38070
votes: 16


I have also just noticed a huge run from Slurp on several domains - full complete crawls.
9:52 pm on Aug 30, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 17, 2000
posts:2924
votes: 0


Unfortunately I don't think this is going to be the golden goose. All the spidering is coming out of Japan (goo3**.goo.ne.jp 202.212.5.**) - when this has happen before it was to maintain the Japanese db. This pattern seems very similar to the other times.
12:15 am on Aug 31, 2000 (gmt 0)

Full Member

10+ Year Member

joined:July 22, 2000
posts:329
votes: 0


Hi,

I am not seeing goo in the listings at all.. do they have a spider of their own?

1:18 am on Aug 31, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 17, 2000
posts:2924
votes: 0


Dr Bill,
It will come in as Slurp, but the IP and host name are out of Japan. It still is inktomi, but it is an isolated database for the Japanese portals.
1:24 am on Aug 31, 2000 (gmt 0)

Full Member

10+ Year Member

joined:July 22, 2000
posts:329
votes: 0


Hi Littleman.

Is the Slurp/si or Slurp/cat or both?

1:57 am on Aug 31, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 17, 2000
posts:2924
votes: 0


The Japanese Slurp I have seen over the last couple of days is actually Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...] - they hit one server 26 thousand times over the last two days - I really hate that they do that, its almost like a DOS. Slurp/si only came in 250 times on the same server. There have been no visits from Slurp/cat for me. So maybe something is happening on your particular sight? That would be nice, eh?
3:06 pm on Aug 31, 2000 (gmt 0)

Junior Member

10+ Year Member

joined:July 28, 2000
posts:134
votes: 0


Ha ha littleman, 26k in two days huh? it only happens when you have large number of subdomains setup
I wonder what you're doing...
;)
5:31 pm on Aug 31, 2000 (gmt 0)

Full Member

10+ Year Member

joined:July 22, 2000
posts:329
votes: 0


Littleman,

Sofar I have had close to 30,000 pages taken and it is still going. I hope it never stops :)

1:24 am on Sept 2, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38070
votes: 16


I've been visited by the entire stable of inktomi spiders in the last 5 days. (notice the very last one).

hits id : ip domain agent
78 SLURP: 216.35.116.108 wm3023.inktomi.com Slurp/cat (slurp@inktomi.com; [inktomi.com...]
53 SLURP: 216.35.116.105 wm3020.inktomi.com Slurp/cat (slurp@inktomi.com; [inktomi.com...]
40 SLURP: 216.35.103.77 j4024.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
40 SLURP: 216.35.103.78 j4025.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
39 SLURP: 216.35.103.54 j4014.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
39 SLURP: 216.35.103.58 j4018.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
37 SLURP: 216.35.103.52 j4012.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
37 SLURP: 216.35.103.60 j4020.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
34 SLURP: 216.35.103.53 j4013.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
34 SLURP: 216.35.103.74 j4021.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
32 SLURP: 216.35.103.59 j4019.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
32 SLURP: 216.35.103.75 j4022.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
31 SLURP: 216.35.103.55 j4015.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
31 SLURP: 216.35.103.57 j4017.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
31 SLURP: 216.35.103.76 j4023.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
31 SLURP: 216.35.116.53 j3013.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
30 SLURP: 216.35.103.56 j4016.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
28 SLURP: 216.35.116.58 j3018.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
26 SLURP: 216.35.116.43 j3003.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
26 SLURP: 216.35.116.54 j3014.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
25 SLURP: 216.35.103.51 j4011.inktomisearch.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
24 SLURP: 216.35.116.57 j3017.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
23 SLURP: 216.35.116.49 j3009.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
23 SLURP: 216.35.116.55 j3015.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
22 SLURP: 216.35.116.41 j3001.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
21 SLURP: 216.35.116.42 j3002.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
20 SLURP: 216.35.116.52 j3012.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
19 SLURP: 216.35.116.45 j3005.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
19 SLURP: 216.35.116.48 j3008.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
19 SLURP: 216.35.116.50 j3010.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
18 SLURP: 216.35.116.59 j3019.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
16 SLURP: 216.35.116.44 j3004.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
15 SLURP: 216.35.116.46 j3006.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
13 SLURP: 216.35.116.47 j3007.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
13 SLURP: 216.35.116.51 j3011.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
13 SLURP: 216.35.116.88 wm3008.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
12 SLURP: 209.185.143.96 q2000.inktomisearch.com Slurp/si (slurp@inktomi.com; [inktomi.com...]
12 SLURP: 216.35.103.71 j110.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
12 SLURP: 216.35.103.72 j111.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
11 SLURP: 216.35.116.56 j3016.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
10 SLURP: 216.35.116.104 wm3019.inktomi.com Slurp/cat (slurp@inktomi.com; [inktomi.com...]
7 SLURP: 216.35.116.106 wm3021.inktomi.com Slurp/si (slurp@inktomi.com; [inktomi.com...]
6 SLURP: 202.212.5.30 goo212.goo.ne.jp Slurp/si (slurp@inktomi.com; [inktomi.com...]
6 SLURP: 209.185.143.84 j5004.inktomisearch.com Slurp/si (slurp@inktomi.com;
6 SLURP: 216.35.103.61 j100.inktomi.com Slurp/si
6 SLURP: 216.35.103.62 j101.inktomi.com Slurp/si
6 SLURP: 216.35.103.73 j5006.inktomi.com Slurp/si
6 SLURP: 216.35.103.79 si4000.inktomi.com Slurp/si
6 SLURP: 216.35.103.80 si4001.inktomi.com Slurp/si
6 SLURP: 216.35.103.81 si4002.inktomi.com Slurp/si
6 SLURP: 216.35.116.103 wm3018.inktomi.com Slurp/si
6 SLURP: 216.35.116.109 wm3024.inktomi.com Slurp/si
6 SLURP: 216.35.116.90 si3000.inktomi.com Slurp/si
6 SLURP: 216.35.116.91 si3001.inktomi.com Slurp/si
6 SLURP: 216.35.116.92 si3002.inktomi.com Slurp/si
6 SLURP: 216.35.116.93 si3003.inktomi.com Slurp/si
5 SLURP: 209.1.13.232 j301.inktomi.com Slurp.so/1.0
4 SLURP: 209.1.13.231 j300.inktomi.com Slurp.so/1.0
4 SLURP: 209.185.122.201 q2005.inktomi.com Slurp/si
3 SLURP: 216.35.116.106 wm3021.inktomi.com Slurp/cat (slurp@inktomi.com; [inktomi.com...]
2 SLURP: 209.131.48.140 crusade.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
2 SLURP: 216.35.116.112 y2300.inktomi.com Slurp/3.0 (slurp@inktomi.com; [inktomi.com...]
SLURP: 164.124.250.232 Slurp/si (slurp@inktomi.com; [inktomi.com...]
SLURP: 209.185.141.226 y400.inktomi.com Slurp/2.0-KiteWeekly (slurp@inktomi.com; [inktomi.com...]
SLURP: 209.67.206.127 j521.inktomi.com Slurp.so/1.0 (slurp@inktomi.com; [inktomi.com...]
SLURP: 211.169.241.21 Slurp/si (slurp@inktomi.com; [inktomi.com...]
y2404.inktomi.com Slurp/2.0-GreatWhiteCrawl (slurp@inktomi.com; [inktomi.com...]

1:01 pm on Sept 14, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38070
votes: 16


Any one else seen "the great white crawl"?
1:12 pm on Sept 14, 2000 (gmt 0)

Full Member

10+ Year Member

joined:July 22, 2000
posts:329
votes: 0


Brett,

I have yet to see Great White :( Would like him/her to visit me.

10:35 pm on Sept 16, 2000 (gmt 0)

New User

10+ Year Member

joined:Feb 13, 2004
posts:5
votes: 0


Inktomi has been poundingmy sites as well, not as much as others are saying, however my site isn't really that big either. Out of those different spiders, which one are the good ones?
10:38 pm on Sept 16, 2000 (gmt 0)

New User

10+ Year Member

joined:Feb 13, 2004
posts:5
votes: 0


Do you have a IP Address for that great white??
10:59 pm on Sept 16, 2000 (gmt 0)

New User

10+ Year Member

joined:Feb 13, 2004
posts:5
votes: 0


Has any one seen this

Tue Sep 12 07:43:25 2000 -- ¦Mozilla/4.72 [en] (X11; U; NetBSD 1.4.2 i386; Nav)¦209.185.141.185¦

When I do a look up on the IP address it comes back as j6000.inktomi.com what's up with that. ??

Air

12:13 am on Sept 17, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 10, 2000
posts:1253
votes: 0


Websurfer,

That's the Ink spider that retrieves new submissions, been around for a while now, you must have submitted pages recently.

7:40 pm on Sept 20, 2000 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 12, 2000
posts:395
votes: 0


Please clarify for me if Slurp.so is the permanent database for Ink? I'm hoping so, hit very hard on September 17 by both Slurp.so and Slurp.

And could you tell me more about the great white crawl?

8:00 pm on Sept 20, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 17, 2000
posts:2924
votes: 0


websurfer - You have an interesting format for IP logging. Are you running a home grown logging script? Is that the raw perl local(time) function? Just curious.
6:55 am on Sept 21, 2000 (gmt 0)

Senior Member

joined:July 21, 2000
posts:1402
votes: 0


I got hit by j6000.inktomi.com, slurp/si and slurp/so on the same day (19th) for a site which dropped out of the DB a week ago and I deliberately have not resubmitted. I wonder what will happen now?
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members