PeteU
msg:18549 | 5:37 am on Sep 7, 2000 (gmt 0) |
What you really want is Slurp.so, thats the deep spider which usually gets you into permanent database.
Slurp/si methinks is just link checker and 404 cleaner
but who knows with the new Inktomi 500mln index things may have changed.
|
drbill
msg:18550 | 1:33 pm on Sep 7, 2000 (gmt 0) |
Hey Pete Thanks for the input. Anybody got anything else on Slurp/si? Is it just a link checker and 404 remover as Pete says? BTW Pete I think the Slurp/so is the spider for Goo or I am led to beleive that by the following post. Slurp/so [webmasterworld.com]
Please look at Littleman's post to me. LMK :) Regards Dr. Bill
|
PeteU
msg:18551 | 2:52 pm on Sep 7, 2000 (gmt 0) |
drbill, yes I think you are led to believe ;)
Slurp.so from j30xx.inktomi.com 216.32.116.yy
xx-starts from 01
yy starts from 41
this is not japanese spider, it goes in many levels deep
and I it used to be money, with new inktomi time will tell.
|
websurfer
msg:18552 | 5:22 pm on Sep 10, 2000 (gmt 0) |
hey all :-) here is something I have noticed, any and all input is greatly appreciated. From my log files;
1) Sat Sep 2 18:46:02 2000 --¦Slurp/cat (slurp@inktomi.com;http://www.inktomi.com/slurp.html)¦216.35.116.108 2) Sat Sep 3 18:46:29 2000 --¦Slurp/cat (slurp@inktomi.com;http://www.inktomi.com/slurp.html)¦216.35.116.108 30 Sat Sep 4 18:46:44 2000 --¦Slurp/cat (slurp@inktomi.com;http://www.inktomi.com/slurp.html)¦216.35.116.108 My point here is notice the time and date, what is the purpose of spidering (same page all three times btw) almost exactly 24 hours apart?
|
websurfer
msg:18553 | 5:24 pm on Sep 10, 2000 (gmt 0) |
ooppsss, the take note, the day in my previous post should be sat, sun and monday.
|
drbill
msg:18554 | 1:41 pm on Sep 11, 2000 (gmt 0) |
websurfer, Since Slurp/cat started (10-12 days ago) I have seen a big boost in GOTO traffic. Not sure what is going down...
|
Brett_Tabke
msg:18555 | 10:25 pm on Sep 11, 2000 (gmt 0) |
Slurpie finally got around to a couple of new domains (3rd level) that we'd been submitting quite heavily three weeks ago (if that is any help).
|
drbill
msg:18556 | 1:33 am on Sep 14, 2000 (gmt 0) |
Well Looks like Slurpie (as Brett calls it) has struck again. 26K sofar today and still going. BTW these are all 3rd level domains. Damn I hope this puts some major pages in INK.
|
|
