| 5:37 am on Sep 7, 2000 (gmt 0)|
What you really want is Slurp.so, thats the deep spider which usually gets you into permanent database.
Slurp/si methinks is just link checker and 404 cleaner
but who knows with the new Inktomi 500mln index things may have changed.
| 1:33 pm on Sep 7, 2000 (gmt 0)|
Thanks for the input. Anybody got anything else on Slurp/si? Is it just a link checker and 404 remover as Pete says?
BTW Pete I think the Slurp/so is the spider for Goo or I am led to beleive that by the following post.
Please look at Littleman's post to me.
| 2:52 pm on Sep 7, 2000 (gmt 0)|
drbill, yes I think you are led to believe ;)
Slurp.so from j30xx.inktomi.com 216.32.116.yy
xx-starts from 01
yy starts from 41
this is not japanese spider, it goes in many levels deep
and I it used to be money, with new inktomi time will tell.
| 5:22 pm on Sep 10, 2000 (gmt 0)|
hey all :-) here is something I have noticed, any and all input is greatly appreciated. From my log files;
1) Sat Sep 2 18:46:02 2000 --¦Slurp/cat (email@example.com;http://www.inktomi.com/slurp.html)¦184.108.40.206
2) Sat Sep 3 18:46:29 2000 --¦Slurp/cat (firstname.lastname@example.org;http://www.inktomi.com/slurp.html)¦220.127.116.11
30 Sat Sep 4 18:46:44 2000 --¦Slurp/cat (email@example.com;http://www.inktomi.com/slurp.html)¦18.104.22.168
My point here is notice the time and date, what is the purpose of spidering (same page all three times btw) almost exactly 24 hours apart?
| 5:24 pm on Sep 10, 2000 (gmt 0)|
ooppsss, the take note, the day in my previous post should be sat, sun and monday.
| 1:41 pm on Sep 11, 2000 (gmt 0)|
Since Slurp/cat started (10-12 days ago) I have seen a big boost in GOTO traffic. Not sure what is going down...
| 10:25 pm on Sep 11, 2000 (gmt 0)|
Slurpie finally got around to a couple of new domains (3rd level) that we'd been submitting quite heavily three weeks ago (if that is any help).
| 1:33 am on Sep 14, 2000 (gmt 0)|
Well Looks like Slurpie (as Brett calls it) has struck again.
26K sofar today and still going. BTW these are all 3rd level domains. Damn I hope this puts some major pages in INK.