Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: open
Thanks for the input. Anybody got anything else on Slurp/si? Is it just a link checker and 404 remover as Pete says?
BTW Pete I think the Slurp/so is the spider for Goo or I am led to beleive that by the following post.
Please look at Littleman's post to me.
2) Sat Sep 3 18:46:29 2000 --¦Slurp/cat (email@example.com;http://www.inktomi.com/slurp.html)¦220.127.116.11
30 Sat Sep 4 18:46:44 2000 --¦Slurp/cat (firstname.lastname@example.org;http://www.inktomi.com/slurp.html)¦18.104.22.168
My point here is notice the time and date, what is the purpose of spidering (same page all three times btw) almost exactly 24 hours apart?