Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: open
we use the ppi for our index page.
in the last 3 months i have been making several server modifications, (e.g. setting up quotas) we have also had one very dodgy afternoon with network problems from our serverhost.
all this resulted in our server being unreachable several times, each for 2 minutes, whilst i rebooted after making modifications and once for several hours during the network connectivity problems.
it appears that the fast crawler was on our site when this happened - which meant immediate dropping from their index.
i wrote to lycos ppi service and was told that if a (ppi) page could not be found or if the spider was thrown off for some reason (e.g. reboot), it would be removed from the index forthwith.
we then had to wait a week for the spider to recrawl our ppi page and we were then reincluded.
i am not sure if anyone can help here ;-)
but i did want to get this off my chest. this is really frustrating - it has never happened with any other engine. and in the case of implementing server quotas - i HAD to reboot the server.
here's hoping someone from fast might read this and is able to consider changing the situation.
p.s. i did mention this in my last mail to the lycos ppi help service, and they said they would pass it on to fast. but the same thing just happened to us again (1 month later)
joined:Oct 23, 2000
I really don't think it would be a major problem as the crawler picks you right up again on the next visit - which usually happens on more frequent basis that once a week.
" jamie, you gotta keep that server running 24/7 and you can't blame them for throwing out a page that's not there when they try to fetch it :) "
at the end of the day i know, i know ;-)
but google / altavista /inktomi have never dropped us for that.
anyway - i hope the server mods are finished now, so we can look forward to a long period of uninterupted spidering!