My web host notified me they will be down for most of a day making upgrades. I don't want to chance the Slurp PFI spider trying to index my site at that time which would put my PFI pages out of commission until the next update.
Is there some way I can tell the Slurp PFI spider to stay away and not try to do a regular update? I assume using the robot.txt file to exclude Slurp would cause my pages to vanish from the Inktomi index.
tigger
9:02 am on Aug 26, 2002 (gmt 0)
I had a problem a little while ago with a hosting company and I contact PT regarding my ink pages, nothing could be done and the pages where offline for about 5 days.
