Forum Moderators: open

Message Too Old, No Replies

Inktomi Slurp

Keeps visiting, but no listing!

         

KMxRetro

7:37 am on Aug 2, 2002 (gmt 0)

10+ Year Member



Hi folks,
For the last 2 months, the Inktomi Slurp spider/robot has visited my site EVERY day, picking up at least 1 page. Yesterday, it picked up 16 pages, which is a significant jump.

I'm not listed in any of the Inktomi powered engines yet though. Is this happening to everyone? Does spidering guarantee a listing?

jdMorgan

7:46 am on Aug 2, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



KMxRetro,

Yeah, I'd guess you'll have to wait another month before you see a listing - they are SLOW! Also,
it seems like they NEVER give up on files long-gone and 404'ed. I had them in my logs almost every
day for three months, and then bingo! - popped into their index and immediately showed up in all of
the other search engines that ink feeds. All of my non-robots.txt-Disallowed pages showed up, too.

I don't know if it's a guaranteed thing, but it can happen (albeit slowly).

Jim

Torben Lundsgaard

7:49 am on Aug 2, 2002 (gmt 0)

10+ Year Member



If you have paid for spidering your site will apear in all Inktomi powered search engines after about 72 hours. In the case you haven't paid it's normal that Slurp is fooling around for some time before the pages are listet.

tigger

8:01 am on Aug 2, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



-Slurp is fooling around for some time before the pages are listed-

if ever !