Spidering Dynamic pages by Inktomi

How can I get my dynamic pages indexed by an inktomi 4.3 spider

[marston]

Hello, For my websites internal search service we use the inktomi 4.3 spider, which works quite well with dynamic pages. The problem is that we can't convince it to spider some dynamic pages on our site even though we have told it to index dynamic urls. Has anybody been successful at this?

The problem may be with the nature of the dynamic pages. They are generated through a servlet which makes a call to a database. The page generated is complete with metatags. We are using apache server.

Here is a sample dynamic page:
[publicbroadcasting.net...]

Would cloaking help or do we need to generate a static copy of the page for indexing?

[Brett_Tabke]

Welcome to the board.

Static page be one answer, but I can certainly understand how much extra work that would involve.

As far as I know, the Ink product _should_ be able to spider those pages. I would double check a) what the server is kicking out for headers on those pages. It may be doing something that the spider isn't tuned for. b) walk back through the options panel and double check things like redirects, dynamic url indexing. (it's been awhile, and maybe a version since I've seen the product - so I'm not sure what options are available today).

After that, get on the phone with ink support and see what they say.