Slurp not hitting dynamic pages anymore?

My logs are telling me it's fact.

[Grumpus]

When I paid for my INK listing, all of the few dozen pages I had in their index vanished except for the one I paid for. After stickying Intomi(guy) here, Slurp's been active at my site and more pages are starting to show up.

Oddly, though, slurp's not hitting the pages it used to - it's only tagging ones that don't have a ? in the URL. (And I assure you, it DID crawl them back in March before I paid). It is TRYING to crawl them as I can see it hitting pages that bots (and people) could only get to by trying a link with a ? in it. Thankfully, I've developed the site so that if there's no parameter passed that it displays default data. The bad news here is that Slurp thinks it's hitting a link that goes to "X" and is getting "Y" content, then is hitting a link that goes to "Z" and is still getting "Y" content.

I know it's not something on my end as google, fast, altavista and all the others have no trouble getting through (it's a matter of whether they bother to, not whether they can).

Anyone else with dynamic pages seeing the same thing? I'm not super worried as the pages Slurp is getting are the ones that are indexes to the newest content, but it'd be nice to have the pages crawled in the proper way and not run the risk of duplicate content penalties because their crawler is inept.

G.

[thejenn]

I'd be curious to here responses to this as well.

To the best of my knowledge, Slurp only spiders dynamic pages if you have used the Paid Inclusion option...

If you were having the spidered for free...that would be very interesting...

[gsx]

Slurp does only spider dynamic pages if they are paid for. Log in to your account and download the search engine information manual from the first page, it is confirmed within that document.

[Grumpus]

Actually, I've been getting two different kinds of hits from Slurp since I posted this.

1) What I was talking about before - she comes and slurps up my paid page and some links from it, stripping the "?" and everything after it. This means that I have all of my "site map" pages in the index and the first entry in the database for each of my detail pages.

2) Slurp comes at another time and crawls any dynamic pages that are listed in the DMOZ.

Strange, but true.

G.