Welcome to WebmasterWorld Guest from 220.127.116.11
Forum Moderators: open
The Slurp info page says:
Slurp now has the ability to crawl dynamic links or dynamically generated documents. It will not, however, crawl them by default.
I'm not sure what that means. Maybe it crawls them for links but doesn't actually list them?
every other spider does the second, but because inktomi does the first it gets a 404.
Is there a reason it does that?
google, ask have done it properly , as well as wisenutbot.
anyone have a clue?