Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: open
The Slurp info page says:
Slurp now has the ability to crawl dynamic links or dynamically generated documents. It will not, however, crawl them by default.
I'm not sure what that means. Maybe it crawls them for links but doesn't actually list them?
every other spider does the second, but because inktomi does the first it gets a 404.
Is there a reason it does that?
google, ask have done it properly , as well as wisenutbot.
anyone have a clue?
A search for aluminum element [search.positiontech.com] returns me lovely spammy results :)
On the second page, there are several those-long-titled-results which redirect to one commercial site..
Is it so easy to spam ink?