Is it the blind leading the blind [hotwired.lycos.com]?
"For those of you who don't know, search engines use programs
known as spiders or crawlers to index content on Web sites.
Crawlers usually avoid pages with URLs that contain /cgi-bin or
bin (and other variants), as well as CGI escape characters like &,
?, =, etc. These pages often lead to massive databases with
recursive links that can easily trap crawlers in a maze of data.
Sometimes this poses a threat to the crawler, but more often
than not, a trapped spider will simply bring a server to its knees
in a couple of minutes. Although database creators usually don't
intend to trap search-engine crawlers, Bad People sometimes
create recursive crawlspaces for no other reason than pure
mischief. Generally, it's just a bad idea for a spider to follow
Unfortunately, there's not much you can do right now to get
the crawlers to index your site correctly. It may be tempting to
try to trick them into believing the pages are static HTML, but I
don't recommend this, since you still run the risk of endangering
your server if the crawler becomes trapped. "