Ways to stop spiders crawling database

Other than cloaking!

[hurlimann]

We need to stop spiders from crawling db generated pages from mysql on apache server.

Any ideas of the best way?

[gcross]

Try sticking these lines into your generated pages.

<meta http-equiv="PRAGMA" content="NO-CACHE" />
<meta name="robots" content="noindex, nofollow" />

Also, there is usually a robots.txt file in the root directory. That may be the best place to enter your instruction.