sinyala1 - 8:21 pm on May 3, 2002 (gmt 0)

Oh yeah...use wild cards. I programmed mine to use wild cards. Here's some listings in my spider trap.

scooter2.* 11 Altavista
scooter.* 11 Altavista
scooterr.* 11 Altavista
bigip1-snat.* 11 Altavista
vscooter.* 12 Altavista
*.sv.av.com 11 Altavista

The * is the wild card. 11 and 12 is how many times it was hit. The * can be anything.

63.173.190.16 <<--- that hit my spider trap 27 times in the past 3 months and only search engines know about it.

DNS wild cards are easy. You can take out most every single spider google has by this DNS:

*.googlebot.com

Agents can be faked, DNS cannot.