tigertom - 10:25 pm on Jan 17, 2007 (gmt 0)

Ok, a simple solution.

To make sure, just add the extra Disallow directives to the Slurp block (and the other Bots'). Problem solved.

User-agent: *
Disallow: /shop/
Disallow: /forum/
Disallow: /cgi-bin/
Disallow: /badbottrap/

User-agent: Slurp
Disallow: /bloop/
Disallow: /blop/
Disallow: /shop/
Disallow: /forum/
Disallow: /cgi-bin/
Disallow: /badbottrap/

User-agent: msnbot
Disallow: /bloop/
Disallow: /blop/
Disallow: /shop/
Disallow: /forum/
Disallow: /cgi-bin/
Disallow: /badbottrap/

I think: Put the wildcard ones first, for all the other, less important bots. Then you put the bot-specific ones.

If a bot comes across its name, it'll stop there.

I _think_ that's right :)

My objective here: To stop the most important SE bots indexing the insubstantial CMS pages in /bloop/ and /blop/, except the Adsense bot :).

Thanks for the help.