Order allow,deny or deny,allow

Not sure which order is best...

[whoisgregg]

I'm not too concerned with any good bots but a couple, so I want to use a

User-agent: *
Disallow: /

and then two or three entries like this:

User-agent: SomeFriendlyBotUA
Disallow:

Which comes first? Yahoo Slurp claims [help.yahoo.com] it only follows either the first Slurp or the first * entry, do all bots work that way (meaning I need to put the * rule last)? Or is Slurp the exception?

[jdMorgan]

Generally, all major robots will accept the first record that matches the robot's name or "*" -- whichever comes first. Some robots are 'smart', and in an attempt to compensate for common webmaster errors, will actually give priority to the record containing the specific robot's name, and if none is found, then accept the policy given for "*" if present.

However, the safe way to design your robots.txt is to assume that the robot will accept it's specific policy or "*", whichever comes first.

Jim

[whoisgregg]

So, "*" entries should always be last because everything that follows one will tend to be ignored.

Thanks jdMorgan! :D