Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: goodroi
(with many more directories disallowed).
In this syntax, does the star at the end of the user-agent line still disallow all other robots from the following directories?
Does adding the "Mediapartners-Google" bit allow that robot anything, or specifically disallow the robot from the following directories?
The old webmaster seems to think this allows google more access, but I've never encountered that syntax before and just want to be sure.
If you want to exclude all bots from that list of directories, then you simply need:
I'm still looking for clarification, though, on the * that appears in the line "User-agent: Mediapartners-Google*"
Does that star add all the other bots to the disallows? If not, what service does it provide in this syntax?