Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: goodroi
What you are using is not even robots.txt, but the meta name="robots". In short: That's not the appropriate solution to your problem.
Robots.txt is a file, which you put in the root of your pages. With robots.txt you can set rules for spiders as to which directories and files they are allowed to visit.
Since robots.txt is based on a convention spiders should adhere to those rules, but rogue bots often just don't care.
Anyhow, here's all about robots.txt [searchengineworld.com]
I would suggest however to look into a solution with .htaccess.
If a page is disallowed in robots.txt, no 'good' bot will not be confused, since it will never fetch that page and thus never see the <meta robots> tag.
Validate your robots.txt file [searchengineworld.com]. If it valid, I'd suspect some other problem, such as alias link-paths to the files you have disallowed.
Bad 'bots pay no attention to either method of robots control, so do not base your debugging decisions on their behaviour.