Hello,
I´ve reading around trying to solve the problem in
this post [webmasterworld.com].
After working out what to do with googlebot, I did a yahoo search and found the same problem; yahoo is also indexing al my /post/trackback and /post/feed pages on my wordpress blogs.
I was wondering if disallowing all robots from those pages might help using this:
User-agent: *
Disallow: /*/feed/
Disallow: /*/trackback/
But I´m not sure wether this might also affect rss aggregation sites like bloglines,newsgator,etc.
Should I just disallow only for msnbot,slurp and googlebot? Do MSNbot and Slurp accept wildcards in Disallow directives?
Thanks!
