Forum Moderators: Robert Charlton & goodroi
I am worried that letting mediabot into my site will result in the site being indexed in Google.
My robots.txt looks like this - is it correct?
User-agent: *
Disallow: /
User-agent: Mediapartners-Google*
Disallow:
User-agent: Slurp
Disallow:
I really don't want to pick up a duplicate site penalty with another site that is already in Google.