Forum Moderators: martinibuster

Message Too Old, No Replies

AS in your robots.txt file

what user agent name are you using?

         

wizarddave

8:36 pm on Oct 17, 2005 (gmt 0)



I've recently added a "permalink" function to one of my blogs. In an effort to prevent the permalink pages from being read as duplicate content in the SERPs, I have given the permalink pages their own directory.

When I create the robots.txt file to allow the AS bot everything and then deny /blog/ to all other bots, what string should I use for the AS bot? Anyone care to share their robot.txt files that they know work?

Thanks!

AdSenseAdvisor

12:04 am on Oct 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



wizarddave - The following AdSense help page will help you grant access to the AdSense crawler through your robots.txt file:

[google.com...]