Page is a not externally linkable
smallcompany - 5:28 pm on Feb 10, 2009 (gmt 0)
Disallow: /*? Depending on your situation, there may be more value in Jim's response about 301. In my example, I have URLs that come from paid search where I throw variables on so I know which campaign, ad group, keyword, etc. the traffic and sale has come from. Since I already have my original page, and the rest is all same page with variables attached (duplicate content), I believe that 301 solution would fit better, rather than blocking via robots.txt. It would be great if someone could comment more on the difference. Thanks
If you want to use robots.txt for that "clean up", this should be the right way: