Forum Moderators: goodroi
Do I include spaces in the filenames of the robots.txt file or do I represent spaces with the "+" symbol. I'm not sure if it matters in the robots.txt file but in Webtrends the "+" messed up the reporting.
Ex.
User-agent: *
Disallow: /Solutions/Solutions+by+Industry/FS/Credit+Test.htm
OR
Disallow: /Solutions/Solutions by Industry/FS/Credit Test.htm
Anyone have any experience with this? Thanks!
So besides changing all my filenames to suit the robots.txt file, does anyone has experience or knowledge with FILENAMES WITH SPACES and robots.txt syntax.
Thanks
We also already use robots META tags, but we want use both the robots.txt and META tags in combination as some search engines are finding duplicate results (we have print version of some content and duplicate information in different channels for navigation reasons) and we don't want to get penalized.
Thanks