homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

warning; robots.txt containing DOS or Mac line

 11:03 am on Apr 24, 2001 (gmt 0)

I've created a robots.txt file using Notepad with the .txt extension and the following entries:

User-agent: *
Disallow: /

When trying to validate the robots.txt at the searchengineworld website, I received the following warning:

warning Robots.txt contains DOS or Mac line enders. Although it is so common most spiders will deal correctly with the file, it is not a valid format.

I would appreciate if someone can help me. Thank you very much and have a great day!




 1:06 pm on Apr 24, 2001 (gmt 0)

DOS or Mac based systems expect each line in a text file to end with a carrige return and a line feed character. *nix based systems expect only a line feed. When you FTP a file using binary mode the file is transferred exactly as created. Text files should be uploaded to *nix systems using ASCII mode which strips out the unneeded carrige returns.

The error you saw is probably due to a transfer using binary mode.


 10:25 pm on Apr 24, 2001 (gmt 0)

I would like to take this opportunity to greatfully thank you for your assistance regarding the robots.txt matter. I did exactly what you recommended and it worked fine. Once again, thank you very much!

Sincerely yours,



 8:10 am on Apr 25, 2001 (gmt 0)

You're very welcome roma! I forgot to say welcome to WmW too, sorry. We're pleased to have you join us. Read and learn and contribute where you can to help others learn too!


 10:35 am on Apr 25, 2001 (gmt 0)

Thank you very much for welcoming me to Webmaster World.com. It would be my pleasure to offer my share of assistance on things I know. Have a wonderfull day and once again, thank you very much for the assistance!

Sincerely yours,


Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved