Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

How to setup robots.txt?



6:07 am on Jun 27, 2008 (gmt 0)

5+ Year Member


I know I would probably upload the text file to my ftp public, but what goes into the rebot.txt file? Correct me if I'm wrong on that first part please..

Thanks for any comments,



10:12 am on Jun 27, 2008 (gmt 0)

10+ Year Member

Hi Cameron, welcome to W.W.

as a first step, I would rename the "rebot.txt" to "robots.txt".

And a second step could be to type "robots.txt" into the search field of www.google.com, with plenty of information coming back then.

Good luck and kind regards,


10:23 am on Jun 27, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

There is some good info here [w3.org]

Do be shy, it's OK to look at other sites robots.txt files. www.example.com/robots.txt will open it up.

Welcome and Enjoy


Featured Threads

Hot Threads This Week

Hot Threads This Month