Welcome to WebmasterWorld Guest from 23.20.238.193

Forum Moderators: goodroi

Message Too Old, No Replies

How to setup robots.txt?

     
6:07 am on Jun 27, 2008 (gmt 0)

5+ Year Member



Hi,

I know I would probably upload the text file to my ftp public, but what goes into the rebot.txt file? Correct me if I'm wrong on that first part please..

Thanks for any comments,

Cam

10:12 am on Jun 27, 2008 (gmt 0)

10+ Year Member



Hi Cameron, welcome to W.W.

as a first step, I would rename the "rebot.txt" to "robots.txt".

And a second step could be to type "robots.txt" into the search field of www.google.com, with plenty of information coming back then.

Good luck and kind regards,
R.

10:23 am on Jun 27, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There is some good info here [w3.org]

Do be shy, it's OK to look at other sites robots.txt files. www.example.com/robots.txt will open it up.

Welcome and Enjoy

 

Featured Threads

Hot Threads This Week

Hot Threads This Month