Welcome to WebmasterWorld Guest from 23.20.147.6

Forum Moderators: goodroi

Message Too Old, No Replies

How to setup robots.txt?

     
6:07 am on Jun 27, 2008 (gmt 0)

New User

5+ Year Member

joined:June 27, 2008
posts:1
votes: 0


Hi,

I know I would probably upload the text file to my ftp public, but what goes into the rebot.txt file? Correct me if I'm wrong on that first part please..

Thanks for any comments,

Cam

10:12 am on June 27, 2008 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 28, 2002
posts:505
votes: 0


Hi Cameron, welcome to W.W.

as a first step, I would rename the "rebot.txt" to "robots.txt".

And a second step could be to type "robots.txt" into the search field of www.google.com, with plenty of information coming back then.

Good luck and kind regards,
R.

10:23 am on June 27, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 27, 2002
posts:1187
votes: 0


There is some good info here [w3.org]

Do be shy, it's OK to look at other sites robots.txt files. www.example.com/robots.txt will open it up.

Welcome and Enjoy

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members