Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

why use and what is robots.txt

this maybe stupid question

12:31 am on May 7, 2008 (gmt 0)

New User

5+ Year Member

joined:Oct 3, 2007
posts: 29
votes: 0

this maybe stupid question

Hi everyone I stubble on this section of the forum, I have no about robo.txt and also it mention robo.text is good SEO can any body explain

And were can I find more beginner information

Thank you

4:41 pm on May 7, 2008 (gmt 0)


WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
votes: 0

No such thing as a stupid question, my friend!

Near the top of the page for every forum here at WebmasterWorld you will find some links to the Forum Charter and the Forum Library that you are viewing. This forum is no different and you will find quite a bit of introductory information regarding robots.txt in the robots.txt forum Charter [webmasterworld.com]

4:45 pm on May 7, 2008 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts: 12166
votes: 51

These days I might be a bit leery about using my robots.txt for too much. No, I'm more apt to do something a bit more technical and not put a directory list in there for prying eyes to see. Especially Googlebot and it's damn URI only listings for references discovered via the robots.txt file.