homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

why use and what is robots.txt
this maybe stupid question

 12:31 am on May 7, 2008 (gmt 0)

this maybe stupid question

Hi everyone I stubble on this section of the forum, I have no about robo.txt and also it mention robo.text is good SEO can any body explain

And were can I find more beginner information

Thank you



 4:41 pm on May 7, 2008 (gmt 0)

No such thing as a stupid question, my friend!

Near the top of the page for every forum here at WebmasterWorld you will find some links to the Forum Charter and the Forum Library that you are viewing. This forum is no different and you will find quite a bit of introductory information regarding robots.txt in the robots.txt forum Charter [webmasterworld.com]


 4:45 pm on May 7, 2008 (gmt 0)

These days I might be a bit leery about using my robots.txt for too much. No, I'm more apt to do something a bit more technical and not put a directory list in there for prying eyes to see. Especially Googlebot and it's damn URI only listings for references discovered via the robots.txt file.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved