homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Include Robots.txt in my site

5+ Year Member

Msg#: 3987397 posted 2:38 pm on Sep 10, 2009 (gmt 0)


Guys i just include robots.txt & sitemap.xml in my website is there any benefit !

or these are useless



WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 3987397 posted 12:00 pm on Sep 11, 2009 (gmt 0)

Welcome to WebmasterWorld ravikant!

Robots.txt and sitemaps are tools. They may or may not be useful depending on what you need and how you use them.

Robots.txt can help you better control what files the search engines crawl on your website.

Sitemaps can help you expose your content to search engine crawlers. It does not guarantee the search engines will index the content but a sitemap can help the search engines discover your content.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved