| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Include Robots.txt in my site
| 2:38 pm on Sep 10, 2009 (gmt 0)|
Guys i just include robots.txt & sitemap.xml in my website is there any benefit !
or these are useless
| 12:00 pm on Sep 11, 2009 (gmt 0)|
Welcome to WebmasterWorld ravikant!
Robots.txt and sitemaps are tools. They may or may not be useful depending on what you need and how you use them.
Robots.txt can help you better control what files the search engines crawl on your website.
Sitemaps can help you expose your content to search engine crawlers. It does not guarantee the search engines will index the content but a sitemap can help the search engines discover your content.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
© Webmaster World 1996-2014 all rights reserved