homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

We can put sitempp.html to robots.txt?
need some advise

10+ Year Member

Msg#: 3386959 posted 2:37 pm on Jul 5, 2007 (gmt 0)

We can make sitemap.xml put to robots.txt, but I don't know we can put other name or other format put ot robots.txt?

Thank you



WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 3386959 posted 12:00 pm on Jul 6, 2007 (gmt 0)

Robots.txt is a specific format that is explained here [robotstxt.org....] Some search engines have made exceptions and will allow robots.txt to do things that are not officially allowed.

One example of an unofficial part of robots.txt that will work with search engines is placing the url of the sitemap file in the robots.txt file. A few search engines will follow that url to the sitemap. You should not include the actual sitemap data, only include the sitemap url.

I hope that helps you. Please let me know if you have another question.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved