homepage Welcome to WebmasterWorld Guest from 54.198.25.229
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Robot.txt
what is the use of Robot.txt
divyarenjudlk



 
Msg#: 4516756 posted 11:58 am on Nov 7, 2012 (gmt 0)

how can i submit robot.txt

 

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4516756 posted 12:13 pm on Nov 7, 2012 (gmt 0)

welcome to WebmasterWorld, divyarenjudlk!

put the robots.txt file in the document root directory for for the hostname in question.
all well-behaved bots will request the /robots.txt file before attempting to crawl any other urls.

jaysh4922



 
Msg#: 4516756 posted 12:52 pm on Nov 7, 2012 (gmt 0)

The robots.txt file is placed in your www or public directory.the correct location where to put this file is in www.yourdomainname.com/robots.txt. This file indicates how deep the spiders can crawl into your website.

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4516756 posted 7:35 pm on Nov 7, 2012 (gmt 0)

It's robots.txt not robot.txt

It will only be read if it is in the root folder of the website.

Upload it via FTP and you're done. You don't "submit" it anywhere.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved