Welcome to WebmasterWorld Guest from 18.206.194.83

Forum Moderators: goodroi

Message Too Old, No Replies

Robot.txt

what is the use of Robot.txt

     
11:58 am on Nov 7, 2012 (gmt 0)

New User

5+ Year Member

joined:Nov 7, 2012
posts: 3
votes: 0


how can i submit robot.txt
12:13 pm on Nov 7, 2012 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11822
votes: 236


welcome to WebmasterWorld, divyarenjudlk!

put the robots.txt file in the document root directory for for the hostname in question.
all well-behaved bots will request the /robots.txt file before attempting to crawl any other urls.
12:52 pm on Nov 7, 2012 (gmt 0)

New User

joined:Oct 19, 2012
posts:4
votes: 0


The robots.txt file is placed in your www or public directory.the correct location where to put this file is in www.yourdomainname.com/robots.txt. This file indicates how deep the spiders can crawl into your website.
7:35 pm on Nov 7, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


It's robots.txt not robot.txt

It will only be read if it is in the root folder of the website.

Upload it via FTP and you're done. You don't "submit" it anywhere.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members