phranque
msg:4516761 | 12:13 pm on Nov 7, 2012 (gmt 0) |
welcome to WebmasterWorld, divyarenjudlk!
put the robots.txt file in the document root directory for for the hostname in question.
all well-behaved bots will request the /robots.txt file before attempting to crawl any other urls.
|
jaysh4922
msg:4516766 | 12:52 pm on Nov 7, 2012 (gmt 0) |
The robots.txt file is placed in your www or public directory.the correct location where to put this file is in www.yourdomainname.com/robots.txt. This file indicates how deep the spiders can crawl into your website.
|
g1smd
msg:4516862 | 7:35 pm on Nov 7, 2012 (gmt 0) |
It's robots.txt not robot.txt
It will only be read if it is in the root folder of the website.
Upload it via FTP and you're done. You don't "submit" it anywhere.
|
|
