| 12:01 pm on Jan 31, 2005 (gmt 0)|
Welcome to Webmasterworld.
The proper format is as follows
|The following allows all robots to visit all files because the wildcard "*" specifies all robots. |
This one keeps all robots out.
You want to place the robots.txt where yout html files are.
| 12:57 pm on Jan 31, 2005 (gmt 0)|
10x for this quick reply
| 4:32 pm on Jan 31, 2005 (gmt 0)|
One more question. Can I have multiple robots.txt files in my site - I mean in multiple directories (I have html files in different folders)? Would this affect the indexing?
| 4:45 pm on Jan 31, 2005 (gmt 0)|
No you dont need multiple robots.txt files, spiders will only request the robots.txt file in the root directory of your site.
| 5:15 pm on Jan 31, 2005 (gmt 0)|
The website is already designed and it's not possible to move the html files to the folder where robots.txt is. Is there a way to make the search engines index my site anyway?
| 5:40 pm on Jan 31, 2005 (gmt 0)|
jbinbpt coment was not quite correct, he should of said you should place the robots.txt file in the root directory of your website which is the same place as your index.html file, you can have html files in other directories.
A robots.txt file will not help to get your site indexed in a search engine, this file is only used to disallow robots not to allow, and thats only if the spider or robot what may visit is a friendly robot, if its not it will just ignore the file and spider your site anyway
See this link below on how to build a successful site
| 1:18 pm on Feb 8, 2005 (gmt 0)|
The problem with my site is that, it is finished now and the major index.html file is not in the root directory. I started my site with good google position 4-th place from 512000 sites. But after that i renew information on my site - meta tags, title etc...I use in one technique in my site- when i upload pictures in it, the script automatically give to the alt tags of the pictures good keywords. can it be the problem with indexing by Googlebot. When my index.html is not in root directory, how i must submit in the search engines-with the full path to the index.html or only to the root directory. Do you know someone who will help me. Even i can pay for supporting of my site and for increasing its rank popularity
10x in advance
ps. can you give me an E-mail of the google support centre to write to them, because it is really important for me and my work