Forum Moderators: goodroi
www.mydomain.com
My robots.txt contains:
User agent:*
Dissalow
I put this txt file in my main directory
But when i type the name of my site i acn see in browsersomething like this:
www.mydomain.com/en/.....
where i must put the robots.txt in main directory or in /en directory?
10x in advance
The proper format is as follows
The following allows all robots to visit all files because the wildcard "*" specifies all robots.User-agent: *
Disallow:This one keeps all robots out.
User-agent: *
Disallow: /
You want to place the robots.txt where yout html files are.
A robots.txt file will not help to get your site indexed in a search engine, this file is only used to disallow robots not to allow, and thats only if the spider or robot what may visit is a friendly robot, if its not it will just ignore the file and spider your site anyway
See this link below on how to build a successful site
[searchengineworld.com...]