Forum Moderators: goodroi

Message Too Old, No Replies

Does robots.txt work per directory or per domain?

         

zver0

5:45 pm on Feb 16, 2006 (gmt 0)

10+ Year Member



Does robots.txt work per directory like .htaccess, or only on robots.txt is allowed per domain/subdomain?

Little_G

5:55 pm on Feb 16, 2006 (gmt 0)

10+ Year Member



Hi,

Only one robots.txt is allowed in the root directory, others will be ignored.

Andrew

Lord Majestic

6:40 pm on Feb 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



One robots.txt per domain (sub-domains considered to be unique) per port (default 80) per protocol (ie HTTP/HTTS).

It is entirely possible that one physical location on a hard drive will result in the same robots.txt being shown in all these scenarios.