site1 dir 1 site1 dir 2 site2 site1 dir 3 site3 ect.
Can I have a robots.txt for each site and should the site1 robots.txt disallow the other sites' directories? Or do I have to reorganize my directory structure?
volatilegx
10:11 pm on Apr 17, 2002 (gmt 0)
you need a separate robots.txt for each "home" directory. The robots.txt shouldn't reference the other sites at all.