Forum Moderators: goodroi
[google.com...]
To verify your robots.txt is correct you can use Google Webmaster Central.
Thanks, but Google's own pages say to leave sub-directories /open without the traling backslash, no /closed/ as with everyone else.
Disallow: /foo
Matches
/foo/
/foo.html
Whereas:
Disallow: /foo[b]/[/b]
Matches the directory only.