Page is a not externally linkable
wilderness - 7:33 pm on Mar 2, 2003 (gmt 0)
It is imperative that you keep in mind that robots.txt is a SUGGESTION to bots of compliance. With the above in mind. . . User-agent: * denies all bots. So your alyernatives are either to deny all or list each "1 by 1"
Javi,
Welcome to Webmaster World.
Many bad bots don't even read robots.txt.
Disallow: /
I seem to recall that Allow is not valid protocol in robots'txt.