tangor - 11:08 am on May 21, 2011 (gmt 0) Whitelist in robots.txt first (ie. allow only what you want) then .htaccess all who disobey... much shorter list of do nots...
Whitelist in robots.txt first (ie. allow only what you want) then .htaccess all who disobey... much shorter list of do nots...
Page is a not externally linkable