Page is a not externally linkable
Dreamquick - 8:53 pm on Jun 29, 2003 (gmt 0)
As for how it differs to robots.txt well... Robots.txt is at a site level and simply acts as a signpost saying "please don't crawl these" and list of paths, however the robots meta-tag allows you to be more granular - for instance "index,nocrawl" would tell them "index this page but don't crawl any of the links that it uses" - something which can't be done with robots.txt. Oh and welcome to WebmasterWorld [webmasterworld.com] aroye! - Tony ...and if I do say so myself a fitting way to hit my 500th post.
For all the info on the tag have a look here;