Page is a not externally linkable
Receptional - 4:34 pm on Nov 29, 2007 (gmt 0)
They were never "formal" rules. They were pretty informal from the off. As I recall it was some long since gone engine that started the protocol and others informally adopted it. Let's see if anything gets beyond a press release.
Most of the existing commands - even the basic ones - seem to be fail way too often, so I'm not sure more will be better than the sitemaps protocol which seems to be stealing a lead over robots.txt. The formal rules allow a site to block indexing of individual Web pages, specific directories or the entire site, though some search engines have added their own commands