and look at the part that says "use this syntax" you will see how to do it.
Dijkgraaf
9:55 pm on Jun 1, 2005 (gmt 0)
Yahoo has two bots "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)" "Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...] Of course it would be nice if the URL in the bot actually worked, the correct URL is [help.yahoo.com...] (Note the trailing slash is required).
Pat1975
5:40 am on Jun 3, 2005 (gmt 0)
Thanks to all.
jdMorgan
5:55 am on Jun 3, 2005 (gmt 0)
MSN also has two crawlers, msnbot, and MSNPTC (their Preview technology crawler, used to generate the thumbnail screenshot "Page previews" visible if you use MSN search as the default IE search provider.)
However, MSNPTC apparently also obeys robots.txt records addressed to "msnbot", and so does not require its own robots.txt record.