Been working on this for a while and feel I'm almost there. If you didn't want any images on your site to be indexed by the search engines and you only wanted Google, Yahoo, MSN and Ask jeeves to visit your website, would the below be ok please ?
User-agent: *
Disallow: /images/
Disallow: /a_images/
Disallow: /cgi-bin/
User-agent: Googlebot-Image
Disallow: /
User-agent: Yahoo-MMCrawler
Disallow: /
User-agent: psbot
Disallow: /
User-agent: *
Disallow: /
User-agent: Googlebot
Disallow:
User-agent: Slurp
Disallow:
User-agent: msnbot
Disallow:
User-agent: Teoma
Disallow:
Any help appreciated.
Dexie
