Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: bakedjake
That is huge. From LookSmart's Robots.txt file [looksmart.com]:
From a log file I just looked up: "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
So if you're correct, good old google is indeed ignoring robots.txt. Unless they have a special deal with LookSmart that we all don't know about.
From: the Robots Exclusion Standard [robotstxt.org]:
To allow all robots complete access: