Page is a not externally linkable
Xoc - 8:06 pm on Mar 4, 2001 (gmt 0)
I think you misread how the a robot is supposed to parse the LookSmart robots.txt (or I misread what you said). The Disallow doesn't have anything after it. So that means that that particular spider is allowed to spider the web site. All non-identified spiders that weren't specifically listed get hit by the Disallow: * So Googlebot should be allowed to spider LookSmart. The Snap robots.txt, though, disallows everything.
Brett-