msndude - 5:43 pm on Aug 30, 2006 (gmt 0)
Are we crawling your site or are we just indexing it? Robots.txt only tells us not to crawl the site. If we see links from other pages that point to your site we'll index it anyway. This happens because we can't tell the difference between a page we haven't discovered yet and a page that we weren't allowed to crawl.