cfel2000 - 4:57 pm on Mar 11, 2002 (gmt 0)

What do you mean by 'checking for pages which are no longer there'? A robot.txt file doesn't contain a list of pages to your site. It just tells the spider which pages you don't want crawled.