Jimville - 3:15 am on Jan 13, 2006 (gmt 0)

Great question. Answer to part A: On my site, Y is indexing internal urls that were removed from the site over three years ago. There is a 301 to the correct urls. Y indexed them for a couple of months, but has reverted to the old ones. At y's request, I have brought it to their attention on their search blog and their site explorer feedback. Zip! Answer to second part: I'm not a tech guy, but it seems to me that if your spider can't follow a 301, maybe you should start from scratch.