jdMorgan - 9:35 pm on Jul 10, 2002 (gmt 0)

luma,

Your robots.txt looks OK to me... The only thing I see is that your
three Disallows for "Microsoft URL Control" are not likely to work -
those user-agents likely won't check robots.txt at all. You should
probably block these in .htaccess (for Apache server) instead.

Does Fast index any of the sites that link to your site? If so, they
should pick up your site quickly.

In your log files, what server code does your server return when Fast
requests robots.txt?

Have you recently moved your site?

Have you tried Fast's "Submit a Site" process?

Another "picky" thing about robots.txt is that is a Unix-format file;
Make sure you don't have carriage-return/linefeed pairs as the end-of-
line characters. Most robots.txt validators will catch this problem
though, so I assume this is not your problem. If so, and you're on a
PC, you can edit it in MS Word and use the "Save as" options, specifying
ASCII text, LF only.

So, good question - Anyone else?

Jim