jdMorgan - 8:05 pm on Sep 26, 2003 (gmt 0)

Prana,

If those page names don't start with something already in your list, then yes, they'd be spiderable. Robots.txt uses prefix matching. For example, the second of these two lines is redundant:

Disallow: /JAVA
Disallow: /JAVASCRIPT

The first line blocks anything the second line also might block. You may find other opportuinties to make use of this prefix-matching to reduce the size of your robots.txt file.

Note that you should have a space after the ":"

As hinted at several times above, you need to read and fully understand the robots.txt standard [robotstxt.org]. We prefer to keep the discussion here general -- and useful to more than one person. Because of that, we prefer not to have "fix my site" threads.

Once you've got your file sorted out, validate it here [searchengineworld.com]

After you're comfortable with the basics, you might find this thread [webmasterworld.com] interesting.

Jim