martinibuster - 8:29 am on Sep 19, 2008 (gmt 0)

...you can see how hard coded directory pages containing millions of websites can't provide the information as well as a search engine.

Directories are not trying to provide information. That's not their mission or goal. Search engines answer queries. Directories do not. Directories are a catalog of notable sites about certain topics. Search engines are not.

Two different functions. Two different methods of discovery.

Now on to the first assumption of that post:

The nature of the web is such that a directory's usefulness will diminish in approximate correlation to the growth of the webs content."

Wikipedia is a catalog of the sum of human knowledge. More vast than indexing the Internet. Do you believe that as more content is added that it's usefulness is going to diminish? Wikipedia follows the same format as a directory. It's an organized collection of information ordered by topic. Change your statement a little and see how the truth comes out:

The nature of the web is such that wikipedia's usefulness will diminish in approximate correlation to the growth of human knowledge.

I can click around Wikipedia for hours discovering nuggets of knowledge. Something I cannot do on a search engine. Directories are the same way, they perform a vastly different function than search engines do and the size of the Internet is entirely irrelevant. Directories do not map the Internet. They catalog notable sites organized by topic.