Hidden pages

[MarkG]

Interesting post at WPG forum. I tried looking for my pages on Hotbot and found 23 but by doing this there were 73. So are they in or are they out? see below

Date: February 24, 2000 02:33 PM
Author: Rick Wilkie ([e-mail])

I have been intentionally tracking a certain site every day and I have noticed that while the pages seem to dissappear they do not really. What I mean is when using wpg to find them with a unique phrase I cannot find them once they appear to drop. Nor can I search directly on hb to find them with that unique phrase. However if I search for the pages themselves they are still there. They jsut seem to have no connection in the database to the indexed content.

You can verify this for yourselves by searching for anything. Find a site that has more pages and click on more pages. The results will have a domain name field. Enter your domain and clear the search field and hit search. You will probably find all your pages for that site but for some reason they do not show up in searches. Anyone have any ideas.

(marketpositiontalk message)
(fixed long url -bt)

[Brett_Tabke]

This is interesting. I've always had a difficult time getting Hotbot to fess up just how many pages I have in the database. Seems like root level domains are ok with thier search form, but try anything that is down in a site and it doesn't work (ie: searchengineworld.com is found but searchengineworld.com/links/ isn't - both of which are legitimate uses.) Also, I sure can't get anything on an isp to be found, yet I know the pages are in there.