Crawlers, Navigation Map, Internal Links,Anchor Text

[Impala]

Hi
I am trying to figure out how the Crawlers which index all the pages figure out the Hierachy of a web site. How do they know where the top node is, and how the links pan out. I guess they must first look for home.html or index.html. Then they see which links are referenced in the index page? OK maybe there are 10 links referenced. How do they know which are incomming (Parent links) and which are outgoing(child links). Someone told me that they also look at the navigation map and the cgi-bin...
What I am trying to sort out is for a given page, how do they know what the incomming link and the anchor text is?
I have a site which has 100 pages, and I thought it was well linked together, but somehow, my anchor text is not being picked up...
Any help would be appreciated..
thanks

[mack]

Hi Impala,

Generally speaking search engines work on a page by page bases. They will follow links from other sites to yours. If the you only have links pointing to your homepage then search engine bots will enter your site via the homepage.

When the bot arrives on your site it will then begin to spider and crawl the links within your site. The bot will follow links to your own internal pages and also to any external link that may be present on your site.

As for site hierarchy, I don't think search engines really take this into account. The bot will index any page that it can locate via links, and will show any page within the search results, based on best ranking.

The most common page to receive links will be your homepage and this is why the homepage will usually become indexed first.

Mack.