Page is a not externally linkable
-- New To Web Development
---- How Do Search Engine Robots Work?
shri - 7:44 am on Jan 9, 2007 (gmt 0)Thread source:: http://www.webmasterworld.com/new_web_development/3206921.htm
Would highly recommend downloading and letting an open source crawler like Nutch do a few crawls to understand how a basic search engine crawler works.
Brought to you by WebmasterWorld: http://www.webmasterworld.com