Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- One sitemap can cover multiple sites - kinda cool.


goodroi - 11:54 am on Jul 6, 2010 (gmt 0)


it is usually best to keep things as simple as possible to have maximum search engine crawling. the bots are getting smarter every day but there are still many, many issues that can cause your website not to be indexed.

its not always possible to avoid the more fancy techniques but simple does equal less headaches.


Thread source:: http://www.webmasterworld.com/robots_txt/4164100.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com