Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- SonicWALL Firewall Blocks Spiders
EquityMind - 11:42 pm on May 3, 2005 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/631.htm
This is major, I think in those instances that you are banging your head trying to figure out why you cant get crawled, this is another think to check off the list of 'stuff that could go wrong'
Brought to you by WebmasterWorld: http://www.webmasterworld.com