- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- SonicWALL Firewall Blocks Spiders
EquityMind - 11:42 pm on May 3, 2005 (gmt 0)
This is major, I think in those instances that you are banging your head trying to figure out why you cant get crawled, this is another think to check off the list of 'stuff that could go wrong'
Thread source: http://www.webmasterworld.com/robots_txt/631.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com