Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- SonicWALL Firewall Blocks Spiders


EliteWeb - 6:02 am on May 4, 2005 (gmt 0)


carguy84, private facility for the host. However even if you are trained inside and out, the robots.txt filter is right in the same spot as requesting /etc/passwd from the web or pulling a CMD along with thousands of other well known exploits, a robots.txt isnt quite an exploit.

Not sure of the OS version of it right now.


Thread source:: http://www.webmasterworld.com/robots_txt/631.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com