Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- SonicWALL Firewall Blocks Spiders


mrMister - 8:43 am on May 4, 2005 (gmt 0)


If this report is true, Sonicwall should be ashamed for such a blunder.

It's not a blunder. Far from it, it's just part of good practice.

The device is a firewall. It is meant to be the first line of defence when protecting a network.

This ain't a Windows workstation we're talking about...

These things come locked down! They're supposed to be like that so that users don't unwittingly leave open a big security hole.

When you buy a firewall, it's locked down so nothing can get through. Nothing. No traffic at all.

It's then the network administrator's responsibility to open up the device to let through the traffic that is needed for the network to operate.

When you install a firewall nothing should work, that's the point! In this case, the network operator opened up the ports to allow normal web traffic, however he/she didn't let through the spider traffic. They didn't read the manual.

I think it's an open and shut case as to who made the blunder:

The firewall's job is to prevent access to a network.

The Network administrator's job is to keep a network running smoothly.


Thread source:: http://www.webmasterworld.com/robots_txt/631.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com