Has anyone experienced issues with implementing a Google Mini?
We have had numerous issues, I have even involved Microsoft since the issue came down to crawling a Windows server.
I would set the Mini to crawl my site and after about 6 minutes I would get a "Connection Reset by Peer during Fetch" error. After a few weeks of trying to troubleshoot the problem with Google, they discovered that changing the MTU of the appliance to 1500 from a default setting of 1400 solved the problem. The mini could now crawl. But if the Mini was rebooted then the setting would default. Google would then have to log back in and reset the MTU. After numerous packet captures it was determined that the Windows server was sending back a packet that was too large. After a few more weeks it was determined that the packet was appropriately sized it just had TCP options turned on which caused it to be 12 bytes larger than the MTU of the Google Mini. Google suggested that the Microsoft server be fixed (which it cannot, it's not a bug rather it's a new feature of Windows Server 2003 R2. And by new feature I do mean new feature and not a bug. The enhancement of sending the options increases performance in a Gigabit ethernet network. The mini being set to an MTU of 1400 will not even function on a Gigabit network. Which I have verified it will not when set to Gigabit, we just thought at the time it didn't like our Cisco gear, now I know it's because of the MTU.) or I use a proxy server for the Mini to crawl through.
Has anyone experienced similar problems? I'm ready at this point to find another vendor.