Key_Master - 12:05 am on Feb 11, 2003 (gmt 0)

I'm surprised that this hasn't been mentioned before but the biggest difference between the two bots is in the MIME types they accept.

Deepbot indexes: text/html text/plain application/pdf application/x-shockwave-flash application/vnd.ms-excel application/rtf application/msword application/vnd.ms-powerpoint application/postscript application/x-gzip application/octet-stream application/*

Freshbot indexes: text/html text/plain