LynxSI - 3:40 pm on May 10, 2010 (gmt 0)
I would agree with this theory although I don't have the sample size you do.
People have incredibly low patience and tolerance for slow sites these days. If they can just click-click-click through pages they will keep searching for what they want. If every click is costing them 2+ extra seconds they will get bored if the page they are on is not directly relevant to their search.
I've definately found that faster page load speeds like the improvements you have done will contribute to a better page rank in google! (People happy = Google happy)