It is straight forward to look at what Google is measuring using Firefox, with the Firebug AddOn. Using Firebug alone look at the "Net" tab analysis and then scrolling to the bottom of the results will show you the time it takes for the "onload" event to fire. This event may occur long after your page has appeared to render. This is the time Google is using to measure your page's performance.
For example if you use Google Analytics the "onload" event will not fire until Google Analytics is done recording it's data. Visible rendering of your webpage might appear to be complete in say 2 seconds, BUT, Google Analytics may not complete for another 4 or 5 seconds, THEN, the onload event "might" fire for a total time of 6 or 7 seconds. I don't know how many times and how many sites I've seen the status bar say "Waiting for Google Analytics" (or something like that).
What it comes down to is probably one single factor is slowing your pageload drastically and you haven't found that factor yet. In many, many cases it's Adsense ads!