You can have te fastest server in the world and your pages might be optimized. But, google still has to test your page loading speed. From there the ago could be slapping some sort of calculations of daily visitors your site can handle. Thus the throttle occurs. If google serves up your visitors quicker than normal and is a bit behind in updating it's ago, that is where you might see your site top out traffic wise before the end of the day. There are a lot of calculations at play there, butt I can see the possibility.
Someone who has this issue should upgrade their server and see what happens.
Also, google could be shutting you down at certain times of the day due to communication hubs and general web traffic. Some hubs could top out at certain points of the day from google to your server thus slowing it down significantly enough for google to deem your site as slow and bam, your traffic goes to a faster site.
There can be a lot of factors here, but I suspect it has to do with page loading speeds at different times of the day.