Yes, I am sure that the load time of ads has a very big impact on earnings. I do not know why it's getting that bad. I wonder if this is on purpose, showing ads "later" , might make them pop up at the eyes of visitors. Or Adsense's servers are too busy, dealing with too sophisticated algorithms. Or, they might be permanently DDos-ed (I guess that Google's servers are constantly facing attacks). Also, it's possible that Google tests/analysis medias on the fly before serving them to detect potential exploits or abuses.
Now, the thing is, for some reasons, my leaderboard banner which is on top of pages, ALWAYS load again more slower, than a 300x600 banner located lower in the page source code. I don't understand this. And by slower, I mean, the leaderboard, will show a graphic may be 2 or 3 seconds after the 300x600. May be inventory is bigger for leaderboard and top of page placement, so it takes longer to select one, I don't know.
By the way, to complete my first answer, in order to optimize a bit more things, you can detect yourself (with server side language, or on client side with javascript) if the page is being displayed on a mobile device or not. If it's on a mobile device, serve a more appropriate ad, like for example 300x100 text-only. My recommendation, always serve text only ads, and avoid 300x250 on mobile devices.
Addendum: by mobile devices, I mean smartphones. On tablets (which are also mobile devices in a way), also prefer text ads only, but you can use traditional desktop sizes.
