It's not a fair game we play ... Who's site is higher quality: The one that needs the slowed down 3rd party software or the one built for speed serving 'essentially the same' information?
IMO the days of the little guy buying (finding) 'off the shelf' software and 'slapping' [term used loosely] a site together and ranking are on the decline ... I think if you want to play this game long-term, you should probably up the ante a bit, and that's probably going to require some expense, either through having a better system built or an investment of time (bunches of time) to build a better one yourself ... The preceding is my opinion only of course, so YMMV by using the concepts.
I haven't ever seen a Google 'be fair' to all sites and pages motto or statement anywhere ... And it really is fair ... If your site is significantly slower than someone else's it's not (imo) the same quality, so the slower site is scored 'fairly' by not ranking as high ... It's actually much more fair to the people who invest considerable time (or pay others to invest considerable time) in developing a top-notch system ... In fact, how is that at all unfair?