#1 Google personalizes rankings so it is very hard to accurately measure your rankings. Tracking how much traffic you gain from Google is smarter. It avoids the inaccuracies caused by personalization and also avoids being tricked by a SEO company that gets you #1 ranking for a keyword with no traffic. Go check how much traffic Google is sending you. Pay attention to what specific landing pages on your site are getting the traffic, this will help you to identify under performing keywords.
#2 Speed is rarely the core reason for a drop in rankings. Of course you want a fast site because users like fast websites (especially mobile users) but Google rarely kills your rankings just because your site is a little slower than the other sites. Wordpress has many setting & plugins that can help you to improve the site performance.
#3 Content & Backlinks - These are two of the bigger influences on rankings. You haven't mentioned anything about them. What have you done to develop great content & backlinks? Duplicate content and spammy backlinks can kill a website's ranking & traffic very fast.