In fact website optimization is a major factor ever since October 2010 when the local search algorithm was divided into two. Additionally in February 2012 Google put additional value on the website SEO, so now it is practically the most major factor in local search.
Other than that, the most important purely local search ranking factor (which you have influence on) is citations. Google reviews seem not to be a direct ranking factor, but they might be indirect, i.e. they could increase the CTR + the time spend on page, which could then increase the overall relevance of the listing.