enigma1 - 11:40 am on Apr 14, 2012 (gmt 0)
First you can't compare local connections with internet ones. Second did you identify where the latencies were coming from? Was it host connectivity issues, MySQL queries, badly written php scripts?
The specs of your current server look very good and the store's requirements low for that hardware. I was working with half of the h/w specs of what you currently have with twice as many products on a multi-store and the server response was under 2 secs, not well optimized in terms of webscripts and mysql.
In my opinion you should identify where the latencies are coming from first. If it's the cart, fix the scripts or change carts, if it's connectivity either ask the host to fix the problems or change hosts, I find it hard to blame the hardware for the given setup. Maybe a second hdd for the mysql db but that can come later.