Forgot to mention that the loading time could be a negative factor (especially for a new visitor versus those already on the site) - check the Time to First Byte(TTFB)?
Time to first byte (TTFB) is the time it takes for a browser to start receiving information after it has start to make the request to the server. In some situations, as much as 75% of the time spent loading a page can be taken up by waiting for the first byte of data to arrive. You can test your time to first byte by using services like Byte Check or Load Impact.
* Have you cached your site effectively?
* Have you minimized the size of the images? there are free online tools to minimize image volume without losing quality.
Look for: JPEG optimizer or PNG optimizer.