We have 20,000 page WP site which had W3 Total Cache web cache plugins installed by our developers. Our developers are not server savvy, by their own admission and have not done sites in excess of 100 pages. But I did not think this would be a big deal. Not long after the web cache the server crept to a halt ( 500 server error ). Not sure if it was connected but over 80,000 files were held in the cache folder, which our administrator ditched.
Being concerned, I asked our server administrators what to do, and this is what they said :
"There are alternatives to cache for wordpress. I was researching that this can actually work against the purpose. There are solutions such as varnish cache, which are installed at a server level (meaning we would need to install and configure this, versus the developer ) which can take the server load from maxing out with 15 users on wordpress, to allowing 1000s of users with no effect on performance. Part of the problem appears to be wordpress, it doesn't perform as well as it used to."
Our sites do operate IP look up's to display the language and currency of user - so not sure if any of these server side implementations will upset this.
A while back during a period of instability on an old server another administrator said that we should install NGIX and Varnish. he also asked at that stage that all plugins for forms and emails be disengaged, until we moved to a more robust server configuration. Which we now have.
I thought Wordpress can handle substantial scale in terms of traffic and pages.
Do any of the experts that frequent these threads have a view of how to manage a stable implementation with regards to Wordpress and speeding up larger sites, in simple language?