Just curious how many memcache users out there do anything special to save the memory state of memcache before doing a reboot, and restore it after?
In my particular use of memcache, we accumulate quite a cache over time. It grows and is kept current via dirty bits, etc. After a few weeks our cache can reach close to 2 GB in size. A reboot of our server really hits our performance since the cache essentially has to be rebuilt slowly (since too fast would take up too many resources and bog the server down for hours).
The pages we cache are extremely heavy database I/O pages, hence once we get them cached, it is a huge benefit, but we can't cache them all quickly or our server would slowly melt.
My plan was to create a simple program that saves the state of memcache before we do any server reboots. Either off in another database, or a flat file. Then when the server comes back up, have a second program take that data and re-populate the memcache RAM with it.
I figured if I worked with a flat file, I could bring the server back online and reload the cache without bogging down the database.
Anyone else have any experience in doing something like this? Pitfalls or words of advice?