How Do You Allocate Ring-Fenced Memory In Linux?

[inbound]

I'm not a regular in the Linux forum, probably know just enough to be dangerous, so please be gentle if I ask dumb questions.

I've got a situation which requires the allocation of large amounts of memory for a C++ program which absolutely must not be tampered with by the OS, even if there is a shortage of memory elsewhere (which would be unlikely).

What I'm thinging of is something along the lines of having servers with 24GB of RAM and only allowing the OS to address the first 8GB for what it needs, with the remaining 16GB being reserved at start-up so that the C++ program can directly address that space in the knowledge that nothing else will use it. (the machines will be running a still-to-be-decided 64 bit version of Linux, if a flavour of Linux makes it easier to ring-fence memory we may pick that version simply for that feature alone).

Is this possible? Am I mad?

I should say that, depending on what data any given machine is holding, the actual data size may be larger or smaller but there would always be at least 8GB of memory for the OS - which we never get anywhere near using in normal usage. The allocated RAM holds huge lookup tables - some may extend to 128GB on a 144GB machine (these tables will be used for very random lookups and lots of them, up to hundreds of millions a second - hence the OS swapping out lookup tables would be very bad for performance).

[mcavic]

I don't think you can hide memory from the OS, because you probably need the OS's help to access it. But, you can allocate a specific amount up-front. And once you write to that memory (by initializing it with all zeros, for example) it can't be reclaimed by another process.

It could be swapped out, but that shouldn't happen along as the machine has enough memory. Unlike Windows, Linux uses all of the RAM before touching a significant amount of the swap space.

With the amount of RAM you're talking about, though, you could turn swapping off entirely.

[inbound]

Thanks, turning off swapping will be given thought - I gues I'll need to find out what happens if the OS then runs out of RAM in that case (I'm presuming it would simply not allow new processes to start - meaning it could get very nasty, I understand that many programs require memory to be allocated for individual tasks like HTTP connections in apache - although 8GB or more of system RAM would need a lot of small allocations to make it fall over; especially if the speed at which tasks are completed is very quick)

Does anyone know exactly what happens when you turn of swapping?

[lammert]

There is no need to turn of swapping and it can be dangerous in some situations because the OS has no escape route when temporary memory spikes occur, for example during scheduled backup tasks.

I have created many real-time embedded software running on Linux. You can mark currently allocated memory and in the future to be allocated memory of your processes as locked in RAM. The call which you should put somewhere in your code is:

mlockall( MCL_CURRENT ¦ MCL_FUTURE );

You should know that many flavors of Linux by default only allow a small amount of memory (only 32 kB on Centos 5 for example) to be locked in memory. You should increase this system limit to your needs. The current setting of this limit (and others) can be viewed with the command ulimit -a.

[mcavic]

mlockall( MCL_CURRENT ¦ MCL_FUTURE );

Thanks lammert, that seems to work as advertised.

[inbound]

Thank you lammert, that seems to be what we need. I will report back on how using the command works with huge allocations - I'll try 32GB on a 48GB server this week.

[lammert]

I have never used memory locking above 25MB, so it will be interesting to hear how it works with 32GB.