Re: OpenVZ containers - how can you manage /dev/shm sharing?

On 09/26/2018 06:39 PM, Rick Moen wrote:
> OOM is a rather blunt tool, killing processes which may or may not be
> the cause of the memory allocation problem.  OOM will only be
> triggered when most pages are allocated and cannot be spilled to disk.
> Since tmpfs has no backing store, all of the physical pages would be
> locked.

tmpfs uses swap as backing store, you're thinking ramfs.

> Over-commit is a problematic policy.  Then again, you currently have
> memory limit failures.

Virtual memory is inherently "problematic", yes. So is multitasking, as
meltdown/spectre/l1tf attest. Doesn't mean you shouldn't do it.

Overcommit is the default policy. Otherwise forking from a large process leads
to memory exhaustion for transient theoretical memory usage that's never
exercised (which is a common case when things like firefox spawn child processes).

Musl-libc uses vfork() under the covers in system() and popen() and such to
mitigate most of that, but last I checked glibc still didn't. Musl mostly did it
to have a thread-friendly fork/exec, but making disabling overcommit finally
feasible seems like a nice side effect.

(Sorry, I've seen multiple admins over the years go "I'll disable overcommit and
just give it dozens of gigabytes of swap I never expect it to use" and then the
first time it goes into a transient overload state it never comes _out_ and the
system is down for hours swap-thrashing because the OOM killer doesn't trigger
if you're still making progress at 0.01% of normal speed. These days that
failure mode has changed to "why do we burn out an SSD every 6 months" which you
live-migrate away from and is your hosting provider's problem, but it's still
bad advice leading to a bad outcome. Don't fear the overcommit. It exists for a
reason.)

> Are the host and containers sharing the same tmpfs file system?  This
> would sound problematic.  Most file systems do not support being
> accessed or modified by two different systems at the same time.  Even
> if one system is RW and the others are RO, you may get errors.  I
> would not mount a file system on both host and container unless the
> file system is designed for this.

No, it's pretty much why containers were invented. They let multiple instances
share the same underlying physical memory pages for filesystem mounts and memory
mapped files (executables) and so on. A container is really just a chroot with
more accounting and access controls layered on top.

Trying to do that in a VM fights against the _concept_ of the VM. (Intel made
its page table mechanism more complicated to try to make that suck less, and I'm
under the vague impression that's where l1tf came from?) But a container is
_not_ a vm, and that's why they're so popular.

I think you can just --bind mount the same instance into multiple containers and
then life is good? You do have to adjust OpenVZ's bean counter subsystem to give
it enough physical memory quota. Accounting for shared resources is always fun,
my experience with openvz is ~8 years out of date and I wasn't administering it.
But conceptually, containers were designed to allow what he's trying to do. I
know how to do it manually with "unshare" and "nsenter" and such, not sure how
to get openvz or docker to do it because I dunno their configuration stuff.

Rob