Re: free: regression due to a different calculation of Used memory

On Sat 03-04-21 16:26:58, Craig Small wrote:
> Hi Michal,
>   You probably need to follow through several changes to get to where we
> are now.  The fact that something else is not reporting the same thing as
> procps happens a lot because its both subjective and a lot of things
> (including procps) have and are wrong about how to calculate certain memory
> values.
> 
> I'd suggest reading the links at
> https://gitlab.com/procps-ng/procps/-/commit/602b9a02ce2a3ed443b0b79fca457826da44d654
> which discusses some of this issue.
> It has a link to the freelist thread where this is change was made.

Thanks for the link. I have read through the discussion and I have to
say I do agree with the reporter. It is good to see that the obviously
broken slab accounting thing got fixed. I haven't checked the version we
are using in our distribution but it likely has that fix as well. But
Slab part was merely something I've spotted in the referenced commit and
not something that is causing the main problem. That is indeed ths
swap backed page cache. More on that below.

> On Thu, 1 Apr 2021 at 18:30, Michal Hocko <[email protected]>
> wrote:
> 
> > From a semantic point of view it is quite hard to understand what the
> > value is actually trying to represent. I assume that reducing the page
> >
> This is actually the main problem. What does used mean?

Well, kernel doesn't expose counter like that. It does provide and
estimation of the usable memory which is likely what you want to
be expressed in its complement.

If there is no clear semantic behind the value then it likely shouldn't
try to be clever. I do understand that many people are confused by their
system consuming a lot of memory for caching. This is not something new
and I am afraid that hiding that fact from them is just going to
continue confusion elsewhere. As you can see by this bug report.
 
[...]
> The calculation is now (see
> https://gitlab.com/procps-ng/procps/-/blob/master/proc/sysinfo.c#L789 )
> 
> Memory Used = Memory Total - Memory Free - ( Cache + Slab Reclaimable ) -
> Main Buffers
> 
> The first two parts are easy enough, total - free = used
> 
> So the question is why are we, in effect, ignoring cache, reclaimable slab
> and buffers?
> 
> Cache includes tmpfs which is file backed not memory

No, this is not quite right. tmpfs/shmem is a general concept for
shared memory with swap as a backing storage. That can be MAP_SHARED | MAP_ANON
mapping, tmpfs mounted filesystem, memfd, System V shm  and many others.
It is accounted as Cached because internally it is accounted as a Page
cache, mostly for historical reasons. Something many of us in the MM
area consider a mistake but something we cannot really change because
that would be suprising for users who already know that is the case.
There have been several attempts to change.

> Reclaimable slab, well it's reclaimable so it's not used.
> Buffers, I'm not 100% sure why this is removed now but it made sense later.
> 
> If you follow the mail archive at
> https://www.freelists.org/post/procps/OmegaPhilxxxxxxxxxxxxx-Bug799716-free-considers-cached-to-include-SUnreclaim
> you can see how we got to these figures.
> 
> If you really want to go down the Memory Used rabbit hole, try
> https://github.com/brndnmtthws/conky/issues/130

Well, the problem is that there are users relying on free(1) and they
complain that some monitoring alerts are invalid because output of free
shows the system in a healthy state while it is not because MemAvailable
or other more precise metrics show a serious memory overutilization. We
can sure go and tell those users that Used: of the free(1) output is
incorrect but I believe it would be better to fix this in free instead.
One potential way to go ahead would be Total - MemAvailable. I would
still argue that Total - Free would be likely the only one which is
always correct but using Memavailable can give at least arguably
coherent view.
-- 
Michal Hocko
SUSE Labs