Re: How to estimate the upper bound of the peak memory consumption of cryptsetup itself?

On 20/06/2022 02:19, Coiby Xu wrote:
> On Sat, Jun 18, 2022 at 05:12:56PM +0200, Milan Broz wrote:
>> On 16/06/2022 06:43, Coiby Xu wrote:
>>> Hi,
>>>
>>> Recently, I notice cryptsetup itself consumes significant amount of
>>> memory (~256M) when estimating the memory requirement for dumping vmcore
>>> to a LUKS-encrypted disk,
>>>
>>> $ time -v cryptsetup luksOpen encrypted.img volume --key-file mykey.keyfile | grep "Maximum resident set size"
>>>           Maximum resident set size (kbytes): 1309828
>>> $ cryptsetup luksDump encrypted.img
>>> ...
>>> Keyslots:
>>>     0: luks2
>>>           PBKDF:      argon2id
>>>           Memory:     1048576
>>>           ...
>>>
>>>
>>> So is there a way to estimate the upper bound of the peak memory
>>> consumption of cryptsetup itself without running cryptsetup?
>>
>> As you already found, the major memory consumption is by memory-hard KDF.
>> But this memory is used only while calculating keyslot encryption key,
>> it is released immediately after the Argon call is finished.
>> I do not think we have better estimation here.
> 
> Thanks for the reply! Sorry I meant the way to estimate the overhead of
> crypsetup itself i.e. ~256M in the above example. Previously I only take
> the memory consumption by memory-hard KDF into consideration and
> neglected the memory consumption of cryptsetup itself. This obviously
> leads to an underestimation of the memory requirement of cryptsetup. I
> need to overestimate the memory requirement a bit to make sure OOM won't
> happen that's why I am asking if there is a way to estimate the
> upper bound of memory requirement of cryptsetup itself.

There is no generic way to get a number - it depends on configuration
of the distro, libc, translations, everything that is locked including
shared libraries.

If it is about RHEL, you can perhaps know exact configuration - please
ask people in Red Hat.

>> (Another story is locking all memory, including big areas used by libc,
>> but that should not be problem here, I hope.)
> 
> Do you mean locking all memory first in order to know the memory
> requirement?

We use (mlockall(MCL_CURRENT | MCL_FUTURE) so it locks all used memory
+ all future allocated memory.

Today, it is not the best option and we will probably lock only specific
region with stored keys in the future.

Milan