Re: Hard system lock-ups when using encrypted swap and RAM is exhausted

--Sig_/.XCJlYJqF5QI/5Sz1Tc2Qnf
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Thu, 27 Nov 2025 08:59:13 +0100
Milan Broz <[email protected]> wrote:

> Hi,
>=20
> On 11/12/25 6:18 AM, Aaron Rainbolt wrote:
> > Not sure if this is a memory management issue, a LUKS issue, or
> > both, so I wrote both mailing lists. =20
>=20
> It is not a LUKS issue; cryptsetup/LUKS activates the encrypted
> device, so it is only the kernel/dm-crypt handling IOs.
>=20
> Adding cc to dm-devel as this would be another combination
> device-mapper and encrypted swap that could cause issues...
>=20
> However, could you please specify exactly your storage configuration?
>=20
>  From the subject, I expected you to have an encrypted swap, but it is
> not clear if there are other encrypted devices.
>
> Please paste at least lsblk, lsblk -f output, and also luksDump
> (or crypttab if it is not LUKS) for LUKS/dm-crypt configuration.

Mikulas seems to have reproduced the issue already, but just in case
it's still useful, here's the requested data, along with /etc/fstab.
(/dev/sda2 is a BIOS boot partition.)

-----

[sysmaint ~]% lsblk   =20
NAME   MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINTS
loop0    7:0    0    4G  0 loop =20
=E2=94=94=E2=94=80swap 254:0    0    4G  0 crypt [SWAP]
sda      8:0    0  100G  0 disk =20
=E2=94=9C=E2=94=80sda1   8:1    0  100M  0 part  /boot/efi
=E2=94=9C=E2=94=80sda2   8:2    0    1M  0 part =20
=E2=94=94=E2=94=80sda3   8:3    0 99.9G  0 part  /

-----

[sysmaint ~]% lsblk -f
NAME   FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUS=
E% MOUNTPOINTS
loop0  swap   1           0a06fb5a-39f5-4a89-b8a8-fdc2470ee0f3   =20
=E2=94=94=E2=94=80swap swap   1     swap  f9aa44bb-2214-4a32-ac7e-34360b0f3=
13b                [SWAP]
sda   =20
=E2=94=9C=E2=94=80sda1 vfat   FAT32 EFI   6907-5000                        =
                   /boot/efi
=E2=94=9C=E2=94=80sda2   =20
=E2=94=94=E2=94=80sda3 ext4   1.0         26ada0c0-1165-4098-884d-aafd2220c=
2c6   64.7G    29% /

-----

[sysmaint ~]% cat /etc/crypttab   =20
# <target name> <source device>         <key file>      <options>
swap /swapfile /dev/urandom swap,noearly

-----

[sysmaint ~]% cat /etc/fstab   =20
# /etc/fstab - created by grml-debootstrap on Sun Nov  2 20:23:37 UTC 2025
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/disk/by-uuid/26ada0c0-1165-4098-884d-aafd2220c2c6 /  auto    defaults,=
errors=3Dremount-ro,x-systemd.growfs 0   1  =20
UUID=3D6907-5000  /boot/efi       vfat    umask=3D0077      0       1  =20
proc           /proc        proc    defaults                      0   0  =20
/dev/cdrom     /mnt/cdrom0  iso9660 ro,user,noauto                0   0  =20
# some other examples:
# /dev/sda2       none         swap    sw,pri=3D0             0   0  =20
# /dev/hda1       /Grml        ext3    dev,suid,user,noauto 0  2
# //1.2.3.4/pub   /smb/pub     cifs    user,noauto,uid=3Dgrml,gid=3Dgrml 0 =
0=20
# linux:/pub      /beer        nfs     defaults             0  0
# tmpfs           /tmp         tmpfs   size=3D300M            0  0
# /dev/sda5       none         swap    sw                   0  0
/dev/mapper/swap none swap sw 0 0

--
Aaron

> Thanks,
> Milan
>=20
>=20
> >=20
> > I'm seeing an issue with both the latest mainline kernel (6.18-rc5)
> > and Debian 13's 6.12 kernel package. When physical memory fills up,
> > the entire system locks up hard, as if it hit rather severe
> > thrashing, despite the fact that there appears to be disk cache
> > that can still be evicted, and there is ample amounts of swap space
> > remaining (gigabytes of it). This issue did not occur with the 6.1
> > kernel in Debian 12. I'm seeing this occur in very low-memory
> > Debian VMs, with between 512 and 900 MB RAM, running under
> > VirtualBox and KVM. (I suspect, but have not verified, that I'm
> > seeing similar behavior under Xen as well.) These VMs generally use
> > a swappiness of 1, though I have seen a lockup occur even with a
> > swappiness of 60. The filesystem in use, in case it matters, is
> > ext4.
> >=20
> > To reproduce on a system running Linux 6.18-rc5, with :
> >=20
> > * Follow the steps from
> >    https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQues=
tions,
> >    section "2.3 How do I set up encrypted swap?", but creating a
> >    swapfile rather than a swap partition. I created an 8 GB swapfile
> >    with fallocate. Reboot the system when done.
> > * In a TTY, open a terminal multiplexer (or something you can abuse
> > as one, Vim works well), and open two terminals. In one terminal,
> > run `htop` so you can observe memory and swap usage.
> > * In the `htop` terminal, sort by M_RESIDENT.
> > * In the other terminal, create a new file `test.py`, that will
> >    gradually fill memory at a relatively fast pace and print an
> >    indicator that it's still alive. I used the following code for
> > this:
> >=20
> >      import time
> >=20
> >      count =3D 0
> >      mem_list =3D []
> >      while True:
> >          mem_list.append([x for x in range(2048)])
> >          count +=3D 1
> >          time.sleep(0.002)
> >          print(count)
> >=20
> > * Run the script with `python3 test.py`.
> > * While the script runs, observe the growing memory usage in `htop`.
> >    Swap usage should start at or near 0, RAM usage will gradually
> >    increase. Once RAM usage starts getting high, some data will
> > start being swapped out as expected, but after a short while the
> > whole VM will lock up despite there being gigabytes of swap left.
> > (On my KVM VM, the last time htop updated its screen, it showed RAM
> > usage of 712M/846M, and swap usage of 328M/7.40G. The python3
> > process running the script was consuming 551M memory. The VM is
> > entirely unresponsive. Incidentally, the python3 process also was in
> >    uninterruptible sleep when htop last updated its screen, but that
> >    could mean nothing since it might have come out of sleep between
> > the last screen update and the VM lockup.)
> >=20
> > Under Bookworm with Linux 6.1, the Python script would occasionally
> > freeze, but the VM would remain responsive, and the script would
> > eventually resume. Even with kernel 6.12, both unencrypted
> > swapfiles and swapfiles that are technically unencrypted but live
> > on a LUKS volume both behave as expected. It's only swapfiles that
> > are themselves encrypted that seem to trigger these lockups.
> >=20
> > I haven't looked at the code at all, but it seems like maybe memory
> > LUKS needs available in order to operate is being consumed, thus
> > making it impossible to swap anything in and out of the swapfile?
> > That seems like it would cause these symptoms or similar, though I
> > don't know.
> >=20
> > Let me know if I can provide any further information on the issue.
> > I'm happy to bisect the kernel if it will help.
> >=20
> > --
> > Aaron =20
>=20


--Sig_/.XCJlYJqF5QI/5Sz1Tc2Qnf
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEudh48PFXwyPDa0wGpwkWDXPHkQkFAmko4J4ACgkQpwkWDXPH
kQmMIhAAvF1ssmiCQ+nw1++89u8VMQnbSRCywdjHphhTMKbxhnnwEsNiH55N1w6b
5ssVxIvA2KrJ5+Ac3KVNr7h4o7t3hb5nEQL/MXWpfMISdyaBZR5n6GlG2Y/AIhJA
dzv3yEyCSXCYvH+TIhe8l+eBzrEUeVJg7vh88Tmm7XwMawBuhPx4UxA25POEzh07
rLwCy+o1AagTJJ7Ugvj+Ok9Oj5uZ8dMo68wWWDJuiLX5LXZ1Cvy2YGguco66xB6D
YaHKTVrsKAo6C57vXVfoiISXepgHD9+2SVd8jJdA5O0SFc907ITPMCqD68QQcdWi
jr2V1CKktd0HnzjuY2+IlOp1qVALUHmxhqULO6eNKaBDhSejwEamGeuWmBzUfv2f
STm6PTrWBjI93laXCkWW1cCQWvwRubAte/IorJfvoj2P4j/pk1MrMb+19FFGeRCb
rE1H12uJYP88PiZOxtdEdtnlBiR06pI4kOWVvjv6lHhlsTg9y3BUxyXPB8ydzNV8
wDqkPQPnFkQSQ1kHpNYomcmBw8izoTojjglpL3ifcpeuWQFzMKWJ6PFpA/236e/k
7WOpgp2FcF8ZuWaoGsq8n+QhTmac78P3LjYqBiEaq0KdaKRBppIkOTC6IJkVoe49
2PK+nPjCpq6HzTzw6hCrNGmPn++2ryJkiJKuIdor38SRRhuV4EA=
=snQJ
-----END PGP SIGNATURE-----

--Sig_/.XCJlYJqF5QI/5Sz1Tc2Qnf--