2.7 pre-release: Virtual memory exploding and dar locks up

Hi,

I am very new to dar as I need/want to create archives which are more or 
less randomly accessible unlike tar.xz which I have been using until now.

One of my current tasks is to put quite large datasets into an archive 
format, thus I tried dar 2.6.2 on Debian buster but the compression 
performance is a bit lacking:

A test date set (1,540,323MByte according to du) is being archived into 
100GByte slices for the past ~24h and is currently working on the 11th 
slice[1].

 From previous tests with tar, usually the compression stage was 
responsible for throughput limitations, therefore, I turned to 2.7.0pre4 
yesterday for some evaluation. But with a command like [2] it reliably 
locks up after reaching a 256GByte virtual memory footprint.

While running, RSS memory usage stays quite (a few 100 MBytes at most) 
low but VSS explodes fast and then locks up, i.e. nothing happens 
anymore and needs to be killed:

ps aum
[...]
root     30370 24.9  0.1 268795548 338880 pts/3 -   07:37   0:54 
/opt/dar/bin/dar [options from [2]]
root         -  1.3    -      -     - -        SLl+ 07:37   0:02 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
root         -  0.0    -      -     - -        SLl+ 07:38   0:00 -
[...]

on disk 85MByte are written but that's all.

Is there anything at fault in my command or can I assist in fixing this 
bug - if this is one?

Cheers

Carsten

[1] dar --create test --verbose --fs-root /data/test-dataset 
--alter=binary --slice=100G --min-digits=4 --compression=gzip 
--slice-mode=0600 --hash=sha512

[2] /opt/dar/bin/dar --create test-parallel --verbose --fs-root= 
/data/test-dataset --alter=binary --slice=100G --min-digits=4 
--compression=zstd:20:100k -G 1,12 --slice-mode=0600 --hash=sha512
-- 
Dr. Carsten Aulbert, Max Planck Institute for Gravitational Physics,
Callinstraße 38, 30167 Hannover, Germany, Phone +49 511 762 17185

_______________________________________________
Dar-support mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dar-support