Re: mount points are not extracted in sequential read mode

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Thanks for your feedback!

I could reproduce the problem, in fact it does not only concern
mounted directory excluded by -M option but any directory that is
excluded from the backup (-P, -], ...):

In other words -D option (store an empty directory rather than nothing
when a directory is excluded) does not let you see that directory in
sequential read mode, only in direct mode. You can list the archive
content in normal or sequential read mode you'll see the difference
there too.

I'm looking forward to fix that: it also affects 2.5.x so it will be
fixed there and ported to 2.6.x just afterward.

Regards,
Denis

On 21/05/2019 02:15, Mike wrote:
> Consider an archive created as follows: $ dar -c /mnt/a -R / -M -D
> 
> Assuming that / and /mnt are distinct mounted filesystems, mount
> points (e.g. /proc, /sys, etc.) will be saved correctly as empty
> directories due to -M and -D.
> 
> When extracting normally (i.e. random I/O), these mount points are 
> restored correctly as empty directories: $ dar -x /mnt/a
> 
> In sequential read mode, they are not extracted at all
> (unexpected): $ dar -x - -0 < /mnt/a.1.dar
> 
> I notice that the man page notes some limitations about sequential
> read mode but none of them seem related to this case.  I'm guessing
> this is a bug; i.e. either these empty directories should be
> extracted in sequential read mode just as they would be in "random
> I/O" mode, or at least a warning should be issued indicating that
> they've been skipped.
> 
> Sincerest apologies if I have overlooked something reasonably
> obvious.
> 
> I have attached a shell script to demonstrate.
> 
> usage:  dar-test.sh [noclean]
> 
> Please note the following: 1)  sudo is called to mount/unmount a
> 1MiB tmpfs filesystem 2a)  When arg 1 is noclean:  the script will
> echo "cleanup" commands instead of executing them. 2b)  Otherwise,
> the script will execute cleanup commands which includes !!!! rm -fr
> !!!!
> 
> To be on the safe side, I recommend running in a "scratch"
> environment.
> 
> Below is a transcript from running this script on my host: $
> ./dar-test.sh Warning: using insecure memory!
> 
> dar version 2.6.3, Copyright (C) 2002-2019 Denis Corbin Long
> options support         : YES
> 
> Using libdar 6.1.0 built with compilation time options: Libz
> compression (gzip)      : YES Libbz2 compression (bzip2)   : YES 
> Liblzo2 compression (lzo)    : YES Liblzma compression (xz)     :
> YES Strong encryption (libgcrypt): YES Public key ciphers (gpgme)
> : NO Extended Attributes support  : YES Large files support (> 2GB)
> : YES ext2fs NODUMP flag support   : YES Integer size used
> : 64 bits Thread safe support          : YES Furtive read mode
> support    : YES Linux ext2/3/4 FSA support   : YES Mac OS X HFS+
> FSA support    : NO Detected system/CPU endian   : little Posix
> fadvise support        : YES Large dir. speed optimi.     : YES 
> Timestamp read accuracy      : 1 microsecond Timestamp write
> accuracy     : 1 microsecond Restores dates of symlinks   : YES 
> Multiple threads (libthreads): NO Delta compression support    :
> NO Remote repository support    : NO
> 
> compiled the Apr  1 2019 with GNUC version 9.0.1 20190312 (Red Hat 
> 9.0.1-0.10) dar is part of the Disk ARchive suite (Release 2.6.3) 
> dar comes with ABSOLUTELY NO WARRANTY; for details type `dar -W'.
> This is free software, and you are welcome to redistribute it under
> certain conditions; type `dar -L | more' for details.
> 
> mkdir: created directory '/tmp/tmp.562G7984gj/fs-root' mkdir:
> created directory '/tmp/tmp.562G7984gj/extract' mkdir: created
> directory '/tmp/tmp.562G7984gj/extract-sequential' mount: none
> mounted on /tmp/tmp.562G7984gj/fs-root/z.
>>>>> BEGIN: dar --create /tmp/tmp.562G7984gj/a --fs-root
> /tmp/tmp.562G7984gj/fs-root -M -D -v Warning: using insecure
> memory! Creating low layer: Writing archive into a plain file
> object... Writing down the archive header... Adding a new layer on
> top: Caching layer for better performances... Adding a new layer on
> top: Escape layer to allow sequential reading... All layers have
> been created successfully Building the catalog object... Processing
> files for backup... Furtive read mode requires either root
> permission and FOWNER capability, falling back to normal filesystem
> read Adding file to archive: /tmp/tmp.562G7984gj/fs-root/a building
> delta signature with block size of 2048 bytes Saving Extended
> Attributes for /tmp/tmp.562G7984gj/fs-root/a Writing down archive
> contents... Closing the escape layer... Writing down the first
> archive terminator... Writing down archive trailer... Writing down
> the second archive terminator... Closing archive low layer... 
> Archive is closed.
> 
> 
> -------------------------------------------- 1 inode(s) saved 
> including 0 hard link(s) treated 0 inode(s) changed at the moment
> of the backup and could not be saved properly 0 byte(s) have been
> wasted in the archive to resave changing files 0 inode(s) with only
> metadata changed 0 inode(s) not saved (no inode/file change) 0
> inode(s) failed to be saved (filesystem error) 1 inode(s) ignored
> (excluded by filters) 0 inode(s) recorded as deleted from reference
> backup -------------------------------------------- Total number of
> inode(s) considered: 2 
> -------------------------------------------- EA saved for 1
> inode(s) FSA saved for 0 inode(s) 
> -------------------------------------------- Making room in memory
> (releasing memory used by archive of reference)... Final memory
> cleanup...
>>>>> END: dar --create /tmp/tmp.562G7984gj/a --fs-root
> /tmp/tmp.562G7984gj/fs-root -M -D -v
>>>>> BEGIN: dar --extract /tmp/tmp.562G7984gj/a -v
> Warning: using insecure memory! Opening archive a ... Opening the
> archive using the multi-slice abstraction layer... Reading the
> archive trailer... Opening construction layer... Considering
> cyphering layer... No cyphering layer opened Opening escape
> sequence abstraction layer... Opening the compression abstraction
> layer (compression algorithm used is none)... All layers have been
> created successfully Loading catalogue into memory... Locating
> archive contents... Reading archive contents... Restoring file's
> data: /tmp/tmp.562G7984gj/extract/z 
> /tmp/tmp.562G7984gj/extract/zCould not restore original file
> ownership: Operation not permitted Restoring file's data:
> /tmp/tmp.562G7984gj/extract/a Restoring file's EA:
> /tmp/tmp.562G7984gj/extract/a
> 
> 
> -------------------------------------------- 2 inode(s) restored 
> including 0 hard link(s) 0 inode(s) not restored (not saved in
> archive) 0 inode(s) not restored (overwriting policy decision) 0
> inode(s) ignored (excluded by filters) 0 inode(s) failed to restore
> (filesystem error) 0 inode(s) deleted 
> -------------------------------------------- Total number of
> inode(s) considered: 2 
> -------------------------------------------- EA restored for 1
> inode(s) FSA restored for 0 inode(s) 
> -------------------------------------------- Final memory
> cleanup...
>>>>> END: dar --extract /tmp/tmp.562G7984gj/a -v BEGIN: dar
>>>>> --extract - -0 -v
> Warning: using insecure memory! Opening archive - ... Opening
> standard input to read the archive... Reading the archive
> header... Considering cyphering layer... No cyphering layer opened 
> Opening escape sequence abstraction layer... Opening the
> compression abstraction layer (compression algorithm used is
> none)... All layers have been created successfully The catalogue
> will be filled while sequentially reading the archive, preparing
> the data structure... Restoring file's data:
> /tmp/tmp.562G7984gj/extract-sequential/a Restoring file's EA:
> /tmp/tmp.562G7984gj/extract-sequential/a
> 
> 
> -------------------------------------------- 1 inode(s) restored 
> including 0 hard link(s) 0 inode(s) not restored (not saved in
> archive) 0 inode(s) not restored (overwriting policy decision) 0
> inode(s) ignored (excluded by filters) 0 inode(s) failed to restore
> (filesystem error) 0 inode(s) deleted 
> -------------------------------------------- Total number of
> inode(s) considered: 1 
> -------------------------------------------- EA restored for 1
> inode(s) FSA restored for 0 inode(s) 
> -------------------------------------------- Final memory
> cleanup...
>>>>> END: dar --extract - -0 -v
> /tmp/tmp.562G7984gj/extract/: a  z
> 
> /tmp/tmp.562G7984gj/extract-sequential/: a umount:
> /tmp/tmp.562G7984gj/fs-root/z unmounted removed
> '/tmp/tmp.562G7984gj/a.1.dar' removed
> '/tmp/tmp.562G7984gj/extract-sequential/a' removed directory
> '/tmp/tmp.562G7984gj/extract-sequential' removed
> '/tmp/tmp.562G7984gj/extract/a' removed directory
> '/tmp/tmp.562G7984gj/extract/z' removed directory
> '/tmp/tmp.562G7984gj/extract' removed directory
> '/tmp/tmp.562G7984gj/fs-root/z' removed
> '/tmp/tmp.562G7984gj/fs-root/a' removed directory
> '/tmp/tmp.562G7984gj/fs-root' removed directory
> '/tmp/tmp.562G7984gj'
> 
> 
> 
> _______________________________________________ Dar-support mailing
> list [email protected] 
> https://lists.sourceforge.net/lists/listinfo/dar-support
> 
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEOzEprx3d76WjfYGPCDGwvQPYsYIFAlzkN3gACgkQCDGwvQPY
sYLSSg/+ITAGio4olEk/t69SnHP9kKVJVfiBlAxsTQPm9abFNgoP+fGZGldXMoIN
RwMjlsFL6FaPnfVdxBpaJ74w7NCpjNCCRPV9UwLjWbGEw5BT0VTZPqTyOwLbsR2X
7kWGbk9jcdJK0o7s6m0iTHY8A8oKaCT2A3HVJmRmQQuJ1CdDRjGp54fr7puGkalh
to7BIqN7XSmiD+woaQljzMT955UjBoD9sL8Vc18ZQZ/PkmkBtex1OuXd03Rrmjxh
3yNhbltSSME2TjEmRL9CU63PE7g7zxmzaaWIkrtawLNsHGgXj93taPcO6TiGttAi
L7eRhJGVX8gxmvfU5t/143YdsCtoD4JZUrlTqEu+nsR7R86E2x5ckvFH/XNeC2n9
yiClJU8+GJyf1ehtMutcw/jTIG6z//1BSIGkqdoKpoqMhMlwdtExFAso7jcyJhgu
6BMK91FPL0rQU0ip5awAFmQFkiZy4zIS/sjEQM16CJUiLIZQCCFWEla+gdXe+x3Y
FIW5AIMhAb0J0fJbaUfhgWyv/WPCgsMUiIKU8DDSAl3yInc6uLqL4XXqXOMY/v9k
VVKAiBpaQEbNo8aelmlhbTQO7clftZwhhgnCBZaDo6xHf1y6bpmFRmbQO2MvJm38
baRSVulgdfNC67a4fzh3w8xq3RTuLdVN0lM4WiJDtqGOztoWVh8=
=KNFm
-----END PGP SIGNATURE-----