syncing huge files/devices: bmapfs

Heiko Schlittermann via rsync <[email protected]> Fri, 10 Feb 2023 07:56:51 +0100

Newsgroups	gmane.network.rsync.general
Organization	schlittermann -- internet & unix support
Message-ID	<[email protected]>

Hi,

recently I had to sync really huge files (VM images), represented
either as block devices (on one or both sides) or as regular files.

It *seemed* that Rsync didn't work well with block devices (the
--copy-devices didn't 'work for me, maybe I'm stupid or it's broken in
the Rsync that ships with Debian bullseye). And additionally I ran into
performance and resource issues using Rsync for huge files. 

While this all can be my fault, I started a small project "bmapfs": a
fuse driver that maps the huge file (or device) as a directory of files
representing blocks of the backing file (or device).

Using Rsync now on these smaller files I circumvented the above
mentioned issues (be they real or just a result of my stupidity).

Maybe you find bmapfs useful for other scenarios too. I'd be happy if
you provide me feedback of *any* kind.

https://gitea.schlittermann.de/hs12/bmapfs

    Best regards from Dresden/Germany
    Viele Grüße aus Dresden
    Heiko Schlittermann
--
 SCHLITTERMANN.de ---------------------------- internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --------------- key ID: F69376CE -

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

signature.asc (application/pgp-signature, 488 B)