Re: CPU eating tar process
Kees Meijs | Nefos <[email protected]>
| Newsgroups | gmane.comp.archivers.amanda.user |
|---|---|
| Message-ID | <[email protected]> |
Hi, Thanks as well! The source is spinning rust with SSD cache. The target is RAID-Z2 ZFS on spinning rust. And yes: way too much tiny files. To be continued... K. On 21-09-2021 17:53, C. Chan wrote: > Is the file storage media hard disk, and are the files mostly > small and IOPS/latency bound? If so, you may have to live with sloooow > backups. We eventually added NVMe SSDs to our ZFS array as special > devices > and sent files smaller than 128KB to NVMe, which along with the metadata, > helped speed up backups quite a bit. > > It may be worth trying star instead of tar, but I think it too is single > threaded. There's something called tarsplitter on github which is > multi-threaded if you don't mind experimenting: > > https://github.com/AQUAOSOTech/tarsplitter