Re: What is difference between zfs and ufs?
Johan Helsingius <[email protected]>
| Newsgroups | gmane.os.freebsd.questions |
|---|---|
| Message-ID | <[email protected]> |
Was this generated by a LLM? Julf On 16/08/2026 3:19 pm, Edward Sanford Sutton, III wrote: > On 8/16/26 04:36, kirolos gerges wrote: >> Same as title , i dont really get difference between them > > ZFS is a feature loaded filesystem that has some overhead that comes > with it. The short answer is to look at the features of ZFS over > traditional filesystems and assume most are probably accurate. There are > some technical shortcomings of both that don't impact most users but can > be woth looking into. > > Some main points of ZFS vs UFS for consideration off the top of my head: > > ZFS integrates many common technologies into a collection and then > manages them by its own proprietary tools; replaces how some things > function like mount and fstab function (can fallback to the old ways for > those) and breaks how some tools work that have more basic expectations > (df reports wrong stats because of how ZFS pools data across datasets > creates the impression of partitions that change size as other datasets > consume space in the pool) and reimplemented its own way to copy its > different structure but became incompatible with other common tools (zfs > replication through `zfs send|zfs recv` instead of `dump|restore` which > added the limitation of `zfs recv` requires it be written to a future > ZFS filesystem while `restore` can write to any filesystem. > Though ZFS replication does not support writing its output to a > non-zfs filesystem, it has the advantage of being able to send > incremental transfers from one filesystem to another and only what ZFS > records have been updated will be transferred which is almost always > more efficient than using tools like cp, tar, and rsync to do the > transfer + maintains ZFS details properly. Compared to UFS dump, ZFS > snapshots offer a lot more flexibility + performance. An unfortunate > side effect is by default a `zfs recv` of a filesystem to a backup drive > will have unintended consequences like automatically mounting the backup > filesystem over the original filesystem thanks to ZFS integrating its > own management of mount and the transferred datasets containing records > of where and when to mount are by default the same as the source. This > can be eliminated by using options to temporarily or permanently write > the destination dataset properties differently with commandline > parameters but those changes need to be accounted for on a future > restore. Unfortunately these details aren't clearly brought up to the > users by the commands and documentation so when users use a root-on-ZFS > filesystem and try to send+recv to a live disk its causes users to make > unexpected mistakes which can go as far as to corrupt their setup if > they have bad timing and aren't aware of it. Once aware, it still takes > a string of options to likely do such transfer as desired so for safety > reasons the command should be stored in a script or copied from a file > on each run. > ZFS adds checksums to all data structure by default while UFS would > require it come from another source like a geom provider. ZFS has the > ability to recover the data if multiple copies exist when it becomes > aware that one is unreadable while other providers like geom won't > contain such thorough recover and test capability. ZFS uses a 'scrub' to > force analysis of all data to take place where UFS uses fsck to only > check for some types of filesystem issue. This comes with a tradeoff of > ZFS will know if anything changes on disk and by default will refuse to > return any corrupted data whereas UFS will return corrupted data if its > in a file or in something that is wrong but unknown like a messed up > file name or timestamp (passes fsck) but UFS will keep working when such > problems occur whereas ZFS will be faster to shut down. > ZFS combines multiple data sources (disks/partitions) into a pool and > that pool can assign them different specific meanings to create RAID > layouts. UFS requires separate abstract layers be added outside it like > geom mirrors to mirror partitions/disks. By ZFS managing things top to > bottom, it can determine things like a bit is corrupt on a disk and > recover a larger chunk of data from another disk to replace it when a > copy exists. A basic mirror or other geom/hardware implementation of > redundancy won't know data is corrupt because of the data alone and uses > a more drastic divider of "disk dropped off, do not use anymore until it > has been fully rebuilt". This makes recovering by reconnecting and > replacing a disk usually faster as only sectors that need to be written > will be written instead of the whole disk but as the disk becomes more > full and being fully written (=disk replacement) it likely means slower > due to not always being a start to finish write sequence. > ZFS works under the assumption that ZFS is not buggy and redundancy > will make data readable. A bug in ZFS could write bad data with a > matching checksum and that issue may not be detectable after it happens. > There is no fsck to go looking for such issues because its thought that > any data mistakes that happen will be exposed by checksums and any data > that is wrong is undesirable but too much unreadable data can cause ZFS > to stop reading the pool altogether which makes recovering anything > other than from a separate backup annoying at best. > ZFS's redundancy+checksum capabilities+snapshots make it easy for > users to fall into the trap of not having backups and not maintaining a > plan for how to replace+grow in the future. Redundancy is to maintain > uptime and snapshots give a ways to undo some issues but some user, > software, and hardware failures will require proper backups and future > replacement+upgrades need a plan to connect enough disks to do it. This > is not a ZFS only issue, but the features make people think they are > okay without making proper backups become even more common; backups may > be less likely to be needed but are not otherwise less important. > ZFS is a copy-on-write (COW) design. Any file being written uses 1 or > more "records" (default 128k max size) to write a file's contents. When > a part of a file is rewritten, COW writes the updated data to a new part > of the disk and then invalidates the reference to the old data by > pointing to the new data. This sequence avoids being able to have a > partial data write be referred to before it was completed for better > data integrity. It comes at the cost of write amplification for > rewriting + adding data (an entire record must be reread to rewrite only > a small part of it) and because a record is never immediately rewritten > it will always cause fragmentation of the file in recordsize increments. > Fragmentation of ZFS is further made worse by multiple records > storing metadata and its copies (some metadata is too important to not > make copies by default) that forms the chain that points to the file. > Overall, ZFS's on disk layout is bad from a performance perspective; its > focus was reliability and performance is not a focus. ZFS tries to hide > its bad performance in other ways like ZFS ARC (=adaptive replacement > cache=store things in RAM or on other faster media to hope to not have > to perform a slower read of it later). Once data is not available in ARC > (reboot, not enough RAM to hold it, ARC was flushed out by other process > needing RAM like Firefox or poudriere building rust and/or some other > big things), you have to go back to disk and if its magnetic then > related pains come back. Even reading a directory structure of a folder > containing many files+folders causes massive seeking if the structure > was written in many separate transactions and not written all in one ZFS > transaction (default=5 seconds) so if you use a magnetic disc and update > a ports tree repeatedly over time then those hundreds of thousands of > files metadata will read quickly when initially cloned and slow down > more and more as each update causes more and more fragmentation of even > its metadata on disk; a `find` can end up taking minutes to return from > fast magnetic media whereas it goes back to under a second once ARC > contains metadata. A 20gig ccache starts out simple enough but can take > tens of minutes to navigate and hours to fully read over time. > ZFS compression allows data to be compressed on disk; any compressed > data reduces disk I/O which can then be spent moving more data assuming > the system is not spending too much time compressing or decompressing > it. Being implemented by the filesystem means you get the benefits of > compression without programs and users being aware that > compression/decompression is happening. By default it uses lz4 > compression which is fast enough and with such low overhead that any > overhead is small enough that people just ignore it. For better > compression people switch it to zstd and you can further change what > level of compression is used by zstd but it starts adding more CPU+RAM > requirements as you push it further. There is a fast abort that occurs > with zstd's higher settings which lessens the overhead of writing data > that seems unlikely to compress enough to compress it to disk. ZFS > requires a file to compress by a certain amount before it will compress > it at all and compression happens for each ZFS record separately. If it > cannot be compressed enough then it is written without compression and > decompression will not be necessary to read it. ARC stores the > compressed data in memory which means ARC can store more data in the > same amount of RAM; extraction is written to a much smaller cache of RAM > so extraction overhead can impact performance of cached data. I'm not > aware of any good way to test latency but compression likely always > increases latency to access data. > More recently ZFS added a feature known as block cloning. When a file > gets copied from one place to another in the pool, block cloning will > cause pointers to the data to be created instead of copying the data. > Unlike symbolic links, each file will be its own and once one is updated > only that one file points to the updated data on disk. This can be > particularly beneficial for some workflows like an uncompressed ccache > which will copy the cached compiler output as a file to the work > directory; this saves I/O and disk space to write that output. > Unfortunately block cloning is not understood by zfs replication so a > `zfs send|zfs recv` will revert to cloned copies being expanded to being > no longer copies. > ZFS has deduplication which avoids writing the same ZFS records to > disk twice. There is a lot of RAM overhead for this feature to where it > is usually undesirable to turn it on fro most hardware+workloads. I > think it is called 'fast deduplication' that was trying to minimize the > overhead requirement but then only sometimes deduplicates data. For the > ccache example above, deduplication would also work to avoid writing the > data a second time. If using the ports tree to build software it is > usually turned into a package and that package is installed which makes > it more than just a 'copy file' step. Block cloning would not save > anything when a package is installed even if the data is the same as the > port's work directory on a ccache directory's contents but deduplication > will. > ZFS has snapshots which make it easy to mark a point in time for any > dataset in the pool to be able to refer back to and to rollback to > later. Creating snapshots are nearly instantaneous and almost no > additional space is used at that time. As ZFS records change in the > future it will cause the snapshot-referenced records to still exist so > deleted files don't yet recover space, and file updates take the > additional space of the new records. Removing snapshots is when > snapshots take time as the old records that are no longer referenced > need to finally be cleaned up. This causes disk space to become much > more confusing to keep track of. UFS has some snapshot capabilities but > it is very limited by comparison. > Though 'boot environments' exist on both ZFS an UFS, ZFS uses > snapshots whereas UFS uses additional copies of data and I think it > needed additional partitions for each copy so its worse on disk space > use and the associated I/O while only getting worse if trying to > maintain many boot environments. ZFS uses snapshots that are promoted to > clones to make boot environments work. ZFS clones have a disadvantage of > requiring the original snapshotted state to exist on disk even once both > the original + clone have replaced the underlying data. >