Re: What is difference between zfs and ufs?
"Edward Sanford Sutton, III" <[email protected]>
| Newsgroups | gmane.os.freebsd.questions |
|---|---|
| Message-ID | <BL4PR11MB882218A12B6E3C5F18A01454E6D82@BL4PR11MB8822.namprd11.prod.outlook.com> |
On 8/16/26 04:36, kirolos gerges wrote: > Same as title , i dont really get difference between them ZFS is a feature loaded filesystem that has some overhead that comes with it. The short answer is to look at the features of ZFS over traditional filesystems and assume most are probably accurate. There are some technical shortcomings of both that don't impact most users but can be woth looking into. Some main points of ZFS vs UFS for consideration off the top of my head: ZFS integrates many common technologies into a collection and then manages them by its own proprietary tools; replaces how some things function like mount and fstab function (can fallback to the old ways for those) and breaks how some tools work that have more basic expectations (df reports wrong stats because of how ZFS pools data across datasets creates the impression of partitions that change size as other datasets consume space in the pool) and reimplemented its own way to copy its different structure but became incompatible with other common tools (zfs replication through `zfs send|zfs recv` instead of `dump|restore` which added the limitation of `zfs recv` requires it be written to a future ZFS filesystem while `restore` can write to any filesystem. Though ZFS replication does not support writing its output to a non-zfs filesystem, it has the advantage of being able to send incremental transfers from one filesystem to another and only what ZFS records have been updated will be transferred which is almost always more efficient than using tools like cp, tar, and rsync to do the transfer + maintains ZFS details properly. Compared to UFS dump, ZFS snapshots offer a lot more flexibility + performance. An unfortunate side effect is by default a `zfs recv` of a filesystem to a backup drive will have unintended consequences like automatically mounting the backup filesystem over the original filesystem thanks to ZFS integrating its own management of mount and the transferred datasets containing records of where and when to mount are by default the same as the source. This can be eliminated by using options to temporarily or permanently write the destination dataset properties differently with commandline parameters but those changes need to be accounted for on a future restore. Unfortunately these details aren't clearly brought up to the users by the commands and documentation so when users use a root-on-ZFS filesystem and try to send+recv to a live disk its causes users to make unexpected mistakes which can go as far as to corrupt their setup if they have bad timing and aren't aware of it. Once aware, it still takes a string of options to likely do such transfer as desired so for safety reasons the command should be stored in a script or copied from a file on each run. ZFS adds checksums to all data structure by default while UFS would require it come from another source like a geom provider. ZFS has the ability to recover the data if multiple copies exist when it becomes aware that one is unreadable while other providers like geom won't contain such thorough recover and test capability. ZFS uses a 'scrub' to force analysis of all data to take place where UFS uses fsck to only check for some types of filesystem issue. This comes with a tradeoff of ZFS will know if anything changes on disk and by default will refuse to return any corrupted data whereas UFS will return corrupted data if its in a file or in something that is wrong but unknown like a messed up file name or timestamp (passes fsck) but UFS will keep working when such problems occur whereas ZFS will be faster to shut down. ZFS combines multiple data sources (disks/partitions) into a pool and that pool can assign them different specific meanings to create RAID layouts. UFS requires separate abstract layers be added outside it like geom mirrors to mirror partitions/disks. By ZFS managing things top to bottom, it can determine things like a bit is corrupt on a disk and recover a larger chunk of data from another disk to replace it when a copy exists. A basic mirror or other geom/hardware implementation of redundancy won't know data is corrupt because of the data alone and uses a more drastic divider of "disk dropped off, do not use anymore until it has been fully rebuilt". This makes recovering by reconnecting and replacing a disk usually faster as only sectors that need to be written will be written instead of the whole disk but as the disk becomes more full and being fully written (=disk replacement) it likely means slower due to not always being a start to finish write sequence. ZFS works under the assumption that ZFS is not buggy and redundancy will make data readable. A bug in ZFS could write bad data with a matching checksum and that issue may not be detectable after it happens. There is no fsck to go looking for such issues because its thought that any data mistakes that happen will be exposed by checksums and any data that is wrong is undesirable but too much unreadable data can cause ZFS to stop reading the pool altogether which makes recovering anything other than from a separate backup annoying at best. ZFS's redundancy+checksum capabilities+snapshots make it easy for users to fall into the trap of not having backups and not maintaining a plan for how to replace+grow in the future. Redundancy is to maintain uptime and snapshots give a ways to undo some issues but some user, software, and hardware failures will require proper backups and future replacement+upgrades need a plan to connect enough disks to do it. This is not a ZFS only issue, but the features make people think they are okay without making proper backups become even more common; backups may be less likely to be needed but are not otherwise less important. ZFS is a copy-on-write (COW) design. Any file being written uses 1 or more "records" (default 128k max size) to write a file's contents. When a part of a file is rewritten, COW writes the updated data to a new part of the disk and then invalidates the reference to the old data by pointing to the new data. This sequence avoids being able to have a partial data write be referred to before it was completed for better data integrity. It comes at the cost of write amplification for rewriting + adding data (an entire record must be reread to rewrite only a small part of it) and because a record is never immediately rewritten it will always cause fragmentation of the file in recordsize increments. Fragmentation of ZFS is further made worse by multiple records storing metadata and its copies (some metadata is too important to not make copies by default) that forms the chain that points to the file. Overall, ZFS's on disk layout is bad from a performance perspective; its focus was reliability and performance is not a focus. ZFS tries to hide its bad performance in other ways like ZFS ARC (=adaptive replacement cache=store things in RAM or on other faster media to hope to not have to perform a slower read of it later). Once data is not available in ARC (reboot, not enough RAM to hold it, ARC was flushed out by other process needing RAM like Firefox or poudriere building rust and/or some other big things), you have to go back to disk and if its magnetic then related pains come back. Even reading a directory structure of a folder containing many files+folders causes massive seeking if the structure was written in many separate transactions and not written all in one ZFS transaction (default=5 seconds) so if you use a magnetic disc and update a ports tree repeatedly over time then those hundreds of thousands of files metadata will read quickly when initially cloned and slow down more and more as each update causes more and more fragmentation of even its metadata on disk; a `find` can end up taking minutes to return from fast magnetic media whereas it goes back to under a second once ARC contains metadata. A 20gig ccache starts out simple enough but can take tens of minutes to navigate and hours to fully read over time. ZFS compression allows data to be compressed on disk; any compressed data reduces disk I/O which can then be spent moving more data assuming the system is not spending too much time compressing or decompressing it. Being implemented by the filesystem means you get the benefits of compression without programs and users being aware that compression/decompression is happening. By default it uses lz4 compression which is fast enough and with such low overhead that any overhead is small enough that people just ignore it. For better compression people switch it to zstd and you can further change what level of compression is used by zstd but it starts adding more CPU+RAM requirements as you push it further. There is a fast abort that occurs with zstd's higher settings which lessens the overhead of writing data that seems unlikely to compress enough to compress it to disk. ZFS requires a file to compress by a certain amount before it will compress it at all and compression happens for each ZFS record separately. If it cannot be compressed enough then it is written without compression and decompression will not be necessary to read it. ARC stores the compressed data in memory which means ARC can store more data in the same amount of RAM; extraction is written to a much smaller cache of RAM so extraction overhead can impact performance of cached data. I'm not aware of any good way to test latency but compression likely always increases latency to access data. More recently ZFS added a feature known as block cloning. When a file gets copied from one place to another in the pool, block cloning will cause pointers to the data to be created instead of copying the data. Unlike symbolic links, each file will be its own and once one is updated only that one file points to the updated data on disk. This can be particularly beneficial for some workflows like an uncompressed ccache which will copy the cached compiler output as a file to the work directory; this saves I/O and disk space to write that output. Unfortunately block cloning is not understood by zfs replication so a `zfs send|zfs recv` will revert to cloned copies being expanded to being no longer copies. ZFS has deduplication which avoids writing the same ZFS records to disk twice. There is a lot of RAM overhead for this feature to where it is usually undesirable to turn it on fro most hardware+workloads. I think it is called 'fast deduplication' that was trying to minimize the overhead requirement but then only sometimes deduplicates data. For the ccache example above, deduplication would also work to avoid writing the data a second time. If using the ports tree to build software it is usually turned into a package and that package is installed which makes it more than just a 'copy file' step. Block cloning would not save anything when a package is installed even if the data is the same as the port's work directory on a ccache directory's contents but deduplication will. ZFS has snapshots which make it easy to mark a point in time for any dataset in the pool to be able to refer back to and to rollback to later. Creating snapshots are nearly instantaneous and almost no additional space is used at that time. As ZFS records change in the future it will cause the snapshot-referenced records to still exist so deleted files don't yet recover space, and file updates take the additional space of the new records. Removing snapshots is when snapshots take time as the old records that are no longer referenced need to finally be cleaned up. This causes disk space to become much more confusing to keep track of. UFS has some snapshot capabilities but it is very limited by comparison. Though 'boot environments' exist on both ZFS an UFS, ZFS uses snapshots whereas UFS uses additional copies of data and I think it needed additional partitions for each copy so its worse on disk space use and the associated I/O while only getting worse if trying to maintain many boot environments. ZFS uses snapshots that are promoted to clones to make boot environments work. ZFS clones have a disadvantage of requiring the original snapshotted state to exist on disk even once both the original + clone have replaced the underlying data.