Re: [Question] UFS/FFS: why a block is always allocated at EOF when truncating up

On Tue, Mar 17, 2026 at 09:55:07AM +0000, Frank Leonhardt wrote:
> On 17/03/2026 04:04, David Timber wrote:
> > Hi,
> > 
> > I've been working on some things related to SEEK_DATA and SEEK_HOLE in
> > Linux kernel(you'll find my work on linux-fsdevel). I still believe
> > there's future with POSIX so portability has always been of my
> > interests. They will probably hit the future revision of the C
> > programming language, so we might as well be preprared.
> > 
> > SEEK_DATA and SEEK_HOLE are rather recent addition to the spec, so it
> > seems that only the three "Unices"(Solaris, FreeBSD, Linux) implement
> > them at the moment. While observing the bevaviour of the API, I found
> > something strange in ffs: when a file gets "truncated up"(i.e. the new
> > file length becomes larger), a block at the new EOF is always allocated.
> > 
> > The culprit(sys/ufs/ffs/ffs_inode.c:360):
> > 
> > 	/*
> > 	 * Lengthen the size of the file. We must ensure that the
> > 	 * last byte of the file is allocated. Since the smallest
> > 	 * value of osize is 0, length will be at least 1.
> > 	 */
> > 	if (osize < length) {
> > 		vnode_pager_setsize(vp, length);
> > 		flags |= BA_CLRBUF;
> > 		error = UFS_BALLOC(vp, length - 1, 1, cred, flags, &bp);
> > 		if (error) {
> > 			vnode_pager_setsize(vp, osize);
> > 			return (error);
> > 		}
> > 		ip->i_size = length;
> > 		DIP_SET(ip, i_size, length);
> > 		if (bp->b_bufsize == fs->fs_bsize)
> > 			bp->b_flags |= B_CLUSTEROK;
> > 		ffs_inode_bwrite(vp, bp, flags);
> > 		UFS_INODE_SET_FLAG(ip, IN_SIZEMOD | IN_CHANGE | IN_UPDATE);
> > 		return (ffs_update(vp, waitforupdate));
> > 	}
> > 
> > 
> > So, using the tool I made
> > <https://github.com/dxdxdt/gists/tree/master/writeups/lshole> to list
> > "holes" in files, say if a whole file(i.e. no wholes. empty or filled
> > with data) gets truncated upwards more than the block size of the file
> > system, you get this behaviour:
> > 
> > 	$ gmake test-freebsd-ufs-phantom-data-at-eof
> > 	truncate -s 0  phantom-data && [ -e phantom-data ]
> > 	dd status=none conv=notrunc bs=1M if=/dev/urandom of=phantom-data count=1 seek=0
> > 	truss ./lshole phantom-data 2>&1 | grep -E '^(lseek|phantom-data:)'
> > 	lseek(3,0x0,SEEK_HOLE)                 = 1048576 (0x100000)
> > 	lseek(3,0x100000,SEEK_DATA)             ERR#6 'Device not configured'
> > 	lseek(3,0x0,SEEK_END)                 = 1048576 (0x100000)
> > 	phantom-data:
> > 	truncate -s 2M phantom-data
> > 	truss ./lshole phantom-data 2>&1 | grep -E '^(lseek|phantom-data:)'
> > 	lseek(3,0x0,SEEK_HOLE)                 = 1048576 (0x100000)
> > *lseek(3,0x100000,SEEK_DATA) = 2064384 (0x1f8000)*
> > 	lseek(3,0x1f8000,SEEK_HOLE)             = 2097152 (0x200000)
> > 	lseek(3,0x200000,SEEK_DATA)             ERR#6 'Device not configured'
> > 	lseek(3,0x0,SEEK_END)                 = 2097152 (0x200000)
> > 	phantom-data: 1048576-2064384
> > 	dd status=none conv=notrunc bs=1M if=/dev/urandom of=phantom-data count=1 seek=2
> > 	truss ./lshole phantom-data 2>&1 | grep -E '^(lseek|phantom-data:)'
> > 	lseek(3,0x0,SEEK_HOLE)                 = 1048576 (0x100000)
> > 	lseek(3,0x100000,SEEK_DATA)             = 2064384 (0x1f8000)
> > 	lseek(3,0x1f8000,SEEK_HOLE)             = 3145728 (0x300000)
> > 	lseek(3,0x300000,SEEK_DATA)             ERR#6 'Device not configured'
> > 	lseek(3,0x0,SEEK_END)                 = 3145728 (0x300000)
> > 	phantom-data: 1048576-2064384
> > 	truncate -s 4M phantom-data
> > 	truss ./lshole phantom-data 2>&1 | grep -E '^(lseek|phantom-data:)'
> > 	lseek(3,0x0,SEEK_HOLE)                 = 1048576 (0x100000)
> > 	lseek(3,0x100000,SEEK_DATA)             = 2064384 (0x1f8000)
> > 	lseek(3,0x1f8000,SEEK_HOLE)             = 3145728 (0x300000)
> > *lseek(3,0x300000,SEEK_DATA) = 4161536 (0x3f8000)*
> > 	lseek(3,0x3f8000,SEEK_HOLE)             = 4194304 (0x400000)
> > 	lseek(3,0x400000,SEEK_DATA)             ERR#6 'Device not configured'
> > 	lseek(3,0x0,SEEK_END)                 = 4194304 (0x400000)
> > 	phantom-data: 1048576-2064384
> > 	phantom-data: 3145728-4161536
> > 
> > While if you run the same script in ZFS, you get the layout you'd pretty
> > much expect:
> > 
> > 	lseek(3,0x0,SEEK_HOLE)				 = 1048576 (0x100000)
> > 	lseek(3,0x100000,SEEK_DATA)			 = 2097152 (0x200000)
> > 	lseek(3,0x200000,SEEK_HOLE)			 = 3145728 (0x300000)
> > 	lseek(3,0x300000,SEEK_DATA)			 ERR#6 'Device not configured'
> > 	lseek(3,0x0,SEEK_END)				 = 4194304 (0x400000)
> > 	phantom-data: 1048576-2097152
> > 	phantom-data: 3145728-4194304
> > 
> > 
> > Yes, I've read the forum guidelines. This is not "why FreeBSD does X
> > while Y doesn't" type of question or complaint. I'm just curious about
> > the technical background that lead to this behaviour in UFS. I tried to
> > hunt it down myself, but the code predates VCS era(1994) and probably
> > way back to 386BSD era. It's probably documented somewhere, but I
> > decided asking would be much faster in this case.
> > 
> > My theories are:
> > 
> >   1. to serve the EOF block as "size markers" for fsck in case of power
> >      loss when inode write is deferred
> >   2. to reduce fragmentation on HDDs back in the days??
> > 
> > I think it's probably both although I don't have much understanding in
> > the design of UFS to explain why. So... any attempt to enlighten me
> > would be much appreciated!
> > 
> > Btw, FreeBSD's ext2 mirrors the exact same behaviour when the standard
> > implementation(Linux) doesn't. I wonder what benefit it could bring to
> > other extent-based filesystems like Ext? Or perhaps it was just one of
> > "monkey see, monkey do" thingies?
> > 
> > Davo
> 
> I think this is likely something to do with FFS allocating fragments of
> blocks. Unless the last block was allocated it won't be able to deduce the
> fragment size from the metadata. And as FFS used ordered metadata updates
> instead of journaling (and IME on block devices it was a nightmare) you
> really wanted to avoid the situation where the inode size was larger than
> the allocated blocks to give fsck a fighting chance. In other words, the
> block containing the last byte (whether written or not) must always be
> allocated because of fragment-based block allocation.
> 
> In think the ext2 implementation simply reused the FFS vnode/block
> allocation logic.
> 
> But I always blame fragments for stuff I don't fully understand. An expert
> should be along in a minute...

I also wonder why.  I was never able to formulate satisfactory answer for
myself, except that the current code in fsck assumes that.

I once asked Kirk about it, but I do not remember any different statement.