[Question] UFS/FFS: why a block is always allocated at EOF when truncating up

Hi,

I've been working on some things related to SEEK_DATA and SEEK_HOLE in
Linux kernel(you'll find my work on linux-fsdevel). I still believe
there's future with POSIX so portability has always been of my
interests. They will probably hit the future revision of the C
programming language, so we might as well be preprared.

SEEK_DATA and SEEK_HOLE are rather recent addition to the spec, so it
seems that only the three "Unices"(Solaris, FreeBSD, Linux) implement
them at the moment. While observing the bevaviour of the API, I found
something strange in ffs: when a file gets "truncated up"(i.e. the new
file length becomes larger), a block at the new EOF is always allocated.

The culprit(sys/ufs/ffs/ffs_inode.c:360):

	/*
	 * Lengthen the size of the file. We must ensure that the
	 * last byte of the file is allocated. Since the smallest
	 * value of osize is 0, length will be at least 1.
	 */
	if (osize < length) {
		vnode_pager_setsize(vp, length);
		flags |= BA_CLRBUF;
		error = UFS_BALLOC(vp, length - 1, 1, cred, flags, &bp);
		if (error) {
			vnode_pager_setsize(vp, osize);
			return (error);
		}
		ip->i_size = length;
		DIP_SET(ip, i_size, length);
		if (bp->b_bufsize == fs->fs_bsize)
			bp->b_flags |= B_CLUSTEROK;
		ffs_inode_bwrite(vp, bp, flags);
		UFS_INODE_SET_FLAG(ip, IN_SIZEMOD | IN_CHANGE | IN_UPDATE);
		return (ffs_update(vp, waitforupdate));
	}


So, using the tool I made
<https://github.com/dxdxdt/gists/tree/master/writeups/lshole> to list
"holes" in files, say if a whole file(i.e. no wholes. empty or filled
with data) gets truncated upwards more than the block size of the file
system, you get this behaviour:

	$ gmake test-freebsd-ufs-phantom-data-at-eof
	truncate -s 0  phantom-data && [ -e phantom-data ]
	dd status=none conv=notrunc bs=1M if=/dev/urandom of=phantom-data count=1 seek=0
	truss ./lshole phantom-data 2>&1 | grep -E '^(lseek|phantom-data:)'
	lseek(3,0x0,SEEK_HOLE)                 = 1048576 (0x100000)
	lseek(3,0x100000,SEEK_DATA)             ERR#6 'Device not configured'
	lseek(3,0x0,SEEK_END)                 = 1048576 (0x100000)
	phantom-data:
	truncate -s 2M phantom-data
	truss ./lshole phantom-data 2>&1 | grep -E '^(lseek|phantom-data:)'
	lseek(3,0x0,SEEK_HOLE)                 = 1048576 (0x100000)
*lseek(3,0x100000,SEEK_DATA) = 2064384 (0x1f8000)*
	lseek(3,0x1f8000,SEEK_HOLE)             = 2097152 (0x200000)
	lseek(3,0x200000,SEEK_DATA)             ERR#6 'Device not configured'
	lseek(3,0x0,SEEK_END)                 = 2097152 (0x200000)
	phantom-data: 1048576-2064384
	dd status=none conv=notrunc bs=1M if=/dev/urandom of=phantom-data count=1 seek=2
	truss ./lshole phantom-data 2>&1 | grep -E '^(lseek|phantom-data:)'
	lseek(3,0x0,SEEK_HOLE)                 = 1048576 (0x100000)
	lseek(3,0x100000,SEEK_DATA)             = 2064384 (0x1f8000)
	lseek(3,0x1f8000,SEEK_HOLE)             = 3145728 (0x300000)
	lseek(3,0x300000,SEEK_DATA)             ERR#6 'Device not configured'
	lseek(3,0x0,SEEK_END)                 = 3145728 (0x300000)
	phantom-data: 1048576-2064384
	truncate -s 4M phantom-data
	truss ./lshole phantom-data 2>&1 | grep -E '^(lseek|phantom-data:)'
	lseek(3,0x0,SEEK_HOLE)                 = 1048576 (0x100000)
	lseek(3,0x100000,SEEK_DATA)             = 2064384 (0x1f8000)
	lseek(3,0x1f8000,SEEK_HOLE)             = 3145728 (0x300000)
*lseek(3,0x300000,SEEK_DATA) = 4161536 (0x3f8000)*
	lseek(3,0x3f8000,SEEK_HOLE)             = 4194304 (0x400000)
	lseek(3,0x400000,SEEK_DATA)             ERR#6 'Device not configured'
	lseek(3,0x0,SEEK_END)                 = 4194304 (0x400000)
	phantom-data: 1048576-2064384
	phantom-data: 3145728-4161536

While if you run the same script in ZFS, you get the layout you'd pretty
much expect:

	lseek(3,0x0,SEEK_HOLE)				 = 1048576 (0x100000)
	lseek(3,0x100000,SEEK_DATA)			 = 2097152 (0x200000)
	lseek(3,0x200000,SEEK_HOLE)			 = 3145728 (0x300000)
	lseek(3,0x300000,SEEK_DATA)			 ERR#6 'Device not configured'
	lseek(3,0x0,SEEK_END)				 = 4194304 (0x400000)
	phantom-data: 1048576-2097152
	phantom-data: 3145728-4194304


Yes, I've read the forum guidelines. This is not "why FreeBSD does X
while Y doesn't" type of question or complaint. I'm just curious about
the technical background that lead to this behaviour in UFS. I tried to
hunt it down myself, but the code predates VCS era(1994) and probably
way back to 386BSD era. It's probably documented somewhere, but I
decided asking would be much faster in this case.

My theories are:

 1. to serve the EOF block as "size markers" for fsck in case of power
    loss when inode write is deferred
 2. to reduce fragmentation on HDDs back in the days??

I think it's probably both although I don't have much understanding in
the design of UFS to explain why. So... any attempt to enlighten me
would be much appreciated!

Btw, FreeBSD's ext2 mirrors the exact same behaviour when the standard
implementation(Linux) doesn't. I wonder what benefit it could bring to
other extent-based filesystems like Ext? Or perhaps it was just one of
"monkey see, monkey do" thingies?

Davo