support for Apple UFS in NetBSD

[ posted to [email protected], and Bcc'd to port-macppc and port-next68k.
  Please direct discussion to [email protected]. ]

I have submitted the pr kern/17345 which contains patches to support
Apple's UFS file system under NetBSD.  Their version of UFS is very similar
to our FFS with some minor changes, mostly inherited from NeXTstep.
The PR can be retrieved on the web at:
  <URL:http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=17345>

I would like to pursue a review and discussion in the [email protected]
email forum prior to possibly committing the patches contained in the PR.
What follows is a brief review of information in the PR, as well as some
insight into the decisions involved and a glimpse at alternate options.

Last December, I posted an analysis of the differences in the Apple UFS
file system to the [email protected] mailing list:
  <URL:http://mail-index.netbsd.org/port-macppc/2001/12/13/0012.html>
In addition to the changes discussed in that email, Apple added a
volume label at offset 7k, just before the super block, which the patches
optionally use to identify a file system as being Apple UFS.

The most significant difference is that NeXTstep changed the value
of DIRBLKSIZ from 512 to 1024, most likely for performance.  As long
as the fragment size is at least as large as DIRBLKSIZ, the requirement
that DIRBLKSIZ be an atomic transfer is maintained.  These patches
accommodate this change by replacing all uses of DIRBLKSIZ with a run
time dynamic value, determined by checking a UFS mount specific flag.
Although this results in significant diffs to the kernel sources,
it does maintain consistent and separate values for DIRBLKSIZ for
traditional NetBSD file systems and Apple UFS file systems.

A second difference is that they shift the cylinder group cluster
summary count array forward by 4 bytes.  An apple PR made public
by request to the Darwin project reveals that this is to eliminate
a perceived overlap with the free block bitmaps.  Although this appears
to be true at first glance, this is actually a change instigated by
a red-herring.  Since the count of available 0 length blocks is
undefined, this array is indexed by 1 and not zero.  Therefore,
the fact that the the zero position of this array overlaps is harmless.
Although this change never fixed an actual bug, it must be accommodated
in newfs and fsck_ffs in order to avoid fsck reporting the problem
every time the operating system is switched.

The third difference is that they force MAXSYMLINKLEN to 60.  This is
the maximum length of symlinks that can be stored directly in the inode.
Since this matches our value under normal operation, this difference
is minor.  However, NeXTstep implemented storing symlinks directly in
the inode while still using FS_42INODEFMT, which NetBSD and our ancestors
did not.  Therefore, in order to accommodate NeXTstep file systems, we
also force this value to 60 when apple UFS file system emulation is in
effect.  Since NeXTstep uses the older FS_42INODEFMT, NetBSD PR bin/15449
is also needed to fix our compatibility support for NeXTstep file systems.

Although these primary functional differences were first introduced in
NeXTstep, apple incorporated these changes in the Apple UFS used by
Darwin and MacOS X.  Unfortunately, there is no certain way to analyse
a UFS file system to determine whether it is Apple/NeXT or traditional
BSD since there was no additional tag or information provided to
differentiate or for backward compatibility.  However, we can use the
one additional change introduced by Apple to differentiate the
file systems when there is not a distinct disklabel fstype tag
available.

Apple introduced a volume label at offset 7k, just before the
super block.  This volume label is CRC protected, versioned, and
contains a volume name used by MacOS X, but not needed by NetBSD.  Our
patches to newfs allow creating this label, and our kernel and fsck
patches can check, validate and repair it.  We also use the existence
of this label as an additional hint that the file system is Apple UFS
flavor.  This is only used as a backup hint to avoid the situation where
the corruption of this label would cause additional corruption due to
misinterpretation of the file system.

These patches introduce a new disklabel fstype, FS_APPLEUFS, which is
the primary indicator that the file system is of type Apple UFS.  On
macppc, an A/UX partition type tagged in the apple partition table as
`APPLE_UNIX_SVR2' is still preferred as the `a' partition when
available.  An apple partition table type of `APPLE_UFS' is tagged as
FS_APPLEUFS so that the kernel automatically chooses the correct type
of ffs.  Similarly, Darwin's lead was followed to choose FS_APPLEUFS
from the DOS partition table tag.  Since Apple's UFS is always
big-endian on all platforms, the kernel configuration option FFS_EI is
needed for use on little endian platforms.

NeXTstep also introduced support for media with block size that is not
512 bytes, the traditional value for DEV_BSIZE.  Hard disks with
sector sizes of 1024 are commonly found on NeXT computers.  MacOS X
and Darwin continue this support, although actual media formatted this
way is far less common.  Since assumptions about DEV_BSIZE are
pervasive in our code base and there are several outstanding PRs and
attempts to fix this problem, these changes do not address media with
block sizes other than the default for NetBSD of 512 bytes.  I am,
through code review, familiar with the choices made by Apple and NeXT
regarding the DEV_BSIZE problem and am interested in guiding
compatibility should this ever get resolved in NetBSD.

While working on this project at the MIT SIPB, I had the fortunate
opportunity to discuss my approach with Ted Ts'o, who offered a
suggestion for an alternate implementation.  Rather than select the
DIRBLKSIZ for the entire filesystem, we could require only that if the
current directory size is a multiple of 1024, then after any
modification the new directory size is also a multiple of 1024.  We
would also require that no single directory entry ever cross a 512
byte boundary.  This would imply the possibility of having two empty
directory entries to pad out a directory listing, but it is unlikely
to bother any current implementations.  It would also mean that under
normal operation the NetBSD kernel would slowly replace 512 byte
directory entries with 1024 byte ones.  This option would require a
much smaller change to the kernel, but has a greater impact on systems
that have nothing to do with Apple UFS.  I am interested in hearing
opinions on whether this would be a preferred route for NetBSD to
take.

I am asking for feedback, and encourage review and testing of my
patch.  Since this is a substantial change to our file system code,
I would like some guidance before committing it to our tree.  Please
direct comments or questions to the [email protected] mailing list.

Thank you,
Darrin

Darrin B. Jewell <[email protected]>