ZFS: device permission checks (review help wanted!)

Hi all,

I've just posted this OpenZFS PR, but I need someone with enough
experience with GEOM and privileges/MAC to check some of my assumptions.

    https://github.com/openzfs/zfs/pull/18978

The short version of it is that if you have privilege to admin a pool,
you implicitly have access to all GEOM providers, because GEOM itself
has no access control of its own, and ZFS just calls straight into it.

Pool admin privilege is PRIV_ZFS_POOL_CONFIG, but most understanding of
device access is about whether or not you can access /dev/foo, which is
controlled by normal file permissions, or DAC override privs like
PRIV_ZFS_READ/PRIV_ZFS_WRITE.

So currently, if you had a user with PRIV_ZFS_POOL_CONFIG, that didn't
also have access to /dev/foo, they would still be able to do:

- zpool create tank /dev/foo
- zpool attach tank mirror-0 /dev/foo
- zpool add tank spare /dev/foo
- zpool import -c zpool.cache tank # zpool.cache references /dev/foo

The PR fixes this by doing an access check against /dev/foo with the
calling user's permissions before attempting to attach the device.

Linux has effectively the same privilege split for pool admin vs device
access, and I recently made the same change there[1]. That was much
easier to confirm though, partly because I just know Linux a lot better,
partly because its privilege model is different - privileges
(capabilities) are just flags, which can be added or removed from a
process as you like, vs FreeBSD where they are "active" or "live"
checks, and require code to confirm them each time (ie a MAC module). So
its harder to test, and I'm not totally sure what the common/out-of-the-
box configs look like, so I'm not sure if this change is going to break
things for a lot of existing configs.

There's also other questions in there, like:
- can I be sure GEOM providers will always have a /dev node? (at least
  in common configs)
- can a jail root be given PRIV_ZFS_POOL_CONFIG?
- are device access checks required during boot, when the root pool is
  handed to the OS by the bootloader?

And more!

If you can help with review or have any useful insights. I'd really
appreciate it if you could comment on the PR, reply here, hit me up in
chats or send your very smart friend my way[2].

Thank you!
Rob.

1. https://github.com/openzfs/zfs/pull/18960
2. https://robn.au/