Preferring pointers to vm_offset_t

On CHERI architectures, pointers (and uintptr_t) are not the same size as
addresses.  CHERI introduces a new ptraddr_t type (that is basically an
alias of size_t) to represent a scalar integer type that can hold addresses.

Various places in FreeBSD have conflated pointers and addresses.  For example,
rtld and the kernel linker often treat Elf_Addr as if it can hold a uintptr_t
which is not true on CHERI.  Another example is vm_offset_t in the VM system
and related APIs.

When porting FreeBSD to CHERI, we have usually taken the approach of migrating
to uintptr_t for these cases (e.g. see recent changes from Jessica that
converted much of the TLS handling in rtld to use uintptr_t instead of Elf_Addr).
For the kernel, CheriBSD currently adds a new vm_pointer_t type that is an alias
of uintptr_t and leaves vm_offset_t as a type that only describes addresses.

However, this approach can have some downsides.  Notably, C allows silent
promotion/demotion of integer types, and in turn this means that assiging a
uintptr_t to a ptraddr_t (or vice versa) compiles fine without any warnings.
This does mean that if you accidentally round trip a pointer via a ptraddr_t
(e.g. using vm_offset_t instead of vm_pointer_t in the kernel), you silently
strip the metadata and validity from the pointer making it unusable.

At the same time, many of our current in-kernel APIs that operate on kernel
pointers (i.e. the calling code intends to dereference these pointers
directly) use vm_offset_t and require casts.

To that end, the approach we believe we want to take for upstreaming CHERI to
the kernel is to adjust APIs that work with kernel pointers to use pointer types
such as void * (or occasionally char * if pointer arithmetic is commonly used
on the value or struct member) instead of vm_offset_t.  In practice this seems
to remove the need for casts in lots of places, and it enforces compile-time
type checking both for CHERI and non-CHERI architectures (so that amd64 builds
also break if you get this wrong, not just compiles on CHERI architectures).

I've already upstreamed a few changes in this vein:

e48770de6831 arm64: Use void pointers for arguments to arm64_get_writable_addr
1e3f42b6bad5 arm64: Switch the address argument to cpu_*cache* to a pointer
eab7ae7811f6 arm64 gicv3: Use void pointers instead of vm_offset_t
7ae99f80b666 pmap_unmapdev/bios: Accept a pointer instead of a vm_offset_t.

I have a larger batch of these changes queued up in a branch for review.  Due to
the size of the branch, I've posted it on GH as phab really doesn't scale well
for large branches:

https://github.com/freebsd/freebsd-src/pull/2068

This affects APIs such as PHYS_TO_DMAP, kva_*, pmap_quick_*, pmap_map, pmap_qenter,
td_kstack, and some others.

-- 
John Baldwin