Re: Arla on FreeBSD

On Thu, Feb 15, 2007 at 11:16:00AM +0100, Tomas Olsson wrote:
> Kostik Belousov <[email protected]> writes:
> > > I'm already funded and can work full time on this, but a FreeBSD hacker
> > > would help a lot.  Any volunteers?
> > 
> > Sorry for me pointing out obvious, why not continue to use fs@ as place
> > where to ask ?
> >
> You're very right, I'm just to shy to do it... Thanks.
> 
> Anyway;
> 
> Arla is built around a "small" caching fs driver (nnpfs) servicing user
> requests by asking the 'arlad' daemon for help or just operating on local
> files created/fetched by arlad. They communicate over a char device.
> 
> A simple read would be handled as such:
>         getnode/getdata rpc to arlad
>         installnode/installdata + wakeup msgs from arlad
>         VOP_READ() on newly fetched cache file
> 
> Subsequent reads on the same data would skip the rpc part, unless arlad has
> invalidated the node.
> 
> Previously, there was a 1:1 mapping between nnpfs vnode and cache file. The
> installdata message was then handled by fetching the cache file's vnode (in
> arlad's context), storing it in the nnpfs_node for future reference/access.
> Now we ended up with one cache file per "block" (large) of data, and
> decided that it would be better to open/access/close the cache "block" file
> on each access.  The closest we could get to the olden ways was to open the
> directory where a node's cache blocks reside, in arlad's context.
> 
> The interesting part is how we open and access the cache files, and from
> what context. arlad is in chroot() to avoid recursive lookups across /, and
> it seems like a good idea to avoid such lookups now too.
> 
> So the main question is how to properly do VOP_{LOOKUP,CREATE,WRITE} etc on
> cache files in this dual context world, without mixing identities in bad
> ways or confusing the OS too much.
> 
> The currently messed up code lives in
> http://cvsweb.stacken.kth.se/cvsweb.cgi/arla/nnpfs/bsd/
> 
> Most interesting is nnpfs_vnodeops-common.c (nnpfs_write_common()) and
> nnpfs_blocks.c (open_file())

I made really quick look at the places you mentioned. I have some
comment for open_file(). For FreeBSD >= 6.x, the right way to open vnode
from the kernel code is to use vn_open() (and then vn_close()) API.
Something along the lines (this is for already existing file):

	td = curthread;
	NDINIT(&nd, LOOKUP, FOLLOW | MPSAFE, UIO_USERSPACE, fname, td);
	flags = FREAD | FWRITE;
	error = vn_open(&nd, &flags, 0, -1);
	if (error)
		return (error);
	vfslocked = NDHASGIANT(&nd);
	NDFREE(&nd, NDF_ONLY_PNBUF);
	vp = nd.ni_vp;
vp is now locked, shall be unlocked by VOP_UNLOCK() before returning to
usermode. Giant is conditionally locked based on MP-safeness of the fs vp
belongs to. When Giant-protected region shall be leaved, use
	VFS_UNLOCK_GIANT(vfslocked);

To close the vnode, use
	vn_close(vp, FREAD|FWRITE, td->td_ucred, td);

See, for instance, kern/kern_ktrace.c, ufs/ufs/ufs_quota.c or
security/audit/audit_syscalls.c for real code that does this.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (FreeBSD)

iD8DBQFF1DnRC3+MBN1Mb4gRAiqAAKCSlzaxnV3rWWufc/oKuB54/U0/sgCg3gY0
jAtwlMRvtskEz0U0iWUVkuA=
=jnpE
-----END PGP SIGNATURE-----