Re: virtual mountpoints (was: AVFS (but not only) + FUSE : access() returns W_OK, but opening r+ fails)

On 5/2/07, Miklos Szeredi <[email protected]> wrote:
> [CC'd avf-user as well]

Oh, that was the first thing I looked at, but it seemed that all the
latest archived message on the SF.net site were spam ... which is why
I reverted to the FUSE list for my query :)

> It's definitely not abandoned, Ralf is taking good care of it, and
> does releases once in a while.

That's very nice to know. Since it wasn't even listed in the FUSE
wiki, I was afraid this jewel would be lost :) I've just added an
entry to the wiki, in the ArchiveFileSystem page (I'm not sure it
should be there, though, given its wide scope...), I hope it's all
right.

> The biggest problem with avfs/fuse is I think the lack of the "magic"
> access to archived files in the normal filesystem.  What I mean, is
> that
>
>   ls foobar.zip#/
>
> or
>
>   ls foobar.zip/
>
> should just work, as it used to with the "LD_PRELOAD hack" and later
> the "redir hack".  On a moder linux system this doesn't work any more.
> Progress in linux should not be blamed for this, it is rather that the
> old hacks which made it possible have been slowly cleaned up.  And so
> now we're forced to do this properly, which is not trivial.

Yes, you probably noticed I realized that by the bug report about
trying to compile CODA support with the latest kernels :) I'm
currently working around this limitation with a hand-written script
that given a file creates a symlink to the corresponding directory in
.avfs

I suspect the only solution would be to implement some kind of virtual
mountpoint in the Linux VFS, with hooks to have userspace control it.
This would allow for example the implementation of many of the
reiserfs4 claiimed features in a more filesystem-agnostic way.

The idea I had in mind was something like this: when an app is
enumerating the entries in a particular directory, the VFS will return
all the entries 'really' in the directory (as provided by the
underlying filesystem) and then query all the registered VMPP (Virtual
MountPoint Providers) for additional entries, passing the directory
and the list of entries enumerated so far as parameters. Each VMPP
would then use this information to add more entries, making sure that
the 'constructed' entries don't exist already.

These VMPs (Virtual MountPoints) would be handled by the corresponding
VMPP, and would not necessairly be directories to other filesystems
(for example, somefile.ext.gz could have a corresponding virtual entry
somefile.ext which is the gunzipped file content), but I'm not sure
about this, see below.

I'm not a kernel hacker, but I don't see this as particularly
difficult to implement code-wise. However, I do realize there are some
issues when it comes to usage with *some* userspace clients. The
biggest problem, as I see it, are tar, cp and other similar archiving
program.

The most obvious solutiong, which would however require userspace
support before deployment, is to add some IS_VIRTUAL flag to
direntries, so that programs would know to skip it in particular
circumstances (or with appropriate userspace flags).

A different approach, to be used possibly together with the previous,
at least in the initial deployment stage (i.e. while userspace isn't
still very aware of the virtuality of some mountpoints) would be to
make virtual mountpoints available to direct access, but not
enumerable via standard methods. System calls to enumerate virtual
mountpoints could be added, and some basic userspace program to call
them.

Also, if we only allow directories as virtual mountpoints, since their
content would be marked as being on separate filesystems, it would be
possible to skip them more easily even while userspace doesn't
understand them yet.

BTW, is this mailing list appropriate or should this kind of
discussion be moved to something where it's more on topic?

-- 
Giuseppe "Oblomov" Bilotta

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/