seekable pipes

Wow, what happened to Ian Turner's "seekable pipes" proposal? 

http://marc.theaimsgroup.com/?l=linux-kernel&m=110153524529112&w=2

I have wanted to propose something very similar. I think it would be a
great way of implementing a lot of functionality that we think of as
belonging to more complicated userspace filesystems. For instance, if
the kernel had seekable pipes then I could write a command that just
serialized the protocol and allowed me to read and write files over an
'ssh' session, with very simple syntax:

some-editor <(ssh-file host:path)
some-editor <(cmd-file ssh host open-file path)

or inside a tar file (maybe reading only), or on an FTP site, etc.

some-editor <(url-file ftp://host:path)

It might even make more sense than the AVFS way of doing things,
because commands can take multiple file arguments:

some-editor <(patched-file file.c file.patch) # writes changes to file.patch

- difficult to do with AVFS file system modules which are "mounted" on
a single base file.

Kernel support could mean support for mmap and exec, which you
couldn't easily get from a library implementation.

Something like this could be the basis and core of a more complete
userspace file system protocol. But since the seekable pipe concept
also has independent applications I think it would be a good thing to
factor out and focus on getting right independently of
namespace-dependent functionality. Plus even without kernel support
for allowing processes to export virtual directories to each other,
seekable pipes would still give shared-library based userspace
filesystems such as AVFS-preload a way to present host processes with
a much more complete file system interface, by solving their mmap and
exec problems.

Ian admits that his idea of a protocol was too simple, since - if you
will allow me to use the terms 'master' and 'slave' - it didn't allow
multiple slaves for one master. And it didn't allow the slave half of
the pipe to write data, which I think would be an important feature;
also, for good random-access performance I think the master should be
able to provide a block of data with an expiration time which is
cached independently of slave seek position.

But aside from that, the protocol need not be complex. Except for the
problem of creating the special file descriptors, the rest could be
done on the master side with read/write (read a request for data,
write a block of data with a description header). Creating the
descriptor pair could be done with an ioctl "promoting" the pipe
descriptors as in Ian's proposal, although since seekable pipes
wouldn't share much in common with regular pipes, the kernel pipe data
structures would probably be created and then destroyed unnecessarily
in this case - ugly if not a performance issue. I guess there are
several other possibilities. Being able to name the slave as with a
devpts-like solution could have advantages but perhaps not enough to
justify the complexity.

I feel that this kind of feature, in some form, could be the shortest
path to a lot of neat applications. What do other people think?

Frederik

-- 
http://ofb.net/~frederik/
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html