Re: SetFS

On Tue, Feb 27, 2007 at 08:01:40PM +0000, Tuomo Valkonen wrote:
> On 2007-02-27, Sam Mason <[email protected]> wrote:
> > If you can think up any cases where you're not going to have a similar
> > number of tags to files then I'd be interested to know, I'm failing.
> 
> Well, I suppose all the cases I can come up with are quite artificial.
> There do tend to be a lot of files with unique names, and the rest
> of the components of the path simply help locating them.

That sounds similar to where I ended up!

> > It's up to the users to find the optimal way to use the tools isn't it?
> 
> And the optimal tools.

Indeed.

> > Wouldn't there be the same number of occurrences for each tag? If so,
> > how does the pruner know which one to pick?
> 
> It depends on all the results. And, yes, if there are tags with the
> same frequencies (and same covered objects), the decision is arbitrary.

Good to know, I thought I was missing something.

> The point is to remove any fixed hierarchy. But the user isn't going
> to like looking for a needle in a haystack.

I'm wondering if the name should be specific attribute seperate from the
tags.  This would shrink the set of tag down a lot and I think that may
be all that's needed.

I'd also be tempted to split the tags into two groups "identifier"
tags and "informational" tags.  The unique name check would be done on
(identifier tags,name) and the informational tags would exist to make
searching easier.

> Therefore you want to 
> present a dynamic hierarchy of results for each query: to somehow
> prsent in multiple clusters all the files whose sets of tags contain
> the search set of tags. These are naturally partially ordered by further
> set inclusions, and in some interfaces you might want to show clusters 
> by that partial order, i.e. show the files with minimal tag sets among 
> the results. (These are also expensive to find: if we had the minimal tag
> sets, the pruning would be trivial.) However, to map the results to the 
> unix file system, that is not sufficient: readdir should return a list of 
> tags, not a list of sets of tags. Therefore we must essentially add
> the intervening "directories" between the search set, and the minimal
> result tag sets.

I'm still not sold on the pruning idea.  It could be because I've not
done anything like it before and therefore I'm just worried about it
because I don't know much about it.  I've rationalised this as thinking
there will be problems with scaling, but to be honest I don't know.

> >:) you said you wanted to keep it simple earlier and now you're
> > proposing some ad-hock language to "fix" the pruning process?
> 
> Well, additional search filters, for which I've thought the 
> #filter:params path elements (and that are ignored for single
> file operations as opposed to listings), is one of the few 
> features that I've considered something to have there, although
> not necessarily in the first versions. They are also a quite
> simple faeture: you just essentially 'and' the tag bitmaps with
> yet another bitmap. (On the implementation level, this should 
> probably be just a functional composition of filters, like unix
> pipes.) Obviously it should be possible to add such filters by 
> plugins. The #prune control is not exactly such a filter, but 
> you could still have it there without significantly complicating
> things...

I don't like plugin for things like this, they force you to expose
too many implementation details, make things brittle and resistent to
change.

> You mentioned that hierarchy naturally suits code trees, and I
> agree. So I've thought about ways of having such things on SetFS,
> but it just gets too complicated. SetFS simply isn't a tool for
> code trees, at least not as we presently think of them.

I agree.  I've got a couple of ideas about how to reconsile the view,
but I'll leave it to another note as this one is getting pretty
gigantine already.

> > I would be personally interested to see if we could turn this into
> > something that would be usable as a file system that could be mounted
> > to look after most of the files in a user's home directory.
> 
> Not necessarily most files in the home directory (there's a lot of
> code trees in my $HOME), but most of the data that tends to collect
> there almost unorganised -- largely thanks to the cumbersomeness
> of saving random cruft on a hierarchical FS, or at least so it would
> seem. One needs to experiment and have tools that support the tagged
> way of working to find out if it really helps.

I was meaning, at least initally, that you would probably have a couple
of file systems in your home directory.  One hierachial for code and
other things that suit the hieralial view and another for the tagging.
It would obviously be better if these differences could be reconsiled,
but one step at a time.

> > p.s. I hope this is somewhat interesting to you, I tend to get carried
> > away with these sorts of discussions!
> 
> Sure.. and hopefully others on the list too :).

Lets hope so!


  Sam