Re: SetFS [Was: Re: Feature freeze imminent]

On 2007-02-27, Sam Mason <[email protected]> wrote:
> If you can think up any cases where you're not going to have a similar
> number of tags to files then I'd be interested to know, I'm failing.

Well, I suppose all the cases I can come up with are quite artificial.
There do tend to be a lot of files with unique names, and the rest
of the components of the path simply help locating them.

> It's up to the users to find the optimal way to use the tools isn't it?

And the optimal tools.

> Wouldn't there be the same number of occurrences for each tag? If so,
> how does the pruner know which one to pick?

It depends on all the results. And, yes, if there are tags with the
same frequencies (and same covered objects), the decision is arbitrary.

> I thought the point was to remove the hierarchy!

The point is to remove any fixed hierarchy. But the user isn't going
to like looking for a needle in a haystack. Therefore you want to 
present a dynamic hierarchy of results for each query: to somehow
prsent in multiple clusters all the files whose sets of tags contain
the search set of tags. These are naturally partially ordered by further
set inclusions, and in some interfaces you might want to show clusters 
by that partial order, i.e. show the files with minimal tag sets among 
the results. (These are also expensive to find: if we had the minimal tag
sets, the pruning would be trivial.) However, to map the results to the 
unix file system, that is not sufficient: readdir should return a list of 
tags, not a list of sets of tags. Therefore we must essentially add
the intervening "directories" between the search set, and the minimal
result tag sets.

>> foo/#created:year=2006/#prune:alltags/ (demonstrating also the
>> additional filters).
>
>:) you said you wanted to keep it simple earlier and now you're
> proposing some ad-hock language to "fix" the pruning process?

Well, additional search filters, for which I've thought the 
#filter:params path elements (and that are ignored for single
file operations as opposed to listings), is one of the few 
features that I've considered something to have there, although
not necessarily in the first versions. They are also a quite
simple faeture: you just essentially 'and' the tag bitmaps with
yet another bitmap. (On the implementation level, this should 
probably be just a functional composition of filters, like unix
pipes.) Obviously it should be possible to add such filters by 
plugins. The #prune control is not exactly such a filter, but 
you could still have it there without significantly complicating
things...

> You mentioned earlier that you'd come up with lots of ideas and
> discarded them because they didn't really solve anything.  I'm not
> really sure what problems you're trying to solve!  

You mentioned that hierarchy naturally suits code trees, and I
agree. So I've thought about ways of having such things on SetFS,
but it just gets too complicated. SetFS simply isn't a tool for
code trees, at least not as we presently think of them.

> I would be personally interested to see if we could turn this into
> something that would be usable as a file system that could be mounted
> to look after most of the files in a user's home directory.

Not necessarily most files in the home directory (there's a lot of
code trees in my $HOME), but most of the data that tends to collect
there almost unorganised -- largely thanks to the cumbersomeness
of saving random cruft on a hierarchical FS, or at least so it would
seem. One needs to experiment and have tools that support the tagged
way of working to find out if it really helps.

> How far apart are our goals?

Not necessarily that far apart.

> p.s. I hope this is somewhat interesting to you, I tend to get carried
> away with these sorts of discussions!

Sure.. and hopefully others on the list too :).

-- 
Tuomo