Re: SetFS

On 2007-02-27, Sam Mason <[email protected]> wrote:
> I'm wondering if the name should be specific attribute seperate from the
> tags.  This would shrink the set of tag down a lot and I think that may
> be all that's needed.

You shouldn't need this conceptual split for the optimisation: just
consider every tag with global frequency of just one, a "filename"
in the pruning code, and ignore them initially, only adding them
to the result, for the objects that the other tags didn't cover.
This, of course, depends on knowing the frequency one tags, and
in a basic implementation loading that information from the disk, 
which adds some overhead. You could, however, have a special ID 
range reserved for the frequency one tags. It would complicate
things a bit, though, as you have to change the tag IDs, as the
frequencies change (but only for one object, so it's not expensive).

> I'd also be tempted to split the tags into two groups "identifier"
> tags and "informational" tags.  The unique name check would be done on
> (identifier tags,name) and the informational tags would exist to make
> searching easier.

Identifier tags would essentially be another filter :). You would
also want to mark them in a special manner in the name, so they
can be ignored for single file operations, for storing the
location of a file, and so on.

One related thing that I've thought about (a little but not enough)
is typed tags or meta-tags. So e.g. the tag 'ion' could have the 
meta-tag 'project'. The path component #project would then filter
out everything that doesn't have a 'project' meta-tagged tag. 
Furthermore creating an object with tags ion.project/foo/bar, would 
fail if 'ion' isn't meta-tagged 'project', and if the ion tag doesn't
exist, meta-tag it a project. 

Alternatively, the meta-tags could only apply to tags within objects,
and could be used to further filter the results, so that only things 
with the 'project' meta-tag applied to the 'ion' tag would show up. 
However, the meta-tag would still not contribute to the name, and you
could not have both 'ion.bar/foo' and 'ion.baz/foo', as both would 
still have the name 'ion/foo'.

Or something like that. Like I said, it's a fancy idea that I 
haven't thought enough about, and I'm not sure I want that 
complexity... It's starting to sound like metacrap :).

> I'm still not sold on the pruning idea.  It could be because I've not
> done anything like it before and therefore I'm just worried about it
> because I don't know much about it.  I've rationalised this as thinking
> there will be problems with scaling, but to be honest I don't know.

It is an expensive operation, but as I've mentioned, various 
not-so-expensive approximations can be devised, that might still
provide more useful results than a simple listing of all tags seen.
It's something to be researched and tried out. The particular 
pruning method seems unlikely to be something integral to the 
design of the FS, although some optimisations might be stored
(in just another file within the FS, if it even were a full FS).

> I don't like plugin for things like this, they force you to expose
> too many implementation details, make things brittle and resistent to
> change.

Not necessarily. Just consider the plugins storing another table
within a DB for their index.

-- 
Tuomo