Re: And again memory usage...

On 17 Apr 2003, David Kastrup moaned:
> And then it hit me: the whole directory hierarchy is a big lie,
> anyway.  You can't add any file into it without refreshing the file
> name database correspondingly.  And you have to take care that
> multiple processes don't update it unsynchronized and so on.
> 
> What we need is not a filename database.  What we need is a file
> database.  We need to have all files in the texmf tree stored
> compressed in a database.  Then we let something like PostgreSQL dish

I can't see any advantages to this over the filesystem (which is, after
all, just a hierarchical database).

Plus it breaks the many, many tools which work on filesystems (oops,
grep no longer works, glimpse no longer works, find no longer
works... &c ad nauseam).

> out files on request (using keys similar to what you would throw to
> kpsewhich).  The PostgreSQL server would have to be started by the
> various programs needing to access files, and it would stay a while
> waiting to serve possibly following requests before exiting.  The
> files would be stored under useful keys (basically under the dirless
> filename, while storing the long file name as well to solve
> ambiguities and allow for tree export/import).  One would have the
> ability to import and export files and subtrees from the database (or
> databases: we might well again have system, local and user databases).
> This should also speed up repeated runs.

Um, surely what we're realy asking for is for the silly ls-R file to be
turned off if the machine has enough memory that the OS can be expected
to cache the texmf hierarchy for you? IIRC, right now kpathsea has two
modes, `use ls-R always' and `use ls-R never'; probably we want a `use
ls-R if it exists', and admins can take a local decision whether to use
it or not.

Personally I'd run a mile before storing the texmf tree in a database. I
have half a dozen texmf trees, with identical files hardlinked together;
this saves several hundred MB of space. That's much harder to do with
most (hm, make that `all') conventional databases.

-- 
`It is an unfortunate coincidence that the date locarchive.h was
 written (in hex) matches Ritchie's birthday (in octal).'
               -- Roland McGrath on the libc-alpha list