And again memory usage...

That one could get along with a single executable and pool and format
files for (PDF)?e?TeX, I already clamored about.  And people told me
that it would be peanuts.  Never mind the peanuts, I think it
worthwhile if people don't ask confused why some programs work and
some not (some might need eTeX).

But back to peanuts: the real memory killer are lots of files.  And
they are mostly used read-only.  So the obvious solution is to
compress them and use something like zlib to transparently decompress
them again.  Easy enough, and will probably lead even to a speedup
since fewer sectors need to be read from CD.  But a small file taking
just one cluster will still take one cluster.

And then it hit me: the whole directory hierarchy is a big lie,
anyway.  You can't add any file into it without refreshing the file
name database correspondingly.  And you have to take care that
multiple processes don't update it unsynchronized and so on.

What we need is not a filename database.  What we need is a file
database.  We need to have all files in the texmf tree stored
compressed in a database.  Then we let something like PostgreSQL dish
out files on request (using keys similar to what you would throw to
kpsewhich).  The PostgreSQL server would have to be started by the
various programs needing to access files, and it would stay a while
waiting to serve possibly following requests before exiting.  The
files would be stored under useful keys (basically under the dirless
filename, while storing the long file name as well to solve
ambiguities and allow for tree export/import).  One would have the
ability to import and export files and subtrees from the database (or
databases: we might well again have system, local and user databases).
This should also speed up repeated runs.

Probably a combined kpathsea/web2c feature: I don't know whether the
API of kpathsea would make a completely transparent implementation
with the same API feasible currently.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum