Re: [AVFS] Caching content of extfs archives

> I experienced performance problems when accessing archives in extfs
> archives. For every open, stat, etc. from the inner archive an open on
> the base file in the outer archive is called. Open() in extfs starts the
> corresponding script in the "copyout" mode so it will be extracted to a
> temporary file again and again.
> I implemented a cache for this files. An extfs_open() will use the
> temporary file of an existing cache entry or create a new one. The
> tmpfile is put into the cache so it will be removed again when the disk
> space is needed for another entry.
> 
> At first I used a combination of the cache and the filecache but the
> cache doesn't remove entries itself at the end of the program and the
> filecache doesn't allow access to entries without a key.

You can generate the key from the ventry with av_filecache_getkey().
And filecache should be able to remove the cache entries when avfs
exits: destroy_filecache() will call filecache_delete() which will in
turn unreference the object held in the cache.  If this was the last
reference (as should be at exit), then the object's destructor is
called, where the temporary file can be deleted.

Using it together with "cache" is a bit convoluted, but "cache" has
the advantage of managing the disk space used globally within AVFS.

> So I implemented an own extfs cache which stores the entries up to a
> size limit and kicks out elements which are older than a time limit. It
> works fine so far but of course the latest entry will be in the cache up
> to the end of the program (or avfs daemon).
> 
> I think this is acceptable especially because of the large speedup
> using the cache but what's your opinion about this?

Yes this sounds like a very useful feature.  I think it's OK whether
you do it with the filecache+cache or your own solution, though the
former would be slightly better.

Thanks,
Miklos


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642