IOCache vs. block cache

Howdy,

I've been meditating a bit on how the block cache is favored over the file 
cache ATM. The problem is that the file cache smoothly integrates with the 
page allocation backend -- i.e. when there are no/too few free pages 
available, cached pages are recycled -- while the block cache grows 
unboundedly until a low resource limit is hit (pages, memory, or address 
space). The pages limit cannot be reached as long as there still are cached 
pages, so, unless memory or address space limit are hit first, as soon as 
all pages are used (actually or for caching) the block cache starts 
draining the file caches. I.e. very old cached blocks are favored over 
increasingly newer cached file content.

I think eventually the block cache should be rewritten to be VMCache based 
with on-demand mapping of hot blocks, but for the time being a simpler 
solution would be possible: The IOCache could be extended to make it 
possible to lay it under the IOSchedulerSimple. This would give us a 
VMCache based cache close to the device. The block cache could then work 
with a very small, limited (!) set of unused blocks.

The advantages are:

* The block cache would no longer exert any memory pressure.

* The real caching happens in the IOCache, which integrates smoothly with 
the page allocation backend. The file cache is no longer at a disadvantage.

* There won't be anymore double-caching when mounting image files (file 
cache for the file, block cache for the FS on top of it).

* Since the block cache is rather dumb ATM, there's some chance of serious 
performance improvements, particulary when assuming that FSs store related 
data close to each other. The block cache reads individual blocks, the 
IOCache always reads whole cache lines. So we'd automatically get 
pre-caching.

The only disadvantage I see is the double caching for file content: The 
IOCache caches at device level, the file caches at file level. On the 
bright side, if the the file content is actually read again later, the 
cached pages in the IOCache will eventually be recycled, while the file 
cache pages will prevail. Writes to the file will refresh the IOCache pages 
as well, though. OTOH less aggressive caching in the file cache (e.g. 
detect contiguous reads/writes and drop older pages early) would improve 
the situation -- incorrectly predicted future access behavior would be 
rather harmless.

I haven't looked into the USB mass storage situation yet. There might be 
additional advantages or disadvantages in this department.

CU, Ingo

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate 
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the 
lucky parental unit.  See the prize list and enter to win: 
http://p.sf.net/sfu/thinkgeek-promo