Re: netbsd version of compressed caching

[added tech-kern]

LKM/module or not shouldn't really be a question for a pseudo device, 
AFAIU the new modules framework will allow to easily compile code into the 
kernel or use as a module. I wouldn't worry too much there.

About compressino algorithm, have a look at the vnd(4) driver, which has 
an option to use images that are compressed with in the "cloop" format, 
which internally uses the gzip algorithm (IIRC).

I think the most interesting part will be management of "free" space, as 
you can't assume the compressed blocks to be of a fixed size - if you 
replace one (compressed) "page" with another one, the new one may not fit 
as it's too big. vnd(4) avoids this by insisting that compressed images 
need to be read-only (and compressed in userland, see vndcompress(1)).


  - Hubert


On Wed, 10 Dec 2008, Selcuk AYA wrote:
> Hi,
> I found about the compressed caching project on the netbsd projects
> page.  The page shows tech-embed mailing list as the mailing list to
> contact. I have no NETBSD experience but I have been poking around the
> code to get an understanding of uvm interfaces and pseudo drivers and
> I think I can get the job done. I have also looked at the Linux
> implementation of compressed caching. I belive there are three ways to
> implement compressed caching in NETBSD:
>
> a) as a pseudo device inside the kernel. This pesuodo device would be
> used as a swap partition. Behind the scenes, it would compress pages
> which need to be swapped out and store them in memory.
> b) similar to a) but this would be a pseudo driver implemented as an
> loadable kernel module.
>
> For these two options, I found out that md.c (ram disk driver) in the
> kernel serves as a good starting point to write the pseudo driver.
>
> c)Inside the kernel, swap pager would need to be made aware of
> compressed caching. I imagine swap pager keeps a map to keep track of
> where swapped out pages are on the swap device. Some of these entries
> would point to in memory compressed pages.
>
> For all these options, compressed caching will have to manage its own
> memory and keep track of variable sized chunks (I do not have a
> detailed idea about this yet). I would prefer to go with options a) or
> b) as I do not see a an advantage of making the swap pager aware of
> compressed caching. Would an lkm be OK for an embedded system?
>
>
> Being new to the open source world, I am worried about the licensing
> issues. Linux compressed caching uses LZO algorithm for compression.
> LZO is released under GPL so I guess we cannot use it in NETBSD. But
> there are are other alternatives which do not bear the sign of GPL
> (http://www.cs.utexas.edu/users/oops/compressed-caching/index.html) .
> If I use a non GPL licenced compress/decompress algorithm and
> re-implement compressed caching as above for NETBSD, do I avoid
> licencing issues?
>
>
> thanks.
>