[lc-devel] Compressed cache status

[Nitin G. <nit...@gm...>]

Hi Rodrigo,

    It was great to see your reply!
    I've been working on compressed cache port to 2.6 for about 2
months whenever I can take out the time. Currently not a single line
of code has been written. Since I was new to VMM subsystem, the
initial study took too much time. The details seemed overwhelming but
after a lot of work the picture is much clearer now.
    So, the current status is that I have read a lot on VMM and your
paper on adaptive compressed caching. Now, I'm looking for the most
recent changes in kernel that can help compressed cache design.
    Since, I've done some kernel patches (for CIFS vfs) before, which
involved lot of concurrency issues, I have a general feel of kernel
programming. So, I think once a complete "roadmap" is prepared, work
can really speed up.

    So, before I go hunting for the right kernel hooks, I wanted to
have following on paper:

- Compressed cache structure - currently the cell based design (2-4
pages taken as unit) has problem of fragmentation and relies on
compaction as last resort to free-up memory before starting to
swap-out to disks. Although, fragmentation cannot be eliminated but I
think a different design (for which I am looking for) can surely
reduce it.
=09Also, it needs to be addressable storage (kernel asks for page in
swap at xxxxx where is this in compressed cache?). Although you have
implemented this but I couldn't get it.

- Adaptivity heuristic - I've read your current heuristic method but
as you mentioned that rmap and other recent developments in 2.6 can be
really helpful here. So, I'm now reading more on recent developments.
Heuristic quality will be very significant and can determine success
of this work. As you have shown setting up just the right static cache
size for a particular workload is so good for performance while the
wrong choice produces significant slowdown.

- Compression algorithm - these can simply be reused from your implementati=
on.

With these in hand, I can then proceed with implementation: find the
right kernel hooks, implement a "dummy" compressed cache (not really
do anything but just to make sure I've the right entry points), and
then gradually implement all the parts (from static to dynamic cache).

Also, I'm particularly looking forward to its use in:
- LiveCD environment - swap in not available and CD-ROM read speed is
exceptionally slow.
- Virtualized environment - hard-disk is a file backed on host OS. So,
RAM and disc speed gap is even greater here (many more +ve factors
here).

In general i'm not very busy and can spare a lot of time for this
work. But, unfortunately my semester exams are due to start early
december and last for about 3 weeks. However, I'm extremely interested
in this project and it would be great if I can get your help.


Best regards,

Nitin