[lc-devel] CCache work status

[Nitin G. <nit...@gm...>]

Hi,

This is kind of weekly work summary for ccache work... :)

I originally started this project with focus on "general" desktop systems 
(OpenOffice + KDE/Gnome on 128MB RAM?). But now I'll instead focus it on much 
more exciting case -- The OLPC systems. This should also bring its applicability 
to a wide range of Linux based embedded devices. This does not put original goal 
out of way but it just results in some change of priority - much of things are 
same for both.

The most important things I learnt for OLPC systems w.r.t ccaching are:

-- They DO NOT want on-disk compressed swapping (problems: slow speed for 
writing to flash space, wear leveling).
-- For in-memory compression, they have much higher priority for compressing 
anonymous memory pages instead of filesystem backed (page-cache) pages.

I originally started out to handle page-cache pages. Since I'm now focusing on 
OLPC systems, I'll change gears to anon pages now. This has slowed development 
for time being as I still have only half-baked roadmap for anon pages, but now 
design work has progressed nicely and have started its implementation work :)
--------------------------------------------------------------------

The approach I've taken is this:

* General
It creates a virtual swap device (another swap_info_struct in swap_info[]). Its 
size is set though a /proc entry created by 'cc_control' module (attached). This 
swap is set to highest priority and is identified in other code paths with 
SWP_COMPRESSED flag. So, when adding page to swap cache, it is given 'type' no. 
as for this virt swap until it reaches its max capacity (as set through /proc 
interface).

* Storing page to ccache
When swap to disk occurs in pageout(), same things will occur as is done for 
page-cache pages -- it doesn't need to know if it's anon or fs-backed page. 
These things are - replace corresponding swapper radix tree node to a 
'chunk_head' which then points to the location of 'chunks' in ccache storage 
structure.

* Restoring an anon page from ccache
When page fault occurs, handle_pte_fault() first checks swap cache if it already 
has required page which then calls find_get_page() over swapper space radix 
tree. Here, a compressed page is handled exactly as for page cache page i.e. 
checked if looked-up page has PG_compressed set, decompress it and make radix 
node point back to this uncompressed page.

* Notes
-- This approach deals easily with case where no real swap devices exist. This 
virt swap is a separate entity with all separate data structures like swap_map[] 
as for a normal swap device.
-- This only requires working with arch independent swp_entry_t instead of arch 
dependent representation of it in PTEs.
-- More code sharing for handling page-cache and swap-cache pages - much of code 
wouldn't know if its working with anon or fs-backed page like 
add_to_swap_cache(), find_get_page() (and friends).

* About files attached
-- cc_control module: This creates two proc entries: 
/proc/cc_control/{max_anon_cc_size, max_fs_backed_cc_size}. Write to 
max_anon_cc_size, the size of ccache for anon pages in units of no. of pages. 
Writing this value calls set_anon_cc_size() in mm/swapfile.c which creates this 
virtual swap device with size as passed and highest priority (hard-coded to 
100). (max_fs_backed_cc_size is dummy for now).
-- patch for 2.6.17-rc5 that has this set_anon_cc_size() with some small misc 
handling.

TODO Next: merge work from prev patches (as on CompressedCaching/Code) with this 
one to get on with anon page ccache.

(I'll try to update CompressedCaching page soon for anon page handling)


PS: Marcelo, I'm unable to send mail to marcelo AT kvack DOT org (just get mail 
delivery failure notice), so sending to mtosatti AT redhat DOT com instead. 
Anyway, added your kvack address too, to see if I get it this time... :)



Cheers,
Nitin Gupta