Thread: [lc-devel] Announce: Compressed cache 'alpha-002' release
Status: Beta
Brought to you by:
nitin_sf
Menu
▾
▴
linuxcompressed-devel
|
From: Nitin G. <nit...@gm...> - 2006-08-06 20:41:25
Attachments:
patch-ccache-anon-alpha-002-2.6.18-rc3.diff
|
------------ Project: Compressed Caching for Linux Git (web): http://dev.laptop.org/git.do?p=projects/linux-mm-cc Git (git): git://dev.laptop.org/projects/linux-mm-cc ------------ Hi! This is weekly work summary [10/12] or I thought, better call it 'Announce' ;) Attached is patch for compressed caching for anon pages (2.6.18-rc3) - 'alpha-002' :) Finally, 'better' locking scheme in find_get_page() is working together with compression structure. This equates to 'nearly' complete compressed caching feature for anonymous pages. But, I haven't yet made any benchmarks though! Many of the suggestions from previous post for cache-alpha-1 have been taken to this patch. I will be glad if you can take time to look at this patch too for _any_ of suggestions you have. There are also some known glitches which I couldn't solve now. Known problems: -- Pages once decompressed are not put back on LRU (active?). Doing so is causing system freeze and I couldn't get to the cause yet. -- LZO is still giving problems. System freezes just after some pages are compressed. Biggest problem is that it requires mammoth 64K buffer to compress a single page. -- How to 'migrate' incompressible pages to swap disk? This is not an issue for OLPC as they do not have a swap. So, here's what I aim by end of SoC: (Riel, can you please suggest here?) * Either Start work on to add support for page cache pages and heuristic for adaptive ccache resize. * Or Work to stabilize this much for anonymous pages, code cleanups, testing on OLPC board, (maybe) document all this stuff. Cheers, Nitin |
|
From: Nitin G. <nit...@gm...> - 2006-08-06 21:01:24
|
The patch is against 2.6.18-rc3-git7 which is (I think) same as 2.6.18-rc4. So, please apply it against -rc4 not -rc3. (little) testing was done on x86 _only_ (taken RAM=128M, max_anon_cc_size=1000 pages -- started KDE, launched apps like KMail, firefox, kpaint etc.) On 8/7/06, Nitin Gupta <nit...@gm...> wrote: > ------------ > Project: Compressed Caching for Linux > Git (web): http://dev.laptop.org/git.do?p=projects/linux-mm-cc > Git (git): git://dev.laptop.org/projects/linux-mm-cc > ------------ > > Hi! > > This is weekly work summary [10/12] or I thought, better call it 'Announce' ;) > > Attached is patch for compressed caching for anon pages (2.6.18-rc3) - > 'alpha-002' :) > > Finally, 'better' locking scheme in find_get_page() is working together > with compression structure. This equates to 'nearly' complete compressed > caching feature for anonymous pages. But, I haven't yet made any benchmarks > though! > > Many of the suggestions from previous post for cache-alpha-1 have been taken to > this patch. I will be glad if you can take time to look at this patch too for > _any_ of suggestions you have. There are also some known glitches which I > couldn't solve now. > > Known problems: > -- Pages once decompressed are not put back on LRU (active?). Doing so is > causing system freeze and I couldn't get to the cause yet. > > -- LZO is still giving problems. System freezes just after some pages are > compressed. Biggest problem is that it requires mammoth 64K buffer to compress a > single page. > > -- How to 'migrate' incompressible pages to swap disk? This is not an issue for > OLPC as they do not have a swap. > > So, here's what I aim by end of SoC: > (Riel, can you please suggest here?) > * Either > Start work on to add support for page cache pages and heuristic for > adaptive ccache resize. > * Or > Work to stabilize this much for anonymous pages, code cleanups, testing > on OLPC board, (maybe) document all this stuff. > > > Cheers, > Nitin > > > |
|
From: Rik v. R. <ri...@re...> - 2006-08-08 15:41:40
|
Nitin Gupta wrote: > So, here's what I aim by end of SoC: > (Riel, can you please suggest here?) > * Either > Start work on to add support for page cache pages and heuristic for > adaptive ccache resize. > * Or > Work to stabilize this much for anonymous pages, code cleanups, testing > on OLPC board, (maybe) document all this stuff. Marcelo isn't currently online due to an accident, but I talked to him on the phone today. He said he definately wants to use compressed swap on OLPC, so I think it would be good if you could work on stabilizing the current code for use on the OLPC board. Open source being what it is, I'm sure somebody will add the page cache stuff and the adaptive ccache bits. Especially if they have a stable base to work from :) -- All Rights Reversed |
|
From: Christopher B. <bli...@re...> - 2006-08-09 02:30:37
|
Rik van Riel wrote: > Marcelo isn't currently online due to an accident, but I talked to > him on the phone today. He said he definately wants to use compressed > swap on OLPC, so I think it would be good if you could work on > stabilizing the current code for use on the OLPC board. > > Open source being what it is, I'm sure somebody will add the page > cache stuff and the adaptive ccache bits. Especially if they have > a stable base to work from :) > We've talked on and off about allowing some limited swapping to the flash under emergency circumstances. But no one has come along with a concrete way to do that yet. --Chris |
|
From: Rik v. R. <ri...@re...> - 2006-08-09 03:07:29
|
Christopher Blizzard wrote: > Rik van Riel wrote: >> Marcelo isn't currently online due to an accident, but I talked to >> him on the phone today. He said he definately wants to use compressed >> swap on OLPC, so I think it would be good if you could work on >> stabilizing the current code for use on the OLPC board. > > We've talked on and off about allowing some limited swapping to the > flash under emergency circumstances. But no one has come along with a > concrete way to do that yet. Makes sense, but hopefully with Nitin's code we can just compress less-used memory and avoid swapping :) -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan |
|
From: Christopher B. <bli...@re...> - 2006-08-09 03:28:50
|
Rik van Riel wrote: > Christopher Blizzard wrote: >> Rik van Riel wrote: >>> Marcelo isn't currently online due to an accident, but I talked to >>> him on the phone today. He said he definately wants to use compressed >>> swap on OLPC, so I think it would be good if you could work on >>> stabilizing the current code for use on the OLPC board. >> >> We've talked on and off about allowing some limited swapping to the >> flash under emergency circumstances. But no one has come along with a >> concrete way to do that yet. > > Makes sense, but hopefully with Nitin's code we can just compress > less-used memory and avoid swapping :) > Oh, sure, sure. I was just saying "we're keeping our options open." :) --Chris |
|
From: Nitin G. <nit...@gm...> - 2006-08-09 06:36:46
|
Christopher Blizzard wrote: > Rik van Riel wrote: >> Marcelo isn't currently online due to an accident, but I talked to >> him on the phone today. He said he definately wants to use compressed >> swap on OLPC, so I think it would be good if you could work on >> stabilizing the current code for use on the OLPC board. >> >> Open source being what it is, I'm sure somebody will add the page >> cache stuff and the adaptive ccache bits. Especially if they have >> a stable base to work from :) >> > > We've talked on and off about allowing some limited swapping to the > flash under emergency circumstances. But no one has come along with a > concrete way to do that yet. > I think its already done. When compressed cache reaches its size limit (set by user), pages as sent to swap disks. But, if you meant handling wear leveling issues (and maybe others) for swapping over flash, then that is not yet done :) The usual swapping code is still used which aims to minimize disk seeks. -- Nitin |
|
From: Jim G. <jg...@la...> - 2006-08-09 13:43:31
|
On Wed, 2006-08-09 at 12:06 +0530, Nitin Gupta wrote:
> Christopher Blizzard wrote:
> > Rik van Riel wrote:
> >> Marcelo isn't currently online due to an accident, but I talked to
> >> him on the phone today. He said he definately wants to use compressed
> >> swap on OLPC, so I think it would be good if you could work on
> >> stabilizing the current code for use on the OLPC board.
> >>
> >> Open source being what it is, I'm sure somebody will add the page
> >> cache stuff and the adaptive ccache bits. Especially if they have
> >> a stable base to work from :)
> >>
> >
> > We've talked on and off about allowing some limited swapping to the
> > flash under emergency circumstances. But no one has come along with a
> > concrete way to do that yet.
> >
> I think its already done. When compressed cache reaches its size limit
> (set by user), pages as sent to swap disks.
>
> But, if you meant handling wear leveling issues (and maybe others) for
> swapping over flash, then that is not yet done :) The usual swapping
> code is still used which aims to minimize disk seeks.
>
That's ok, I think. IIRC, IBM released a flash layer recently that will
allow for wear leveling even with multiple partitions. Dave Woodhouse
hopes to get it set up for us to use. Even if not, we could swap to
file, where wear leveling would take place.
Regards,
- Jim
--
Jim Gettys
One Laptop Per Child
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X