Re: [lc-devel] Compressed caching

[John M. <joh...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Nitin Gupta wrote:
> 
> John Moser wrote:
>>> Just a quick note: We can't have floating point in kernel. So, I doubt
>>> how will we have monte-carlo algorithm ported to kernel?
>>>
>>
>> Nods.  I don't know if it can be rewritten in fixed point; however, it
>> is notable that mcarlo is probably not as useful as counting null bytes.
>>  Wilson-Kaplan algorithms treat 0 as a special case; think about
>> anonymous memory:

[......]

> /* --------------- Frame This One
> ---------------------------------------- */
>> At any rate, just running through with a NULL byte counter is probably
>> enough to decide whether to use WK or LZ.
>>
> /* ------- Simple, Fast and Reliable
> ------------------------------------- */
> 
> Great suggestion John Moser :-)
> 
> And..I think, this NULL byte count can be really optimized to a level
> that time to perform this _each_ time when compressing pages should be
> far compensated by time and space we'll save by having a good hit at
> selecting right compression algorithm.
> 

Optimizing, you can probably have it escape out when it hits a certain
number of NULLs; but you still want to avoid running it every time.
Mostly a page of memory is going to stay what it is; the stack, for
example, will always have loads of padding in it except for the case of
large on-stack buffers full of text and such (typically such buffers are
short and negligible; large ones would mean more than say half a page?).

At the very least, there's a lot of memory that won't change, and we can
avoid re-working this when we see that stuff come back.  This is
especially true of mmap()'d segments of executables and libraries,
although those aren't normally swapped anyway (they get invalidated and
cleared from memory, re-read from disk later; in the future compressed
caching could compress that stuff and just invalidate/free it when
memory pressure is high rather than sending it to swap).  There's other
data that behaves this way, though; for example, most of the control
information for GUIs like GTK won't change once the window is initially
built and drawn..

> 
> 
> Cheers,
> Nitin Gupta
> 
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRJnc5gs1xW0HCTEFAQJ5mRAAgfK7ZJFGudGjZpW9yqtXeCUGMCg8E2ss
8KLEl+bT+Jzja8+Qh+DYk1N6mMx4G7l0w/wR+M4JdnDEC2v/EzGMsbQ2IGWsEEl0
7eKpCUqvJUNiwaiNhsdoUy2BrC/6H3EXt4EEoqJtkFGAY1XTqT3kzHzopFlcFDPC
XOQaLMvnDGIIVLarH+CY6SfWvBv6OcEYTLYFg43FWhZyOhmZboQGAy6CAp9XYELq
0MsmSeM1FA4cHQOAqd68HCY23ZubS1sy/IrnfFaxp3z5M/YQr797sMLceyPjM2Ao
E/ZI3eVbtG+Qm+ZKqjpjGB+d/4MlD5brr/MlEpQqVXmGNkNTCMiwLW29r3/6or28
BIwkZ08kLOU/plqa5jum9nvb7IfQVK9ITUTRC39J9+xETCRAIV5mgR7PSCaIo7jm
bbA2L6dizHZIfr7uwYIQB8dCT2+qKZipOCPHyeDtVfacJ1xPwch3hImjh0Jdg6KK
lePMN52MLVlRp6fL7n3eEkMYXb9q2r3BzJ1jT+XBtF4xgfZXFxWpPEvUlhuFcZtD
bxSS/cslgyHDajxMl4N9jffTd6wYKBMwsFKllNQO6YVSSS6T7KGpPRdwIAOj9Wq4
6/3HFmDQRsDjgsnwkFZx1toViSTTsqk518uJPgqFnT2x+6stC8hk8T0GvMCCAmQi
kXZK9r5RaYQ=
=l3kb
-----END PGP SIGNATURE-----