[lc-devel] Dump of my brain real quick

[John R M. <jm...@st...>]

I'm very interested in this project.  Having no job and 256 MB RAM,
running Gnome2, and a ton of crap, and having lost a 40 gig HDD recently,
I'm kind of in need of RAM.  At any rate, here's some of the stuff I've 
been thinking:

1) I'm not getting the 2.4.20 patch from CVS to actually work right.  X
won't run (I have nvidia kernel module), although it does on 2.4.21 with
the same setup (which I slammed the .19 patch into by hand).

2) I'm trying to throw in a configure option to select the default
compression algorithm, and also trying to set up the thing to allow me to 
select anywhere from single to 64x pages, in steps of powers of 2, for my 
page size.  Need a working patch to test it though x.x

3) I've examined the compalg= code and it looks like you actually have to 
enter the algorithm_idx number for the alg :/  I can rewrite that to use 
strcmp against "lzo", "wkdm", and "wk4x4", and go with the default if 
not.  Don't really like having to know the internals to use it, 
especially since the algorithm indexes can change for no apparent 
reason.  For example, you could add "gzip", and I'd rather just type 
"gzip" than "3".

4) It'd be nice to be able to enable/disable compressed page cache while 
running, and change algorithms.  The innards would need a slight 
modification; each alg would have to be initialized upon first use, and 
each compressed fragment/page would need to have the idx of the algorithm 
it's using stored within it.  I can try to work on this but I can't make 
any guarenteed commitment to the projet.  The code's pretty easy to grok 
though.

5) Some form of dynamic algorithm choice should be used.  I'm lacking in 
ideas for this.  The best I can say is the following:

- When a page is initially compressed, compress it with each algorithm 
and compare their relative space/speed tradeoffs
- Get the slope by 'plotting' X as time and Y as percentage saved
- Order these slopes from greatest to least
- Find the last slope in the set that is greater than a minimum
- Use that algorithm initially
- Every N compresses (make N a large number like 10240) rerun this check 
against the block (OPTION)
- Have N adjust for each block dynamically; make sure they're not getting 
this check more than once within a certain time interval and if they are, 
grow the number.  (OPTION)  Similarly, if it's getting run less than once 
every X time intervals, shrink N.  (OPTION)

6) Splitting out ccache's algs into modules would be nice.  Particularly, 
the modules should be rewritten as library functions (like zlib is), and 
ccache should access those.  That way other compression modules can be 
added dynamically.  The ccache dynalg module described in #5 should be 
compiled as a module too to facilitate fast testing of new algs.

Well that's about it for me.

--Bluefox Phoenix Lucid