[lc-devel] "0.24pre8" patch

[John R M. <jm...@st...>]

bluefox@Icechip bluefox $ cat /proc/comp_cache_stat
compressed cache - statistics
 general
 - (S) swap cache support enabled
 - (P) page cache support enabled
 - compressed swap support enabled
 - maximum used size:    1088 KiB
 - comp page size:         32 KiB
 - failed allocations:   5381
 - Default algorithm:  1 (WKdm)

 algorithm WK4x4
 - (C) compressed pages:          0  (S:   0% P:   0%)
 - (D) decompressed pages:        0  (S:   0% P:   0%) D/C:   0%
 - (R) read pages:                0  (S:   0% P:   0%) R/C:   0%
 - (W) written pages:             0  (S:   0% P:   0%) W/C:   0%
   compression ratio:             0% (S:   0% P:   0%)
 
 algorithm WKdm
 - (C) compressed pages:     159233  (S:  76% P:  23%)
 - (D) decompressed pages:    94276  (S:  99% P:   0%) D/C:  59%
 - (R) read pages:             5042  (S:  88% P:  11%) R/C:   3%
 - (W) written pages:         57464  (S: 100% P:   0%) W/C:  36%
   compression ratio:            53% (S:  44% P:  81%)
bluefox@Icechip bluefox $

A few bugs.  It loses some pages once in a while and apps segfault once 
in a while.  We'll say that by Microsoft standards, it's stable.  Quite a 
long way to go before it's safe for production systems.

I'd recommend single or double page size with WKdm (btw I hacked at that
a little. . . looked like you weren't allocating enough ram.  I MAY have
overdone some of it though, because i couldn't tell from the code what 2
of the arrays were supposed to be size-wise).  Also, try out the no
spin_lock() code (under Verbose Options) and the boot-up sanity check
(look at it with dmesg).  That sanity check has caught a few typos for me
that probably would have destroyed my system (fortunately it's just my
laptop I'm testing on).
                                                                                                                                                
                                                                                                                                                
                                                                                                                                                
HACKISH CODE
                                                                                                                                                
Most of my code is VERY hackish.  I don't know how to make SURE that the
algorithm_idx is maintained for each piece of compressed data.  I don't
know how to alter the meta data in the swap pages.  Thus, I just
constantly set the page->algorithm_idx and fragment->algorithm_idx to
default_algorithm_idx whenever they're looked at.  This means that the
algorithms can't be changed or anything!  Until these hacks are removed,
there will be no altering of algorithms while running.  I don't
understand the code enough to make it safe to remove all my hacks.

First thing's first, go into comp_cache.h and find the 
comp_cache_page_metadata structure on lines 275-280.  When reading meta 
data, instead of doing crap like:

fragment_index = meta_data_offset + 4;
size = meta_data_offset;
offset = meta_data_offset + 2;

You should have crap like the following:

meta_data = meta_data_offset;
fragment_index = meta_data->fragment;
size = meta_data->size;
offset = meta_data->offset;

Take a look at proc.c lines 563 to 593 and think about this.  Instead of 
incrimenting by 8 you could incriment by sizeof(struct 
comp_cache_page_metadata).  Thus it would be easy to modify the meta data 
in the page.  It'd also be easier to understand/grok the code.

I'm not fixing the meta data thing.  Castro can, or someone else can fix 
it for him.  I'm done with this at least until that gets taken care of.  
Either way though, don't rely on me to clean up after myself; while 
you're in there fixing that you should also clean up my bad/dirty hacks.  
Most of them are marked.  Just read through proc.c from top to bottom.



UNHANDLED BUSY SIGNALS

Two or three of the functions use the busy signal system I outlined
earlier to be SMP/Pre-empt safe.  When they're busy, they by some method 
return a busy signal.  One of them does it differently and that should be 
rewritten to not be dumb.  Other than that, none of the calls really 
handle the busy signal.  They should wait 200 mS, then try again.  After 
a few tries (maybe 5?  1 whole second is long enough!) they should just 
exit with a failed status.



SMP

Try this code out on SMP if you've got a spare system.  It's buggy, so
YES it will cause quite a few hickups from lost pages, so DO NOT USE ON
SYSTEM WITH CRITICAL DATA!!!  If you enable SMP it will force-enable the
spin_lock() free code.  I'm interested in knowing if this works on SMP 
now.  The following changes have been made:

- No spin_lock() calls.  Functions are safe to call multiple times.
- Separate data for all buffers and algorithms.  Any algorithm can be run 
  multiple times in parallel to compress and decompress, only limited by 
  the amount of physical ram.

I haven't checked for static variables in all functions so be wary of 
that!  It may not yet be SMP safe but I think it is.

--Bluefox Icy