Re: [lc-devel] "0.24pre8" patch

[Bob A. <cu...@pb...>]

ah, i almost forgot...

there is openmosix kernel patch, which is very likely to be compatible with ccache, but i doubt that in SMP mode

i think it could be good start to try to merge oM patch with your vanilla, and then try to make them both live together
just a thought, because compressed cache data migrating over cluster would _greatly_ improve performance (as bandwitch is main issue then) , also shared compressed cache over cluster would be good thing to consider...

anyway , even if you do not plan any development toward this, i would recommend you to at least  lurk @ oM code, and point out possible problems with ccache patch to oM developers
 

On Thu, 16 Oct 2003 16:37:34 -0400 (EDT)
John R Moser <jm...@st...> wrote:

> bluefox@Icechip bluefox $ cat /proc/comp_cache_stat
> compressed cache - statistics
>  general
>  - (S) swap cache support enabled
>  - (P) page cache support enabled
>  - compressed swap support enabled
>  - maximum used size:    1088 KiB
>  - comp page size:         32 KiB
>  - failed allocations:   5381
>  - Default algorithm:  1 (WKdm)
> 
>  algorithm WK4x4
>  - (C) compressed pages:          0  (S:   0% P:   0%)
>  - (D) decompressed pages:        0  (S:   0% P:   0%) D/C:   0%
>  - (R) read pages:                0  (S:   0% P:   0%) R/C:   0%
>  - (W) written pages:             0  (S:   0% P:   0%) W/C:   0%
>    compression ratio:             0% (S:   0% P:   0%)
>  
>  algorithm WKdm
>  - (C) compressed pages:     159233  (S:  76% P:  23%)
>  - (D) decompressed pages:    94276  (S:  99% P:   0%) D/C:  59%
>  - (R) read pages:             5042  (S:  88% P:  11%) R/C:   3%
>  - (W) written pages:         57464  (S: 100% P:   0%) W/C:  36%
>    compression ratio:            53% (S:  44% P:  81%)
> bluefox@Icechip bluefox $
> 
> A few bugs.  It loses some pages once in a while and apps segfault once 
> in a while.  We'll say that by Microsoft standards, it's stable.  Quite a 
> long way to go before it's safe for production systems.
> 
> I'd recommend single or double page size with WKdm (btw I hacked at that
> a little. . . looked like you weren't allocating enough ram.  I MAY have
> overdone some of it though, because i couldn't tell from the code what 2
> of the arrays were supposed to be size-wise).  Also, try out the no
> spin_lock() code (under Verbose Options) and the boot-up sanity check
> (look at it with dmesg).  That sanity check has caught a few typos for me
> that probably would have destroyed my system (fortunately it's just my
> laptop I'm testing on).
>                                                                                                                                                 
>                                                                                                                                                 
>                                                                                                                                                 
> HACKISH CODE
>                                                                                                                                                 
> Most of my code is VERY hackish.  I don't know how to make SURE that the
> algorithm_idx is maintained for each piece of compressed data.  I don't
> know how to alter the meta data in the swap pages.  Thus, I just
> constantly set the page->algorithm_idx and fragment->algorithm_idx to
> default_algorithm_idx whenever they're looked at.  This means that the
> algorithms can't be changed or anything!  Until these hacks are removed,
> there will be no altering of algorithms while running.  I don't
> understand the code enough to make it safe to remove all my hacks.
> 
> First thing's first, go into comp_cache.h and find the 
> comp_cache_page_metadata structure on lines 275-280.  When reading meta 
> data, instead of doing crap like:
> 
> fragment_index = meta_data_offset + 4;
> size = meta_data_offset;
> offset = meta_data_offset + 2;
> 
> You should have crap like the following:
> 
> meta_data = meta_data_offset;
> fragment_index = meta_data->fragment;
> size = meta_data->size;
> offset = meta_data->offset;
> 
> Take a look at proc.c lines 563 to 593 and think about this.  Instead of 
> incrimenting by 8 you could incriment by sizeof(struct 
> comp_cache_page_metadata).  Thus it would be easy to modify the meta data 
> in the page.  It'd also be easier to understand/grok the code.
> 
> I'm not fixing the meta data thing.  Castro can, or someone else can fix 
> it for him.  I'm done with this at least until that gets taken care of.  
> Either way though, don't rely on me to clean up after myself; while 
> you're in there fixing that you should also clean up my bad/dirty hacks.  
> Most of them are marked.  Just read through proc.c from top to bottom.
> 
> 
> 
> UNHANDLED BUSY SIGNALS
> 
> Two or three of the functions use the busy signal system I outlined
> earlier to be SMP/Pre-empt safe.  When they're busy, they by some method 
> return a busy signal.  One of them does it differently and that should be 
> rewritten to not be dumb.  Other than that, none of the calls really 
> handle the busy signal.  They should wait 200 mS, then try again.  After 
> a few tries (maybe 5?  1 whole second is long enough!) they should just 
> exit with a failed status.
> 
> 
> 
> SMP
> 
> Try this code out on SMP if you've got a spare system.  It's buggy, so
> YES it will cause quite a few hickups from lost pages, so DO NOT USE ON
> SYSTEM WITH CRITICAL DATA!!!  If you enable SMP it will force-enable the
> spin_lock() free code.  I'm interested in knowing if this works on SMP 
> now.  The following changes have been made:
> 
> - No spin_lock() calls.  Functions are safe to call multiple times.
> - Separate data for all buffers and algorithms.  Any algorithm can be run 
>   multiple times in parallel to compress and decompress, only limited by 
>   the amount of physical ram.
> 
> I haven't checked for static variables in all functions so be wary of 
> that!  It may not yet be SMP safe but I think it is.
> 
> --Bluefox Icy