linuxcompressed-devel — Linux Compressed Cache Development Mailing List

Re: [lc-devel] Patch 2.4.18-0.23pre8 problem

[Rodrigo S. de C. <rc...@im...>]

Hi Patrick,

On Thu, Jul 04, 2002 at 05:15:23PM +0800, Patrick Lim wrote:
>    I've tried to patch the pre8 into 2.4.18 kernel but fail. Some of
> the files cannot be saved.

I tried it now and indeed there are some problems. Some forgotten
printks() in vanilla files are the cause of these rejects. I rediffed
the patch and now it is supposed to apply.

> Do you have any idea when is this going to fixes?

Already fixed. It's available on the website.

> Btw, I've look at the stats and it really amazed me. I can't wait to
> get the pre8 to work.

Great. If you have any opinions, suggestions, etc, tell us.

> p/s: I think your website status in Source Forge should be put Beta
> rather than Alpha.

That's something I should have changed sometime ago. Thanks for the
suggestion.

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

[lc-devel] Patch 2.4.18-0.23pre8 problem

[Patrick L. <li...@ho...>]

Hi,
    I've tried to patch the pre8 into 2.4.18 kernel but fail. Some of the 
files cannot be saved.

Do you have any idea when is this going to fixes?
Btw, I've look at the stats and it really amazed me. I can't wait to get the 
pre8 to work.

Keep up the good job.

p/s: I think your website status in Source Forge should be put Beta rather 
than Alpha.
By this way, it will convience many more people to try and to give feedback 
such as bugs and motivation. It will speed things up a little bit. I've 
asked many people around. Most of them heard the word 'Alpha', there're 
disappointed no matter how good the software is.



_________________________________________________________________
MSN Photos is the easiest way to share and print your photos: 
http://photos.msn.com/support/worldwide.aspx

[lc-devel] [PATCH] 2.4.18-0.23pre8

[Rodrigo S. de C. <rc...@im...>]

Let's hope this is the last one before 0.23 final. This patch features
lots of bug fixes as well as tweakings to improve compressed cache
performance. It also adds an option to resize compressed cache on
demand. This means that compressed cache will only grow when actually
needed and as soon as it is unneeded, it will shrink. The "compsize="
kernel parameter, in this case, indicates the maximum size of the
compressed cache.

Details:
http://sourceforge.net/project/shownotes.php?release_id=79353

Download
http://prdownloads.sourceforge.net/linuxcompressed/patch-comp-cache-2.4.18-0.23pre8.bz2?download

Direct Link:
http://west.dl.sourceforge.net/sourceforge/linuxcompressed/patch-comp-cache-2.4.18-0.23pre8.bz2

-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Compressed Cache v0.23pre6

[David C. <dav...@sh...>]

Rodrigo Souza de Castro wrote:

>On Sun, Jun 16, 2002 at 11:23:08PM +0800, David Chow wrote:
>  
>
>>I also support some problem in the 0.23pre4 page cache support code,
>>I have my own filesystem and experience page cache oops() after
>>using dirty shared mmap pages (calling to writepage() fs methods). 
>>    
>>
>
>Could you send me a decoded oops?
>
>  
>
>>I think it is wise to take out the pagecache support code, at least
>>a compile time flag to turn on/off . This will simply way of
>>debugging.
>>    
>>
>
>Page cache support is a configuration option. By the way, it is
>disabled by default.
>
>Best regards,
>  
>
I will try out the new patches before send you the bugs, I will try my 
best to trace them for you.

regards,
David

[lc-devel] [PATCH] 2.4.18-0.23pre7

[Rodrigo S. de C. <rc...@im...>]

New patch available. A simple patch that fixes lots of bugs, most of
them reported by Marc-Christian Petersen (thanks for the help). Now it
is not supposed to freeze nor end up in an oops (like reported on our
development mailing list).

Notes:
http://sourceforge.net/project/shownotes.php?release_id=51703

Download:
http://prdownloads.sourceforge.net/linuxcompressed/patch-comp-cache-2.4.18-0.23pre7.bz2?download

Download (direct link):
http://west.dl.sourceforge.net/sourceforge/linuxcompressed/patch-comp-cache-2.4.18-0.23pre7.bz2

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Compressed Cache v0.23pre6

[Rodrigo S. de C. <rc...@im...>]

On Sun, Jun 16, 2002 at 11:23:08PM +0800, David Chow wrote:
> I also support some problem in the 0.23pre4 page cache support code,
> I have my own filesystem and experience page cache oops() after
> using dirty shared mmap pages (calling to writepage() fs methods). 

Could you send me a decoded oops?

> I think it is wise to take out the pagecache support code, at least
> a compile time flag to turn on/off . This will simply way of
> debugging.

Page cache support is a configuration option. By the way, it is
disabled by default.

Best regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Compressed Cache v0.23pre6

[Rodrigo S. de C. <rc...@im...>]

Hi Marc-Christian,

On Fri, Jun 14, 2002 at 03:20:32PM +0200, Marc-Christian Petersen wrote:
> Args, now have a look. Thats the first oops i got, before there were just freezes.
> Just started up xwindows and got the following:

Could you try the attached patch to see if it makes any difference? I
tried the kernel with this patch and it seemed stable. I ran KDE, it
started swapping out (256Mb RAM), no freeze whatsoever.

It's for 2.4.18-0.23pre6.

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Compressed Cache v0.23pre6

[David C. <dav...@sh...>]

Rodrigo Souza de Castro wrote:

>On Fri, Jun 14, 2002 at 03:20:32PM +0200, Marc-Christian Petersen wrote:
>  
>
>>On Fri, 14 Jun 2002, Rodrigo Souza de Castro wrote:
>>    
>>
>>>Did you compile with page cache support?
>>>      
>>>
>>sorry, forgot that in my previous mail. No, without Page cache
>>support.
>>    
>>
>
>Rats, I haven't tested the patch without page cache support before
>releasing! Sorry for that.
>
>  
>
>>Args, now have a look. Thats the first oops i got, before there were just freezes.
>>Just started up xwindows and got the following:
>>    
>>
>
>Nice you sent the oops. Doesn't seem to make much sense, but I will
>check that!
>
>  
>
>>Jun 14 15:00:05 codeman kernel: Call Trace:
>>[grow_comp_cache+191/300] [try_to_free_pages+77/104]
>>[balance_classzone+119/588] [__alloc_pages+262/356]
>>[_alloc_pages+22/24] 
>>Jun 14 15:00:05 codeman kernel:    [do_anonymous_page+76/224]
>>[do_no_page+62/412] [handle_mm_fault+93/240]
>>[do_page_fault+455/1257] [do_page_fault+0/1257]
>>[update_wall_time+11/60] 
>>Jun 14 15:00:05 codeman kernel:    [timer_bh+46/968]
>>[bh_action+38/116] [tasklet_hi_action+93/128] [do_softirq+109/216]
>>[do_IRQ+214 232] [error_code+52/64] 
>>    
>>
>
>Best regards,
>  
>
I also support some problem in the 0.23pre4 page cache support code, I 
have my own filesystem and experience page cache oops() after using 
dirty shared mmap pages (calling to writepage() fs methods). I think it 
is wise to take out the pagecache support code, at least a compile time 
flag to turn on/off . This will simply way of debugging.

regards,
David

Re: [lc-devel] Compressed Cache v0.23pre6

[Rodrigo S. de C. <rc...@im...>]

On Fri, Jun 14, 2002 at 03:20:32PM +0200, Marc-Christian Petersen wrote:
> On Fri, 14 Jun 2002, Rodrigo Souza de Castro wrote:
> > Did you compile with page cache support?
>
> sorry, forgot that in my previous mail. No, without Page cache
> support.

Rats, I haven't tested the patch without page cache support before
releasing! Sorry for that.

> Args, now have a look. Thats the first oops i got, before there were just freezes.
> Just started up xwindows and got the following:

Nice you sent the oops. Doesn't seem to make much sense, but I will
check that!

> Jun 14 15:00:05 codeman kernel: Call Trace:
> [grow_comp_cache+191/300] [try_to_free_pages+77/104]
> [balance_classzone+119/588] [__alloc_pages+262/356]
> [_alloc_pages+22/24] 
> Jun 14 15:00:05 codeman kernel:    [do_anonymous_page+76/224]
> [do_no_page+62/412] [handle_mm_fault+93/240]
> [do_page_fault+455/1257] [do_page_fault+0/1257]
> [update_wall_time+11/60] 
> Jun 14 15:00:05 codeman kernel:    [timer_bh+46/968]
> [bh_action+38/116] [tasklet_hi_action+93/128] [do_softirq+109/216]
> [do_IRQ+214 232] [error_code+52/64] 

Best regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Compressed Cache v0.23pre6

[Marc-Christian P. <m....@gm...>]

On Fri, 14 Jun 2002, Rodrigo Souza de Castro wrote:

Hi Rodrigo,

>> again, testing pre6 of your patch :) ...
> Thank you! :-)=20
np :)

> I want to make this version stable in order to release 0.23 final.
i believe you, but that doesn't look so ;|

>> Unfortunately i must say, pre6 isn't stable as pre4 was ... Following
>> scenario:
> I have already run some tests, but not exactly those mentioned below, l=
ike
> interactive tests. I ran dbench, mmap, fillmem and many times kernel
> compilation. Anyway, I will have a look at those tests to check if I ca=
n
> reproduce the hang. Wait for a feedback.
ok, thanks alot.

> Could you try reproducing without preempt? I don't know very well what =
sort=20
> of changes the preempt patch makes, but I'd like to know if it still
> freezes without it.
Yes, i will do so now, but i really don't think the behaviour will be dif=
ferent. Hang on ...=20

>> Any other informations i can submit to help hunting that bug?
> Did you compile with page cache support?
sorry, forgot that in my previous mail. No, without Page cache support.

Args, now have a look. Thats the first oops i got, before there were just=
 freezes.
Just started up xwindows and got the following:

Jun 14 15:00:00 codeman kernel: [drm] Initialized r128 2.2.0 20020121 on =
minor 0
Jun 14 15:00:05 codeman kernel: divide error: 0000
Jun 14 15:00:05 codeman kernel: CPU:    0
Jun 14 15:00:05 codeman kernel: EIP:    0010:[comp_cache_fix_watermarks+2=
9/92]    Not tainted
Jun 14 15:00:05 codeman kernel: EFLAGS: 00013246
Jun 14 15:00:05 codeman kernel: eax: 0000cff0   ebx: c02a6168   ecx: 0000=
2000   edx: 00000000
Jun 14 15:00:05 codeman kernel: esi: 0000cff0   edi: c02a6168   ebp: c02a=
6168   esp: cd72de0c
Jun 14 15:00:05 codeman kernel: ds: 0018   es: 0018   ss: 0018
Jun 14 15:00:05 codeman kernel: Process XFree86 (pid: 11189, stackpage=3D=
cd72d000)
Jun 14 15:00:05 codeman kernel: Stack: 00000008 000001d2 c02a6168 c014180=
3 00002000 00000006 000001d2 c02a6168=20
Jun 14 15:00:05 codeman kernel:        c02a6168 c3dc93e0 c0138abd c02a616=
8 00000010 cd72c000 00000000 000001d2=20
Jun 14 15:00:05 codeman kernel:        c0139507 c02a62e4 00000120 0000001=
0 00000000 00000000 00000000 c01397e2=20
Jun 14 15:00:05 codeman kernel: Call Trace: [grow_comp_cache+191/300] [tr=
y_to_free_pages+77/104] [balance_classzone+119/588] [__allo
c_pages+262/356] [_alloc_pages+22/24]=20
Jun 14 15:00:05 codeman kernel:    [do_anonymous_page+76/224] [do_no_page=
+62/412] [handle_mm_fault+93/240] [do_page_fault+455/1257]=20
[do_page_fault+0/1257] [update_wall_time+11/60]=20
Jun 14 15:00:05 codeman kernel:    [timer_bh+46/968] [bh_action+38/116] [=
tasklet_hi_action+93/128] [do_softirq+109/216] [do_IRQ+214/
232] [error_code+52/64]=20
Jun 14 15:00:05 codeman kernel:=20
Jun 14 15:00:05 codeman kernel: Code: f7 35 38 e9 2d c0 89 c6 a1 44 e9 2d=
 c0 39 c6 73 04 89 c6 eb=20
Jun 14 15:00:05 codeman kernel:  divide error: 0000
Jun 14 15:00:05 codeman kernel: CPU:    0
Jun 14 15:00:05 codeman kernel: EIP:    0010:[comp_cache_fix_watermarks+2=
9/92]    Not tainted
Jun 14 15:00:05 codeman kernel: EFLAGS: 00013246
Jun 14 15:00:05 codeman kernel: eax: 0000cff0   ebx: c02a6168   ecx: 0000=
2000   edx: 00000000
Jun 14 15:00:05 codeman kernel: esi: 0000cff0   edi: c02a6168   ebp: 0000=
0000   esp: c2fb7f6c
Jun 14 15:00:05 codeman kernel: ds: 0018   es: 0018   ss: 0018
Jun 14 15:00:05 codeman kernel: Process kswapd (pid: 4, stackpage=3Dc2fb7=
000)
Jun 14 15:00:05 codeman kernel: Stack: 00000010 000001d0 c02a6168 c014180=
3 00002000 00000006 000001d0 c02a6168=20
Jun 14 15:00:05 codeman kernel:        c02a6168 00000000 c0138abd c02a616=
8 00000010 c02a6168 00000001 c2fb6000=20
Jun 14 15:00:05 codeman kernel:        c0138b3b c02a60c0 00000000 c2fb625=
1 0008e000 c0138b9e c02a60c0 c2fb6000=20
Jun 14 15:00:05 codeman kernel: Call Trace: [grow_comp_cache+191/300] [tr=
y_to_free_pages+77/104] [kswapd_balance_pgdat+55/132] [kswa
pd_balance+22/44] [kswapd+153/188]=20
Jun 14 15:00:05 codeman kernel:    [kernel_thread+40/56]=20
Jun 14 15:00:05 codeman kernel:=20
Jun 14 15:00:05 codeman kernel: Code: f7 35 38 e9 2d c0 89 c6 a1 44 e9 2d=
 c0 39 c6 73 04 89 c6 eb=20

This was 15mb of free RAM and no swap usage. System is not freezed,
but X-Console is totally freezed. Then i tried to compile a new kernel, a=
fter
some doing of "make dep" system was totally freezed w/o any oops or panic=
s.


ksymoops of the above:
----------------------




--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824 080A 569D E2E3 DB44 1A16
Key available at www.keyserver.net. Encrypted e-mail preferred.

Re: [lc-devel] Compressed Cache v0.23pre6

[Rodrigo S. de C. <rc...@im...>]

On Fri, Jun 14, 2002 at 02:13:07PM +0200, Marc-Christian Petersen wrote:
> again, testing pre6 of your patch :) ...

Thank you! :-) 

I want to make this version stable in order to release 0.23 final.

> Unfortunately i must say, pre6 isn't stable as pre4 was ... Following
> scenario:

I have already run some tests, but not exactly those mentioned below, like
interactive tests. I ran dbench, mmap, fillmem and many times kernel
compilation. Anyway, I will have a look at those tests to check if I can
reproduce the hang. Wait for a feedback.

> :-(( And yes, its reproduceable ... 

That's nice.

> comp_cache_size is 8192
> Kernel 2.4.18-wolk3.5rc1
> - with Preempt

Could you try reproducing without preempt? I don't know very well what sort of
changes the preempt patch makes, but I'd like to know if it still freezes
without it.

> Any other informations i can submit to help hunting that bug?

Did you compile with page cache support?

Best regards,
-- 
Rodrigo S. de Castro <rc...@im...>

[lc-devel] Compressed Cache v0.23pre6

[Marc-Christian P. <m....@gm...>]

Hi Rodrigo,

again, testing pre6 of your patch :) ... Unfortunately i must say, pre6 i=
sn't=20
stable as pre4 was ... Following scenario:

Running XWindows v4.2.0
Running Mozilla v1.0
 - surfing to freshmeat.net and read a bit
 - surf to kernel.org
 - ...
 - wait until Memory (physical ram only) is full
   yes, NO swap is used!! 256MB swapspace is available
 - System stops reacting and freezes!
 - Only Sysrq-b works, nothing else with Sysrq

:-(( And yes, its reproduceable ... Also without websurfin. Using GIMP, o=
pen=20
some images until ram is full (yes, not swap is used at all) and system=20
freezes.

comp_cache_size is 8192
Kernel 2.4.18-wolk3.5rc1
- with Preempt

256MB ram and 256mb swap space. pre4 never showed such a freeze and also =
swap=20
was used earlier than with pre6. I didn't had the time to test pre5 :|

Any other informations i can submit to help hunting that bug?


--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824 080A 569D E2E3 DB44 1A16
Key available at www.keyserver.net. Encrypted e-mail preferred.

Re: [lc-devel] [PATCH] 2.4.18-0.23pre5

[Rodrigo S. de C. <rc...@im...>]

On Thu, May 30, 2002 at 10:02:18AM -0300, Rodrigo Souza de Castro wrote:
> In particular, a bug that was hit under UML which could cause FS
> corruption and a bug in swap buffers management that could hang the
> machine.

Sorry about that, but that was a wrong fix.

I noticed that there had been an increase in IO performed when
compressed cache was enabled and after digging into the problem, the
culprit was this "fix" to the FS corruption bug under UML. I learned
better how buffers work and can say that this fix just papers over
that UML bug by forcing extra disk reads.

This morning I even introduced a checksum for compressed data to
doublecheck if we were returning sane data. My conclusion: there is no
bug in our code regarding this IO corruption problem under UML (that's
likely the reason we can't hit this corrupton bug on a real machine).

I will back out this "fix", so we cannot use UML until it gets
fixed. Although I can't reproduce this bug on vanilla under UML, I've
already reported a reproducible bug (running dbench) that corrupts UML
filesystem, even on a vanilla kernel.

-- 
Rodrigo S. de Castro <rc...@im...>

[lc-devel] [PATCH] 2.4.18-0.23pre5

[Rodrigo S. de C. <rc...@im...>]

This patch features the support for LZO compression algorithm added
back.

There's also a new /proc entry showing what's going on inside the
compressed cache. This entry (comp_cache_hist) displays the free space
histogram for all compressed cache pages, besides how many fragments
are stored in these pages. Very useful data.

Some bugs were fixed. In particular, a bug that was hit under UML
which could cause FS corruption and a bug in swap buffers management
that could hang the machine. About User Mode Linux, its author, Jeff
Dike, fixed a bug that I reported concerning swap addresses, so now we
can use compressed cache under UML (use version 2.4.18-30um or later).

This patch also features our kernel parameter (compsize=) which
accepts input in the format "mem=" does (for example, you can append
"compsize=16M"). Note, however, that /proc/sys/vm/comp_cache/size is
still a number of memory pages.

More details:
http://sourceforge.net/project/shownotes.php?release_id=48824

Download:
http://prdownloads.sourceforge.net/linuxcompressed/patch-comp-cache-2.4.18-0.23pre5.bz2?download
- direct link
http://west.dl.sourceforge.net/sourceforge/linuxcompressed/patch-comp-cache-2.4.18-0.23pre5.bz2

-- 
Rodrigo S. de Castro <rc...@im...>

[lc-devel] Re: [uml-devel] [PATCH] Fix support for > 16 swaps

[Jeff D. <jd...@ka...>]

rc...@im... said:
> First I thought about changing pte_mkuptodate() to clean _PAGE_NEWPROT
> bit only for present ptes, since non-present ptes don't have this bit
> on (do they?). 

No, they don't.  There are two bits which have to be valid for all ptes, 
present (obviously), and new_page (because UML needs to be able to tell when
something has just been swapped out so it can be unmapped).  new_prot has no
meaning for a swapped pte.

> However that's not enough because this non-present pte
> will be taken as having new protection (pte_newprot() == 1) and we
> will try to protect a wrong pte in fix_range(), panicing. Thus I think
> we actually can't have a _PAGE_NEWPROT bit that isn't exclusive for
> swap types bit.

Then we fix pte_newprot too.  Incididentally, I got set_pte right, but missed
pte_newprot and pte_mkuptodate.

> I'd like to know what do you think about backing out this PAGE_NEWPROT
> change and set it back as 0x100 (or even 0x200).

Not much.  That just papers over the bug without fixing it.

This is fixed in my pool and will be in the next patch.  Nice catch, BTW.

				Jeff

[lc-devel] [PATCH] Fix support for > 16 swaps

[Rodrigo S. de C. <rc...@im...>]

Hi Jeff,

I was running 2.4.18-16um until last days and given that I use UML
more intensively from now on, I upgraded my tree to 2.4.18-29um
today. Unfortunately I noticed that some change since 16um broke my
code.

Checking all versions, I first noticed that my code was broken in
19um, when you changed some code that handles pte. In particular, in
include/asm-um/pgtable.h

--- pgtable.h.18        Thu May 23 16:46:52 2002
+++ pgtable.h.19        Thu May 23 16:46:42 2002
@@ -78,23 +78,17 @@
 #define VMALLOC_VMADDR(x) ((unsigned long)(x))
 #define VMALLOC_END    (end_vm)

-/*
- * The 4MB page is guessing..  Detailed in the infamous "Chapter H"
- * of the Pentium details, but assuming intel did the straightforward
- * thing, this bit set in the page directory entry just means that
- * the page directory entry points directly to a 4MB-aligned block of
- * memory.
- */
 #define _PAGE_PRESENT  0x001
 #define _PAGE_NEWPAGE  0x002
 #define _PAGE_PROTNONE 0x004   /* If not present */
 #define _PAGE_RW       0x008
 #define _PAGE_USER     0x010
-#define _PAGE_PCD       0x020
-#define _PAGE_ACCESSED 0x040
-#define _PAGE_DIRTY    0x080
-#define _PAGE_NEWPROT   0x100
+#define _PAGE_ACCESSED 0x020
+#define _PAGE_DIRTY    0x040
+#define _PAGE_NEWPROT   0x080

[snip]

From the excerpt above, we can notice that you changed _PAGE_NEWPROT
from 0x100 to 0x080. Incidentally, that change's broken my code. Let
me explain. I use swap type number NUM_SWAPFILES (in my case 31) as a
virtual swap device. In fix_range() and in pte_to_swp_entry() you call
pte_mkuptodate() macro which will clean _PAGE_NEWPROT bit for the pte
we are handling. In the case _PAGE_NEWPROT is 0x080 (unlike 0x100),
pte_mkuptodate() will clean a bit that is used for my swap type, so I
end up having a swap type number 15 (not 31 like as first set). It's
obvious that swap code won't be able to recognize my swap addresses
any longer.

First I thought about changing pte_mkuptodate() to clean _PAGE_NEWPROT
bit only for present ptes, since non-present ptes don't have this bit
on (do they?). However that's not enough because this non-present pte
will be taken as having new protection (pte_newprot() == 1) and we
will try to protect a wrong pte in fix_range(), panicing. Thus I think
we actually can't have a _PAGE_NEWPROT bit that isn't exclusive for
swap types bit.

I'd like to know what do you think about backing out this PAGE_NEWPROT
change and set it back as 0x100 (or even 0x200). This won't break swap
addresses and will allow us to have safely 32 swap devices (or 64 if
set to 0x200). Recall that 32 is the default maximum number of swap
devices in vanilla. 

I am sending attached a 2-liner patch for 29um that set the value back
to 0x100.

Best regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Compressed Cache v0.23pre4

[Marc-Christian P. <m....@gm...>]

On Thursday 23 May 2002 14:37, Rodrigo Souza de Castro wrote:

Hi Rodrigo,

> Yes, I guess so. It's great to know there are some people besides me
> testing the patch :-)
:-)

> What's your system configuration? Do you have page cache support
> enabled? What's the size you are setting to the compressed cache?
Its a Celeron 800MHz, UDMA66 IDE Disk, 256MB RAM, running Debian SID.
Compressed Cache settings are:

Compressed Cache: 0.23pre4
Compressed Cache: initial size
Compressed Cache:  6552 pages =3D 26208KiB
Compressed Cache: page watermarks (normal zone)
Compressed Cache:  (255, 510, 765) -> (255, 510, 765)
Compressed Cache: hash table
Compressed Cache:  fragment   (32768 entries =3D 131072B)
Compressed Cache:  free space (42 entries =3D 168B)

> > and even much smoother with Lock-Break from Preemptible
> > Kernel. version 0.23pre4 seems to work fine with it, prior versions
> > freezes my mashine.
> So doesn't it freeze with Lock-Break when untarring a kernel any
> longer? Even with page cache support enabled?
as i told in the other email, there was a freeze some hours ago. Now i tr=
y=20
this with out page compressing.

> [ - SNIP: explaination size of compressed cache - ]
thnx, i will play with it a bit around :)

> > Hint: You should readd config help that the size is still tuneable
> > via /proc, not only with adding a boot param to the bootmanager :)
> Nice tip, I will do that.
:-) great!

> > Thanks for your great work Rodrigo. If you need help, let me know!
> Thanks for your help.
you are welcome!


--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [lc-devel] pre4

[Marc-Christian P. <m....@gm...>]

On Thursday 23 May 2002 15:01, Rodrigo Souza de Castro wrote:

Hi Rodrigo,

> > I've noticed, with "Page compressing" the system gets slower, and
> > stopps reacting for less than a second if you do high i/o.
> Is there any specific IO bound program you run that can make this
> slowdown more noticeable? I'd like to fix this behaviour.
i think the best program(s) to reproduce is "tar xzpf linux-2.4.18.tar.gz=
; rm=20
-rf linux-2.4.18; tar xzpf linux-2.4.18.tar.gz"

Its a Celeron 800MHz, UDMA66 IDE Disk, 256MB RAM, running Debian SID.

If i use the above program calls, Mouse in X Windows stopps reacting, eve=
n=20
Midnight Commander stops reacting for input, screen needs more time to op=
en=20
up a new session (Ctrl-A C) etc...

> > With Lockbreak, not part of Compressed Cache, there are still some
> > stopps, but i think, Rodrigo has this stuff on his ToDo-List.
> Let me get this straight. Do you still have problems when running
> compressed cache with lock-break enabled? If you do, even when the
> page cache support is disabled?
Yes, there are still some small stops if high i/o, even with page compres=
sion=20
disabled. Also, i've noticed some hours ago, my system freezes, this is w=
ith=20
Compressed Cache, Page Compression and Preempt/LockBreak. After that i've=
=20
compiled it again w/o Page compression, its up now for 3h 19mins, lemme s=
ee=20
if this works better. But i think, preempt/lockbreak stuff isn't a bad id=
ea=20
to integrate in the compressed cache code :)

> I'd set to something between 15% and 25% of your total memory size
> since we don't have adaptivity yet.
ok, i will set it :)


--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [lc-devel] pre4

[Rodrigo S. de C. <rc...@im...>]

On Thu, May 23, 2002 at 09:47:41AM -0300, Rodrigo Souza de Castro wrote:
> In both cases, the parameter is the number of memory pages (4096
> bytes on i386). I want to make it soon similar to "mem=" kernel
> parameter where you can have the input as megabytes instead of
> memory pages.

I already changed the code to accept the input the way mem= does and
the code is already in CVS (0.23pre5 will have it). Thus, we will be
able to enter values like compsize=16M, 1024K or even 1G (although the
compressed cache cannot be that large) without having to convert the
value into memory pages.

-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] pre4

[Rodrigo S. de C. <rc...@im...>]

On Thu, May 23, 2002 at 12:56:46PM +0200, Marc-Christian Petersen wrote:
> On Thursday 23 May 2002 03:55, Patrick wrote:
>
> > I think the setup procedures have been changed.
> > Can you give me the proper way of selecting the compress cache size and how
> > to compile it?
>
> Compiling doesn't really matter. You can give the size via bootparam
> "compsize=N" or via /proc/sys/vm/comp_cache/size

That's right. Recall that "N" is the number of memory pages to be used
by compressed cache.

> I've noticed, with "Page compressing" the system gets slower, and
> stopps reacting for less than a second if you do high i/o.

Is there any specific IO bound program you run that can make this
slowdown more noticeable? I'd like to fix this behaviour.

> With Lockbreak, not part of Compressed Cache, there are still some
> stopps, but i think, Rodrigo has this stuff on his ToDo-List.

Let me get this straight. Do you still have problems when running
compressed cache with lock-break enabled? If you do, even when the
page cache support is disabled?

> But, anyway, i cannot tell you the exact size of what it should be
> to get maximum compression and also maximum speed. I've asked this
> yesterday too on this mailinglist. :) Let's wait for Rodrigo.

I'd set to something between 15% and 25% of your total memory size
since we don't have adaptivity yet.

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] pre4

[Rodrigo S. de C. <rc...@im...>]

On Thu, May 23, 2002 at 09:55:29AM +0800, Patrick wrote:
> I think the setup procedures have been changed.
> Can you give me the proper way of selecting the compress cache size

Initial compressed cache size can only be set by kernel parameter
now. During your uptime, you can set compressed cache size by echoing
a new value to the file 

/proc/sys/vm/comp_cache/size

In both cases, the parameter is the number of memory pages (4096 bytes
on i386). I want to make it soon similar to "mem=" kernel parameter
where you can have the input as megabytes instead of memory pages.

Note: there's no way to set the compressed cache size in kernel
compilation process anymore.

> and how to compile it?

I updated yesterday some installation instruction on our main page[1],
so take a look at the "How to apply the Patches" section. If you still
have troubles, please send us an email.

[1] http://linuxcompressed.sourceforge.net/

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Compressed Cache v0.23pre4

[Rodrigo S. de C. <rc...@im...>]

On Wed, May 22, 2002 at 08:59:45PM +0200, Marc-Christian Petersen wrote:
> and again, i think, i am the first man, beneath you, who tests this
> version :)

Yes, I guess so. It's great to know there are some people besides me
testing the patch :-)

> This code went into WOLK 3.4 Final version. I've tested it, runs
> fine till now. And i must say, my System is running very very smooth
> with Compressed Cache activated :) ...

What's your system configuration? Do you have page cache support
enabled? What's the size you are setting to the compressed cache?

> and even much smoother with Lock-Break from Preemptible
> Kernel. version 0.23pre4 seems to work fine with it, prior versions
> freezes my mashine.

So doesn't it freeze with Lock-Break when untarring a kernel any
longer? Even with page cache support enabled?

> A question for understanding: What should be the Size of Compressed
> Cache? Is there a way to calculate it by hand if, let us say, RAM is
> 256MB and also SWAP is 256MB?

I don't think there will have one optimal compressed cache size during
all the time your system is up. That's because your system will have
various memory needs and memory references throughout this time and a
single compressed cache size is unlikely to be the optimal for all
these scenarios.

Anyway, while we don't have adaptivity working reasonably well for
compressed cache, you have to pick a size to be able to have
compressed cache benefits (hopefully) by now. I would choose something
like 15% to 25% of your memory size, because you wouldn't have too
much aditional cost since it's not that large and also it isn't so
small that can still provide some benefit, if that's the case. That
way, in your case, I would set the compressed cache size to something
like 40Mb and 64Mb.

> Hint: You should readd config help that the size is still tuneable
> via /proc, not only with adding a boot param to the bootmanager :)

Nice tip, I will do that.

> Thanks for your great work Rodrigo. If you need help, let me know!

Thanks for your help.

Best regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] pre4

[Marc-Christian P. <m....@gm...>]

On Thursday 23 May 2002 03:55, Patrick wrote:

Hi Patrick,

> This is my second attempt since pre1.
Nice. pre2 and pre3 had some unusual behaveness. I think its the right ti=
me=20
you test it again :)

> I think the setup procedures have been changed.
> Can you give me the proper way of selecting the compress cache size and=
 how
> to compile it?
Compiling doesn't really matter. You can give the size via bootparam=20
"compsize=3DN" or via /proc/sys/vm/comp_cache/size

I've noticed, with "Page compressing" the system gets slower, and stopps=20
reacting for less than a second if you do high i/o. Without it, no stopps=
=20
occur. With Lockbreak, not part of Compressed Cache, there are still some=
=20
stopps, but i think, Rodrigo has this stuff on his ToDo-List.

But, anyway, i cannot tell you the exact size of what it should be to get=
=20
maximum compression and also maximum speed. I've asked this yesterday too=
 on=20
this mailinglist. :) Let's wait for Rodrigo.

--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

[lc-devel] pre4

[Patrick <pat...@my...>]

Hi,
    This is my second attempt since pre1. 
I think the setup procedures have been changed.
Can you give me the proper way of selecting the compress cache size and how to compile it?

Thanks 

[lc-devel] Compressed Cache v0.23pre4

[Marc-Christian P. <m....@gm...>]

Hi Rodrigo,

and again, i think, i am the first man, beneath you, who tests this versi=
on :)=20
=2E.. This code went into WOLK 3.4 Final version. I've tested it, runs fi=
ne=20
till now. And i must say, my System is running very very smooth with=20
Compressed Cache activated :) ... and even much smoother with Lock-Break =
from=20
Preemptible Kernel. version 0.23pre4 seems to work fine with it, prior=20
versions freezes my mashine.

A question for understanding: What should be the Size of Compressed Cache=
 ? Is=20
there a way to calculate it by hand if, let us say, RAM is 256MB and also=
=20
SWAP is 256MB ?

Hint: You should readd config help that the size is still tuneable via /p=
roc,=20
not only with adding a boot param to the bootmanager :)

Thanks for your great work Rodrigo. If you need help, let me know!

--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.