linuxcompressed-devel — Linux Compressed Cache Development Mailing List

Re: [lc-devel] Re: Compressed Cache and Preempt/Lockbreak

[Rodrigo S. de C. <rc...@im...>]

On Thu, May 16, 2002 at 07:17:53PM +0200, Marc-Christian Petersen wrote:
> On Thursday 16 May 2002 15:19, Rodrigo Souza de Castro wrote:
> > Yes, but I am not sure it's stable though. My main bug report
> > regarding stability is a hang that happens with Paolo Ciarrocchi when
> > he runs mmap001 program from memtest suite.
>
> oh, and only with mmap001 ? 

Paolo is the only one which reported this problem with mmap001. I
can't reproduce it here, for example.

> Nothing else is known to breakage?

Not that I know of. 

> > Thanks for your request. Even if I can't fix that freeze now, I'd like
> > to know what may be causing it.
> i can tell you:
> 
> - cd /usr/src
> - rm -rf linux-2.4.18
> - tar xzpf linux-2.4.18.tar.gz
>   System freeze while tar do untargzip the file.

Great, it's on our Todo list too.

> Another question: Why not ALL of your compressed cache code is ifdef
> signed?  Some of your code removes existing code and replace new,
> other only apply if compressed cache is selected.

It's not very nice to add #ifdefs all over the original code and that
procedure it's not advisable in Documentation/CodyingStyle document
either.

Actually there are some function calls that are replaced by compressed
cache functions, but all these functions have #ifdefs in our header
file comp_cache.h. That way, if you have compressed cache disabled,
these new compressed cache functions will be static inline functions
that will call the original ones, not adding overhead.

In case you've noticed something that it's not correctly being
disabled or it's introducing overhead, I'd glad to have a look at it.

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Re: Compressed Cache and Preempt/Lockbreak

[Marc-Christian P. <m....@gm...>]

On Thursday 16 May 2002 15:19, Rodrigo Souza de Castro wrote:

Hi Rodrigo,

> Yes, but I am not sure it's stable though. My main bug report
> regarding stability is a hang that happens with Paolo Ciarrocchi when
> he runs mmap001 program from memtest suite.
oh, and only with mmap001 ? Nothing else is known to breakage?

> Please report any bug you may hit (or submit a bug to the Bug Tracking
> System at Sourceforge).
i will do so.

> I already added this request to our Todo list and I am willing to do
> that as soon as I have some spare time. However I don't know when
> that's going to happen :-)
hmm, then i will tell you when: NOW!

;-))

> Thanks for your request. Even if I can't fix that freeze now, I'd like
> to know what may be causing it.
i can tell you:

- cd /usr/src
- rm -rf linux-2.4.18
- tar xzpf linux-2.4.18.tar.gz
  System freeze while tar do untargzip the file.

Another question: Why not ALL of your compressed cache code is ifdef sign=
ed?
Some of your code removes existing code and replace new, other only apply=
 if=20
compressed cache is selected.

--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

[lc-devel] Re: Compressed Cache and Preempt/Lockbreak

[Rodrigo S. de C. <rc...@im...>]

On Thu, May 16, 2002 at 02:56:47PM +0200, Marc-Christian Petersen wrote:
> noticed you made 0.23pre3 available. I will test it this evening :)

Yes, but I am not sure it's stable though. My main bug report
regarding stability is a hang that happens with Paolo Ciarrocchi when
he runs mmap001 program from memtest suite. 

Please report any bug you may hit (or submit a bug to the Bug Tracking
System at Sourceforge).

> Is there any chance you could also make a patch that is compatible
> with Lockbreak from Robert Love? ... I want, and also many users of
> wolk, to use LockBreak and Compressed Cache at the same time, but
> this is not possible yet, if both selected, reproducable system
> freezes occurs. Especially if high I/O is done to the disk :-(

I already added this request to our Todo list and I am willing to do
that as soon as I have some spare time. However I don't know when
that's going to happen :-)

> Any chance for this? :-) ... I could be your testing pal :)

Thanks for your request. Even if I can't fix that freeze now, I'd like
to know what may be causing it.

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

[lc-devel] Compressed Cache and Preempt/Lockbreak

[Marc-Christian P. <m....@gm...>]

Hi Rodrigo,

noticed you made 0.23pre3 available. I will test it this evening :)

Is there any chance you could also make a patch that is compatible with=20
Lockbreak from Robert Love? ... I want, and also many users of wolk, to u=
se=20
LockBreak and Compressed Cache at the same time, but this is not possible=
=20
yet, if both selected, reproducable system freezes occurs. Especially if =
high=20
I/O is done to the disk :-(

Any chance for this? :-) ... I could be your testing pal :)

--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [lc-devel] 0.23pre2

[Marc-Christian P. <m....@gm...>]

On Tuesday 30 April 2002 00:28, Rodrigo Souza de Castro wrote:

Hi Rodrigo,

> > kernel: Call Trace: [flush_comp_cache+43/164] [do_swap_page+338/384]
> > [handle_mm_fault+112/240] [do_page_faul
> > t+388/1204] [do_page_fault+0/1204]
> > kernel:    [timer_bh+46/680] [bh_action+38/116]
> > [tasklet_hi_action+93/128] [do_softirq+97/200] [do_IRQ+214/2
> > 32] [error_code+52/64]
> Two liner fix already in my pool.
great :)

> Thanks for the report.
no problem!

> > AND ALSO MASSIVE!!!! and i mean MASSIVE failures like this:
> >
> > kernel: init_special_inode: bogus imode (0)
> > EXT3-fs error (device ide0(3,10)): ext3_readdir: bad entry in directo=
ry
> > #2063: rec_len %% 4 !=3D 0 - o
> > ffset=3D0, inode=3D0, rec_len=3D25, name_len=3D0
> >
> > ALOT for every second!!! :-(( ... i will downgrade to 0.23pre1 or eve=
n
> > 0.22final. I have some ext3-fs errors with 0.23pre1 too but i forgot =
to
> > report it to you Rodrigo :(
> I had some troubles with the page cache support in the tests I ran in
> the morning too. So don't worry about reporting the ext3 errors given
> that they will report some corrupted stuff (like the bad entry in the
> directory) that it's unlikely to be much useful to find out what are
> actually the causes.
> These errors are reproducible, aren't they? Is there anything you
> usually do to reproduce them?
yes, they are reproduceable. If i start Unreal Tournament (a game) i get =
this=20
failures, cause the Games is swapping on the hd|swap alot.

Or if i do a lot of surfing/downloading through squid! All is reproduceab=
le!

Kind regards,
      Marc

Re: [lc-devel] Re: Patch 0.23pre2 for 2.4.18

[Marc-Christian P. <m....@gm...>]

On Tuesday 30 April 2002 00:20, Rodrigo Souza de Castro wrote:

Hi Rodrigo,

> You are faster than me. I haven't even announced this new version and
> you've almost included it :-) Anyway, I couldn't answer this earlier
> today, but in the morning, before receiving your email, I had compiled
> 0.23pre2 and already hit some bugs, so I don't think it's advisable
> for you to include 0.23pre2. It's very unstable so far.
yes, i've noticed this too.

> Maybe a 0.23pre3 version that will be released in the next days :-)
wonderfull! I will have a look at it.

> > Anyway, this is great and i will include pre2 in my kernel project
> > wolk :-)
> By the way, thank you for you support and for incluing this patch in
> your kernel project. There many things to improve and to be developed
> yet, and being able to get a feedback from you and your users will be
> an excellent way to make this project better.
You are welcome :) ... I noticed, with 0.23pre2 the system is VERY fast,=20
almost as fast as with preempt/lockbreak. Good work so far!!

Kind regards,
=09Marc

Re: [lc-devel] 0.23pre2

[Rodrigo S. de C. <rc...@im...>]

On Mon, Apr 29, 2002 at 11:40:39PM +0200, Marc-Christian Petersen wrote:
> fs/fs.o: In function `try_to_free_buffers':
> fs/fs.o(.text+0x6fbe): undefined reference to `comp_cache_skip_buffer_freeing'
> make[1]: *** [kallsyms] Error 1
> make[1]: Leaving directory `/usr/src/linux-2.4.18'
> make: *** [vmlinux] Error 2

Whoops. Already in my pool too.

Now let's fix that page cache corruption thing. :-)

--- linuxcompressed/fs/buffer.c~        Mon Apr 22 16:08:18 2002
+++ linuxcompressed/fs/buffer.c Mon Apr 29 19:33:51 2002
@@ -47,6 +47,7 @@
 #include <linux/highmem.h>
 #include <linux/module.h>
 #include <linux/completion.h>
+#include <linux/comp_cache.h>

 #include <asm/uaccess.h>
 #include <asm/io.h>

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] 0.23pre2

[Rodrigo S. de C. <rc...@im...>]

On Mon, Apr 29, 2002 at 11:38:50PM +0200, Marc-Christian Petersen wrote:
> kernel: EIP:    0010:[find_comp_page+24/104]    Tainted: P 
(...)
> kernel: Call Trace: [flush_comp_cache+43/164] [do_swap_page+338/384] 
> [handle_mm_fault+112/240] [do_page_faul
> t+388/1204] [do_page_fault+0/1204] 
> kernel:    [timer_bh+46/680] [bh_action+38/116] [tasklet_hi_action+93/128] 
> [do_softirq+97/200] [do_IRQ+214/2
> 32] [error_code+52/64]

Two liner fix already in my pool.

Thanks for the report.

> AND ALSO MASSIVE!!!! and i mean MASSIVE failures like this:
> 
> kernel: init_special_inode: bogus imode (0)
> EXT3-fs error (device ide0(3,10)): ext3_readdir: bad entry in directory #2063: 
> rec_len %% 4 != 0 - o
> ffset=0, inode=0, rec_len=25, name_len=0
> 
> ALOT for every second!!! :-(( ... i will downgrade to 0.23pre1 or even 
> 0.22final. I have some ext3-fs errors with 0.23pre1 too but i forgot to 
> report it to you Rodrigo :(

I had some troubles with the page cache support in the tests I ran in
the morning too. So don't worry about reporting the ext3 errors given
that they will report some corrupted stuff (like the bad entry in the
directory) that it's unlikely to be much useful to find out what are
actually the causes.

These errors are reproducible, aren't they? Is there anything you
usually do to reproduce them?

Thanks.

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] Re: Patch 0.23pre2 for 2.4.18

[Rodrigo S. de C. <rc...@im...>]

Hi Marc-Christian,

On Mon, Apr 29, 2002 at 08:45:29PM +0200, Marc-Christian Petersen wrote:
> great! :-) ... I am working hard on the next release of WOLK, v3.4, almost 
> done, including 0.23pre1 and what are you doing?!?! You are releasing pre2 
> ;-)))))))) 

You are faster than me. I haven't even announced this new version and
you've almost included it :-) Anyway, I couldn't answer this earlier
today, but in the morning, before receiving your email, I had compiled
0.23pre2 and already hit some bugs, so I don't think it's advisable
for you to include 0.23pre2. It's very unstable so far.

Maybe a 0.23pre3 version that will be released in the next days :-)

> Anyway, this is great and i will include pre2 in my kernel project
> wolk :-)

By the way, thank you for you support and for incluing this patch in
your kernel project. There many things to improve and to be developed
yet, and being able to get a feedback from you and your users will be
an excellent way to make this project better.

Best regards,
-- 
Rodrigo S. de Castro <rc...@im...>

[lc-devel] 0.23pre2

[Marc-Christian P. <m....@gm...>]

Hi again,

even there are compile errors if Compressed Cache is deactivated in the k=
ernel=20
config:

fs/fs.o: In function `try_to_free_buffers':
fs/fs.o(.text+0x6fbe): undefined reference to `comp_cache_skip_buffer_fre=
eing'
make[1]: *** [kallsyms] Error 1
make[1]: Leaving directory `/usr/src/linux-2.4.18'
make: *** [vmlinux] Error 2

--=20
Kind regards
=09Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

[lc-devel] 0.23pre2

[Marc-Christian P. <m....@gm...>]

Hi there,

i was too early to say its great :-(((

kernel: signal 11  sent to (telnet:14009) UID(0) EUID(0), parent=20
(checkservices:17744) UID(0) EUID(0)
kernel: Unable to handle kernel NULL pointer dereference at virtual addre=
ss=20
00000018
kernel:  printing eip:
kernel: c0141d60
kernel: *pde =3D 04f9b067
kernel: *pte =3D 00000000
kernel: Oops: 0000
kernel: CPU:    0
kernel: EIP:    0010:[find_comp_page+24/104]    Tainted: P=20
kernel: EFLAGS: 00210296
kernel: eax: 00000018   ebx: 00000000   ecx: c4d33ec0   edx: 0018bb00
kernel: esi: c4d33ec0   edi: 0018bb00   ebp: fffffffe   esp: c4d33ea0
kernel: ds: 0018   es: 0018   ss: 0018
kernel: Process colorize.pl (pid: 27913, stackpage=3Dc4d33000)
kernel: Stack: 02b20067 c10ac800 fffffffe c4f9a8b8 c014071b 02b20067 c10a=
c800=20
0018bb00=20
kernel:        00000000 c012fe42 0822e174 c5d20220 00000001 c72320e0 0000=
0002=20
c14875a0=20
kernel:        c013012c c5d20220 c72320e0 0822e174 c4f9a8b8 0018bb00 0000=
0001=20
c4d32000=20
kernel: Call Trace: [flush_comp_cache+43/164] [do_swap_page+338/384]=20
[handle_mm_fault+112/240] [do_page_faul
t+388/1204] [do_page_fault+0/1204]=20
kernel:    [timer_bh+46/680] [bh_action+38/116] [tasklet_hi_action+93/128=
]=20
[do_softirq+97/200] [do_IRQ+214/2
32] [error_code+52/64]
kernel:=20
kernel: Code: 39 43 18 75 08 8d 43 20 39 43 20 74 3b 8b 0d 30 d0 3e c0 89=
=20
kernel:  <3>swap_free: Unused swap offset entry 0018bb00
kernel: EXT3-fs warning (device ide0(3,10)): ext3_unlink: Deleting nonexi=
stent=20
file (104064), 0
squid[28447]: Squid Parent: child process 14332 exited due to signal 11
codeman kernel: Unable to handle kernel NULL pointer dereference at virtu=
al=20
address 00000018
kernel:  printing eip:
kernel: *pde =3D 00080067
kernel: *pte =3D 00000000
kernel: Oops: 0000
kernel: CPU:    0
kernel: EIP:    0010:[find_comp_page+24/104]    Tainted: P=20
kernel: EFLAGS: 00010296
kernel: eax: 00000018   ebx: 00000000   ecx: c8b3dec0   edx: 0084b100
kernel: esi: c8b3dec0   edi: 0084b100   ebp: fffffffe   esp: c8b3dea0
kernel: ds: 0018   es: 0018   ss: 0018
kernel: Process squid (pid: 14332, stackpage=3Dc8b3d000)
kernel: Stack: 04a2b067 c1128ac0 fffffffe c9083148 c014071b 04a2b067 c112=
8ac0=20
0084b100
kernel:        00000000 c012fe42 08452000 cb400d20 00000001 c509d5a0 0000=
0001=20
00000001
kernel:        c013012c cb400d20 c509d5a0 08452000 c9083148 0084b100 0000=
0001=20
c8b3c000
kernel: Call Trace: [flush_comp_cache+43/164] [do_swap_page+338/384]=20
[handle_mm_fault+112/240] [do_page_faul
t+388/1204] [do_page_fault+0/1204]
kernel:    [__free_pages+27/28] [sys_getrusage+31/44] [error_code+52/64]
kernel:
kernel: Code: 39 43 18 75 08 8d 43 20 39 43 20 74 3b 8b 0d 30 d0 3e c0 89
kernel:  <3>swap_free: Unused swap offset entry 0084b100


AND ALSO MASSIVE!!!! and i mean MASSIVE failures like this:

kernel: init_special_inode: bogus imode (0)
EXT3-fs error (device ide0(3,10)): ext3_readdir: bad entry in directory #=
2063:=20
rec_len %% 4 !=3D 0 - o
ffset=3D0, inode=3D0, rec_len=3D25, name_len=3D0

ALOT for every second!!! :-(( ... i will downgrade to 0.23pre1 or even=20
0.22final. I have some ext3-fs errors with 0.23pre1 too but i forgot to=20
report it to you Rodrigo :(


--=20
Kind regards
=09Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

[lc-devel] Re: Patch 0.23pre2 for 2.4.18

[Marc-Christian P. <m....@gm...>]

Hi Rodrigo,=20

great! :-) ... I am working hard on the next release of WOLK, v3.4, almos=
t=20
done, including 0.23pre1 and what are you doing?!?! You are releasing pre=
2=20
;-))))))))=20

Anyway, this is great and i will include pre2 in my kernel project wolk :=
-)

--=20
Kind regards
=09Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824  080A 569D E2E3 DB44 1A16
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

[lc-devel] unsubscribe

[sangeetha <san...@ya...>]

---
lin...@li...
wrote:
> Send linuxcompressed-devel mailing list submissions
> to
> 	lin...@li...
> 
> To subscribe or unsubscribe via the World Wide Web,
> visit
> 
>
https://lists.sourceforge.net/lists/listinfo/linuxcompressed-devel
> or, via email, send a message with subject or body
> 'help' to
> 	lin...@li...
> 
> You can reach the person managing the list at
> 	lin...@li...
> 
> When replying, please edit your Subject line so it
> is more specific
> than "Re: Contents of linuxcompressed-devel
> digest..."
> 
> 
> Today's Topics:
> 
>    1. Re: 0.23pre1 2.4.18 feeling good (David Chow)
> 
> --__--__--
> 
> Message: 1
> Date: Tue, 23 Apr 2002 23:38:43 +0800
> From: David Chow <dav...@sh...>
> To: lin...@li...
> Subject: Re: [lc-devel] 0.23pre1 2.4.18 feeling good
> 
> Yes it simply happens again and is reproduceable on
> 2.4.18. I found my 
> machine went so unstable even using 2.4.17 with
> mem=128M (wihtout 
> compressed cache but it did not oops) . I think
> there might be some 
> problem using mem=128M, do you know is there any
> issues my doing this?
> 
> regards,
> 
> David
> 
> Rodrigo Souza de Castro wrote:
> 
> >On Wed, Apr 17, 2002 at 11:16:39PM +0800, David
> Chow wrote:
> >
> >>kmem_cache_grow+0x1ca
> >>kemm_cache_alloc+0x1ab
> >>
> >
> >What about a complete stack trace?
> >
> >>Opps in kmem_cache_grow() somewhere, this is
> caused by kjournald as I am 
> >>runing rh72 with ext3 filesystem. 
> >>It happens at start up after I append mem=128M to
> limit my machine
> >>only to use 128M RAM, if I use 256M it does not
> happen. Please
> >>aware, I am running 2.4.18 with 0.23pre1.
> >>
> >
> >Is this bug reproducible without the compressed
> cache? Besides
> >appending mem=128M to your linux image, what do you
> do to reproduce
> >it?
> >
> >Regards,
> >
> 
> 
> 
> 
> 
> --__--__--
> 
> _______________________________________________
> linuxcompressed-devel mailing list
> lin...@li...
>
https://lists.sourceforge.net/lists/listinfo/linuxcompressed-devel
> 
> 
> End of linuxcompressed-devel Digest


__________________________________________________
Do You Yahoo!?
Yahoo! Health - your guide to health and wellness
http://health.yahoo.com

Re: [lc-devel] 0.23pre1 2.4.18 feeling good

[David C. <dav...@sh...>]

Yes it simply happens again and is reproduceable on 2.4.18. I found my 
machine went so unstable even using 2.4.17 with mem=128M (wihtout 
compressed cache but it did not oops) . I think there might be some 
problem using mem=128M, do you know is there any issues my doing this?

regards,

David

Rodrigo Souza de Castro wrote:

>On Wed, Apr 17, 2002 at 11:16:39PM +0800, David Chow wrote:
>
>>kmem_cache_grow+0x1ca
>>kemm_cache_alloc+0x1ab
>>
>
>What about a complete stack trace?
>
>>Opps in kmem_cache_grow() somewhere, this is caused by kjournald as I am 
>>runing rh72 with ext3 filesystem. 
>>It happens at start up after I append mem=128M to limit my machine
>>only to use 128M RAM, if I use 256M it does not happen. Please
>>aware, I am running 2.4.18 with 0.23pre1.
>>
>
>Is this bug reproducible without the compressed cache? Besides
>appending mem=128M to your linux image, what do you do to reproduce
>it?
>
>Regards,
>

Re: [lc-devel] 0.23pre1 2.4.18 feeling good

[Rodrigo S. de C. <rc...@im...>]

On Wed, Apr 17, 2002 at 11:16:39PM +0800, David Chow wrote:
> kmem_cache_grow+0x1ca
> kemm_cache_alloc+0x1ab

What about a complete stack trace?

> Opps in kmem_cache_grow() somewhere, this is caused by kjournald as I am 
> runing rh72 with ext3 filesystem. 
> It happens at start up after I append mem=128M to limit my machine
> only to use 128M RAM, if I use 256M it does not happen. Please
> aware, I am running 2.4.18 with 0.23pre1.

Is this bug reproducible without the compressed cache? Besides
appending mem=128M to your linux image, what do you do to reproduce
it?

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] 0.23pre1 2.4.18 feeling good

[David C. <dav...@sh...>]

Too early to say stable. A bug just discovered

output from kdb

kmem_cache_grow+0x1ca
kemm_cache_alloc+0x1ab


Opps in kmem_cache_grow() somewhere, this is caused by kjournald as I am 
runing rh72 with ext3 filesystem. It happens at start up after I append 
mem=128M to limit my machine only to use 128M RAM, if I use 256M it does 
not happen. Please aware, I am running 2.4.18 with 0.23pre1.

regards,

David

Rodrigo Souza de Castro wrote:

>On Tue, Apr 16, 2002 at 08:11:15PM +0800, David Chow wrote:
>
>>It seems after using 0.23pre1, I don't have the problem with
>>compressed cache as before (0.21 with 2.4.17), I will try to merge
>>0.22 code in my production kernel with some change in 0.23 and
>>available for my users to test with.
>>
>
>Very nice to hear that. :-)
>
>>Whats the major bug fixes between those? I would like to
>>know.. Thanks.
>>
>
>There are many fixes for buglets from old versions which could, even
>being a buglet, still lock up your machine. Perhaps those fixes might
>have solved some issues you noticed in the 0.21 version.
>
>By the way, if you notice any bug, please report, otherwise I can't
>try to fix it.
>
>Best regards,
>

Re: [lc-devel] 0.23pre1 2.4.18 feeling good

[Rodrigo S. de C. <rc...@im...>]

On Tue, Apr 16, 2002 at 08:11:15PM +0800, David Chow wrote:
> It seems after using 0.23pre1, I don't have the problem with
> compressed cache as before (0.21 with 2.4.17), I will try to merge
> 0.22 code in my production kernel with some change in 0.23 and
> available for my users to test with.

Very nice to hear that. :-)

> Whats the major bug fixes between those? I would like to
> know.. Thanks.

There are many fixes for buglets from old versions which could, even
being a buglet, still lock up your machine. Perhaps those fixes might
have solved some issues you noticed in the 0.21 version.

By the way, if you notice any bug, please report, otherwise I can't
try to fix it.

Best regards,
-- 
Rodrigo S. de Castro <rc...@im...>

Re: [lc-devel] related kerenl config

[Rodrigo S. de C. <rc...@im...>]

Hi,

On Mon, Apr 15, 2002 at 10:59:06AM +0900, Á¤Áø¿ì wrote:
> I'm student in korea. I study operating system and have interesting
> compressed caching system.  so I installed compressed caching system
> in my linux system.

Great. Let us know if you have any suggestions or problems with it.

> Your Install Document describe about Kernel Configuration.
> and I have found "Compressed cache" option in "General setup".
> buf I have no found "Swap Out in Compressed Format (Null Padding)"
> option.

That section is outdated. I removed this option when implementing the
page cache support, since it creates a special case that I wouldn't
like to worry at that moment. I will add it back in a near future.

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

[lc-devel] 0.23pre1 2.4.18 feeling good

[David C. <dav...@sh...>]

It seems after using 0.23pre1, I don't have the problem with compressed 
cache as before (0.21 with 2.4.17), I will try
to merge 0.22 code in my production kernel with some change in 0.23 and 
available for my users to test with.

Whats the major bug fixes between those? I would like to know.. Thanks.

regards,

David 

[lc-devel] related kerenl config

[<jj...@dr...>]

SGkuLi4gbmljZSBtZWV0IHlvdS4NCg0KSSdtIHN0dWRlbnQgaW4ga29yZWEuIEkgc3R1ZHkgb3Bl
cmF0aW5nIHN5c3RlbSBhbmQgaGF2ZSBpbnRlcmVzdGluZyBjb21wcmVzc2VkIGNhY2hpbmcgc3lz
dGVtLg0Kc28gSSBpbnN0YWxsZWQgY29tcHJlc3NlZCBjYWNoaW5nIHN5c3RlbSBpbiBteSBsaW51
eCBzeXN0ZW0uDQoNCmJ1dCBJIGhhdmUgb25lIHByb2JsZW0uIA0KWW91ciBJbnN0YWxsIERvY3Vt
ZW50IGRlc2NyaWJlIGFib3V0IEtlcm5lbCBDb25maWd1cmF0aW9uLg0KYW5kIEkgaGF2ZSBmb3Vu
ZCAiQ29tcHJlc3NlZCBjYWNoZSIgb3B0aW9uIGluICJHZW5lcmFsIHNldHVwIi4NCmJ1ZiBJIGhh
dmUgbm8gZm91bmQgIlN3YXAgT3V0IGluIENvbXByZXNzZWQgRm9ybWF0IChOdWxsIFBhZGRpbmcp
IiBvcHRpb24uDQoNCk15IGxpbnV4IHN5c3RlbSBpcyBsaWtlLVJlZGhhdCA3LjEgdmVyc2lvbiBh
bmQgQ29tcHJlc3NlZCBDYWNoZSBQYXRjaCB2ZXJzaW9uIGlzIDIuNC4xOC0wLjIzcHJlbC4NCg0K
SSdtIGxvb2tpbmcgZm9yd29yZCB0byB5b3VyIGFuc3dlci4NCg0KVGhhbmsgeW91Lg0KDQo=

Re: [lc-devel] kdb with compressed cache

[Rodrigo S. de C. <rc...@im...>]

On Sun, Apr 14, 2002 at 12:52:16AM +0800, David Chow wrote:
> Does anyone tried with kdb together with compressed cache in Linux 
> 2.4.18? Thanks

Yes, I've already tried. Have you noticed any issues using kdb and
compressed cache together?

Regards,
-- 
Rodrigo S. de Castro <rc...@im...>

[lc-devel] kdb with compressed cache

[David C. <dav...@sh...>]

Hi all,

Does anyone tried with kdb together with compressed cache in Linux 
2.4.18? Thanks

David

[lc-devel] Basic Rundown?

[Mike L. <Lo...@cl...>]

Could you please give me a basic run down explaining wat is happening in =
each diagram on your page??

Re: [lc-devel] Page cache compression

[Scott K. <sfk...@cs...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David,

Forgive me if I'm misunderstanding some of your pointer, but you seem to 
be raising oversimplified objections to the use of compression.  
``Compressed caching'' does not imply that *all* pages will be compressed,
  thus increasing the access time to *any* page.  Rather, it implies that 
*some fraction* of pages in RAM will be compressed, and only access to 
those pages will incur the overhead of compression/decompression.

The increase in page access time for pages that are compressed is a key 
component for selecting exactly how large a fraction of RAM should be 
compressed.  The other major component is the number of misses on RAM (for 
uncompressed and compressed pages) that thus require disk accesses.  While 
decompressing a page may be slow, having to get it from disk is *much* 
slower.  If the fraction of compressed pages is too small, then more 
misses will occur, and more of the extremely costly disk access will be 
required.  If the fraction of compressed pages is too large, then too much 
compression and decompression will occur, overwhelming the savings gained 
by preventing disk accesses.

The trick -- the adaptivity to which Rodrigo alluded -- is to find the 
fraction of pages that should be compressed that is ``just right''.  The 
goal is to compress a smallish number of pages, but by doing so, being 
able to cache pages that would otherwise have to be evicted.  If 
compressing a few pages can be made to avoid a large number of misses, 
then the compression is likely to be worthwhile.

> For me, I am a filesystem developer for Linux. I use page cache because 
> of performance, it will be a pity to increase loading and delay for page 
> cache access.

So, it won't be a pity at all if can substantially reduce the number of 
disk accesses.

> It will seriously harm performance of network based filesystem, because 
> they are originally slow and is speed up by using page cache.

When RAM is caching pages from a slow device, that's when compression is 
the *best* idea.  You use compression to keep *more* in RAM, thus avoiding 
that hideously slow network delay.  It's a modest increase in access delay 
for some pages vs. an enormous decrease in access delay for other pages.

> I page cache compression is implemented, that means every writes from 
> user space to pages will have to go through compressed cache, I dont' 
> this is a good idea.

No, it doesn't mean that at all.  Again, not *all* pages are compressed -- 
most of them won't be.  More recently/frequently used pages will remain 
uncompressed, exactly because fast access to those is desirable.  Less 
used pages are the ones that will be compressed.

> In my sense, it is only sensible to compress clean page cache so that the 
> amount of cached data in memory will be increased that means less I/O .

For filesystem caching, this may be a very good heuristic for choosing 
pages to compress.  Nothing wrong with applying this idea...

> I think compressing dirty page is worthless, it will just increase the 
> time delay for a dirty page being write to disk.

As you point out, whether or not compression makes sense depends on the 
filesystem.  Having different rules defined by each type of filesystem is 
not a particularly difficult option to have.

Scott
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (Darwin)
Comment: For info see http://www.gnupg.org

iD8DBQE8o+eS8eFdWQtoOmgRAva3AJ9y2kZUwYWgI2w/KpfxOzuSxEOcJQCfUTOS
2dfkADUXC5oDvsXlLnHApBs=
=hRHk
-----END PGP SIGNATURE-----

Re: [lc-devel] Page cache compression

[David C. <dav...@sh...>]

For me, I am a filesystem developer for Linux. I use page cache because 
of performance, it will be a pity to increase loading and delay for page 
cache access. It will seriously harm performance of network based 
filesystem, because they are originally slow and is speed up by using 
page cache.
Rodrigo Souza de Castro wrote:

>Hi David et al,
>
>On Fri, Mar 22, 2002 at 10:15:25PM +0800, David Chow wrote:
>
>>Would anyone tell me what is page cache compression, on the change
>>log for 0.22 Rodrigo says page cache is also compressed, can Rodrigo
>>explain please. I would like to know what to aware when I implement
>>my filesystem, because I also have compression in my page
>>caches. Thanks.
>>
>
>The 0.22 version includes some new features like page cache
>compression and clean pages support. The latter means that not only
>the dirty pages, but also the clean pages, are added to compressed
>cache. It something very sensible for compressed cache idea but wasn't
>implemented so far.
>
As for me, I write compression filesystems to speed up transfer speed on 
slow media. Over the network the effect is obvious, even on slow IDE 
drives, the speed improvements is also obvious.

>
>The page cache support in compressed cache allows all file mapped
>pages (ie, page cache pages) to be added to compressed cache. I
>thought of implementing page cache support after noticing that some IO
>intensive applications didn't have performance gains with the "usual"
>compressed cache (the one which stores only anonymous pages). Some
>tests had been run and out of them some statistics were made available
>on our project page (statistics for version 0.21) where we could
>notice performance drop when running IO intensive applications (for
>example, dbench) on a kernel with compressed cache enabled.
>
>At that moment I didn't think very much on the impact of an idea like
>that and started implementing this support to make compressed cache
>work better for these cases. Along with the clean page support, it
>actually helped IO intensive applications, like dbench (you can have a
>look at the statistics page for 0.22pre7 and later version).
>
I page cache compression is implemented, that means every writes from 
user space to pages will have to go through compressed cache, I dont' 
this is a good idea. From my experience, most of the filesystem uses 
generic_file_read/write paths such that all writes are sychronous and 
will call fs specific commit_write() directy. Only shared mmaped pages 
will delay writes. Cache implementation is an fs specific issue.  I 
can't see many applications uses shared mmaped pages for I/O intensive 
operations.

>
>Nevertheless, this page cache support is new and experimental, never
>implemented in other compressed cache implementations and not even
>mentioned in Scott Kaplan's thesis. We still have to perform tests and
>some deep analyses to check if this support will be kept in the
>future, mainly after implementing adaptivity.
>
>You can notice that a compressed cache like that, with swap cache and
>page cache support, will hold pages with very different
>behaviours. Unlike swap cache pages, dirty file mapped pages need be
>synced to the disk, which will follow OS-specific parameters (kupdated
>parameters in Linux, for example) or user/applications parameters (for
>instance, syscalls or mount with sync flag). That's why I don't know
>exactly when it's worth to compress a file mapped page, since it might
>be decompressed to be synced right after the compression compression
>(thus we might be wasting compressions).
>
In my sense, it is only sensible to compress clean page cache so that 
the amount of cached data in memory will be increased that means less I/O .

>
>Besides that, in both cases storing a page from either swap cache or
>page caches might save us a disk read (if we reference it again and we
>still have it in compressed cache and it would have to be read from
>disk otherwise). In swap cache page, it might even save us a disk
>write if it happens to be compressed and never have to write out to
>swap until program exits (in common case we have dirty swap
>pages). With a page cache page, we are still not sure it might save us
>a disk write, since it will need to synced soon if dirty.
>
As I said, generic read/write paths will sync to all fs specific write 
calls, and whether sync to disk or not is an fs specific issue, think 
about mounting NFS with sync and async.

>
>Those reasons above make me believe we won't have a single parameter
>for adaptivity that will work with swap and page cache at the same
>time. I don't even think we would be able to find a single parameter
>that would work well only with page cache. Therefore, I am not sure
>this support will be kept when adaptivity gets implemented.
>
>Anyway, I think it's been worth to think about this problem no matter
>what happens to this page cache support. This page cache support
>showed us that we were right about some assumptions (for example, that
>dbench bad performance was, even partialy, due to a smaller page and
>buffer caches) and also helped us see some similar problems in other
>compressed cache implementations.
>
>Best regards,
>
I think compressing dirty page is worthless, it will just increase the 
time delay for a dirty page being write to disk. I think this is the 
most challlenging for fs developers wants to sync all dirty page to disk 
as soon as possible and make dirty pages reachs their fs specific calls. 
At the moment, vfs and kswapd takes care of dirty page and fs specific 
code have less control in this. For me I would like the dirty pages come 
to my fs specific code so that I can take control more soon.

Journalling filesystems is a trend, and these filesystems likely want to 
commit dirty pages in all transactions of data writes. So thinking of 
this you will understand my point.

I suggest having a compile time option to support clean page cache 
compression, this will have an effect of upsizing the page cache. It 
also make page cache compression easier to implement by just discards 
invalidated compressed pages for a 1 way implementation.