Re: [lc-devel] Compressed Cache included into CK patchset

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

On Saturday 14 September 2002 22:30, Rodrigo Souza de Castro wrote:

Hi Rodrigo,

> He first included the 0.24pre1 version for 2.4.18 Linux kernel. After
> releasing the patch for 2.4.19, he had some issues (hangs) when
> running with the Andrea Arcangeli's latest VM patches. I helped him to
> merge the compressed cache code and now it works stably, but an issue
> has been noticed by him: some pauses (about 1-2 seconds) after using
> the machine for an hour and loading and unloading many programs. If
> somebody also has this problem, please let me know.
Anyway, as you know Rodrigo, I also do benchmarking alot of kernels/kerne=
l=20
patches/kernel patchsets around, also including 2.4.19 and also -ckX tree=
=2E

Anyway, the stops cannot be caused by Compressed Cache since all 2.4.19=20
kernels (including vanilla) and patchsets (-aa, -rmap, -ck, -jp, -mjc) et=
c.=20
are causing those stops for some seconds on heavy load. I've posted this=20
issues right 1-2 days after releasing 2.4.19 final and also _BEFORE_ 2.4.=
19=20
gone final. I've posted this behaviour when 2.4.19-pre5 came out (-pre4 w=
as=20
ok) ... Anyway, I know some users that are experiencing the same behaviou=
r,=20
but as far as I can see on the kernel mailing list, this either isn't=20
experienced by all or they just don't notice it.
Rodrigo, if this is real and appears for Con Kolivas, you hit a bug in 2.=
4.19,=20
O(1), Lowlat or -aa VM, not in your code (I bet 99,9%, no, almost 100%) ;=
-)

Also note: This behaviour does _not_ occur with 2.4.18 + compcache nor wi=
th=20
2.4.18-wolk3.6 + compcache :) I've done ALOT benchmarking (since no one e=
lse=20
do this with WOLK ;(

Rodrigo, I've compiled 0.24pre4 right after we talked on irc, no slowdown=
s or=20
anything !!

-

Also, what I am really wondering about is, why all the world uses (and al=
so=20
wants) -rmap stuff. My experience with -rmap is not good. In any situatio=
n=20
I've tested those -rmap enabled kernels, it is alot slower than vanilla/-=
aa=20
or similar.

For sure I've also tested 2.4.19-ck7 with -aa (alot faster as with -rmap)=
=2E
Con's purpose is for Desktop environments, if I read it correct. Just do=20
something like this (normal desktop pc doings; ok, almost ;)

1. start unreal tournament _OR_ start quake2/3
    or any other heavy loaded game
2. compile 2 (two) kernel trees at once
3. do "updatedb"
4. do "find /"

Try to play this game without noticing you were doing 2., 3. and 4. (clos=
e=20
your ears if you have loud harddisks ;-)))

2 kernel's performs well, only two, but also with a little difference:

1. 2.4.19-rc5-aaX, 2.4.20-pre5-aaX
    you will notice, that you're doing 2-4 but gameplay is ok.
2. 2.4.18-wolk3.6rc1
    you won't notice anything.

Don't ask me why 2. is as it is :)

wolk3.6-rc1 with standard config, Preempt/Lockbreak/Lowlat-Mini, Mempools

All other kernel's I've tested are really horribly in that circumstance.
For sure, this is definitive NOT a server's purpose, but desktop users mi=
ght=20
do this (even compiling only one kernel tree at once).

Unfortunately all other kernels slows down alot.
2.4.19-ck7 unfortunately stops every 15-20 seconds for 2-5 seconds with h=
eavy=20
load.

Testmashine is:
---------------
- Celeron 800MHz (128k cache)
- 256MB RAM
- 60GB IDE Disk (UDMA5 mode)
- 32MB ATI All-In-Wonder 128PRO

 Env:
 ----
  FrameBuffer (1024x768-8@75)
  ext3 fs (data=3Djournal)
  XFree 4.2.0
  Debian SID (from yesterday's updated)

   sysctl.conf values:
   -------------------
    kernel.random.poolsize =3D 8192=20
    vm.comp_cache.size =3D 8192
    vm.overcommit_memory =3D 1
    vm.bdflush =3D 50 0 0 0 60 300 60 0 0
    fs.file-max =3D 15872
    fs.inode-max =3D 16384
    kernel.threads-max =3D 65535
    vm.max-readahead =3D 512
    vm.min-readahead =3D 64
    vm.pagetable_cache =3D 75 90
    vm.freepages =3D 300 400 500

  and many tcp/ip tuning, but irrelevant for this tests :)

root@codeman:[/] # mount
/dev/hda3 on / type ext3,ext2 (rw,errors=3Dremount-ro)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw,gid=3D5,mode=3D620)
/dev/hda2 on /boot type ext3 (rw,noexec,nosuid,nodev,noatime,data=3Djourn=
al)
/dev/hda5 on /home type ext3 (rw,data=3Djournal)
/dev/hda6 on /opt type ext3 (rw,data=3Dwriteback)
/dev/hda7 on /opt/squid/cache type ext3 \
                               (rw,noexec,nosuid,nodev,noatime,data=3Dwri=
teback)
/dev/hda9 on /usr/src type ext3 (rw,data=3Dordered)
/dev/hda10 on /var/log type ext3 (rw,noexec,nosuid,nodev,noatime,data=3Do=
rdered)
/dev/hda11 on /var/mail type ext3 \
                                 (rw,noexec,nosuid,nodev,noatime,data=3Do=
rdered)
/dev/hda12 on /var/spool type ext3 (rw,noexec,nodev,noatime,data=3Dwriteb=
ack)
tmpfs on /dev/shm type tmpfs (rw,size=3D200M,mode=3D777)
tmpfs on /tmp type tmpfs (rw)
none on /cdrom type supermount (ro,dev=3D/dev/cdrom,fs=3Diso9660)

root@codeman:[/] # cat /proc/swaps=20
Filename                        Type            Size    Used    Priority
/dev/hda1                       partition       1028120 0       1

root@codeman:[/] # cat /proc/meminfo |egrep "Mem|kB"|grep -v "Mem:"
MemTotal:       255420 kB
MemFree:          7440 kB

root@codeman:[/] # dmesg|grep "Kernel command line"
Kernel command line: BOOT_IMAGE=3DWOLK-3.6 ro root=3D303 rootflags=3Ddata=
=3Djournal=20
compsize=3D32M gracl=3Doff video=3Daty128fb:accel,1024x768-8@75 panic=3D6=
0 kbd-reset=20
ether=3D0,0,eth1 debug devfs=3Dnomount idebus=3D33 hda=3Dautotune hdd=3Di=
de-scsi=20
ide0=3Data66 max_scsi_luns=3D1 maxcpus=3D0

> There is also a version of his patchset including rmap VM by Rik van
> Riel. In this case, compressed cache cannot be integrated given that
> we don't have a port to this VM currently.
hehe, if you'll do the merge also to -rmap you probably have to fix the=20
performance issues with -rmap with your -compcache ;)

> Recall that compressed cache was first included into another patchset,
> the Wolk project [2] by Marc-Christian Petersen.
hehe :-) Thanks for mentioning.

Anyway, 0.24pre4 is included right now and works ok, but I wait for grsec=
urity=20
v1.9.7 final to get out WOLK v3.6-final. (there must be a reason why I do=
n't=20
switch over to 2.4.19 as base for WOLK ;)

P.S.: Anyway, all these kernels performs alot _better_ than current 2.5.3=
4 :)
      Also just started up Unreal Tournament (my girlfriend wants to play=
 a
      bit; yes, a woman is playing UT :) woohoo) and I connected to that
      mashine via ssh, just did a "mailq" (sendmail stuff) and she cried:
      "Hey, the game just stops, ah, goas ahead!" ... hmm, I thought, str=
ange.
      Did a "while true; do mailq; done" and the game freezes until I pre=
ssed=20
      Ctrl-C to stop the loop. ... and my girlfriends starts crying again=
 :)

Funny eh? I laughed out loud :)

Hopefully all those bugs with 2.4.19 are gone with 2.4.20 final!

Nice weekend! (Sunday)

--=20
Kind regards
        Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824 080A 569D E2E3 DB44 1A16
Key available at www.keyserver.net. Encrypted e-mail preferred.