Re: [Ziproxy-devel] ziproxy 3.1.x CPU load problem... patch to not do PNG recompress.
Brought to you by:
dmcabrita
|
From: Daniel M. C. <da...@gm...> - 2010-06-22 15:06:12
|
Try the attached patch and see how Ziproxy behaves in your machine. .dan Em Monday 21 June 2010, Andrew Leahy escreveu: > This particular ziproxy machine has an EPIA Eden ESP 600 "Samuel 2" > (600MHz, 64Kb L2 Cache). Which is over 5 years old. It's on the lean-side, > but it still grunts along, fan-less system & consuming only 7-10Watts. > > cpu MHz : 601.471 > cache size : 64 KB > flags : fpu de tsc msr cx8 mtrr pge mmx 3dnow up > > I'll upgrade it eventually! > > I've implemented those DCT changes you suggested, not sure it's had much > affect. > Probably need to set up some test pages to compare/benchmark. > > Andrew > > On 21 June 2010 04:11, Daniel Mealha Cabrita <da...@gm...> wrote: > > That's peculiar. I didn't realize that libpng could be so heavy. > > AFAIR libpng has some kind of fuzzy analysis to guess which is the best > > compression strategy -- perhaps that's the reason. > > > > I will collect some compression statistics later and see what's the best > > to do > > about. > > > > > > Parallel to this, it seems that your CPU is one of the old models with a > > rather slow FPU (how much L2 cache does it have?). > > While keeping the changes you've made, could you try replacing the two > > occurences of: > > cinfo.dct_method = JDCT_FLOAT; > > to > > cinfo.dct_method = JDCT_ISLOW; > > And see if it improves performance? > > > > Em Sunday 20 June 2010, Andrew Leahy escreveu: > > > Dan, I've done a little digging to find why image 'decompression' (JP2 > > > to JPEG) has been slow since 3.1.x on my tiny EPIA box. > > > > > > Things were slow enough that the 90second timeout was sometimes being > > > hit with a maximum of 4 simultaneous connections. CPU Load would sit > > > above 4 (~25% for each ziproxy). The machine was CPU flatlining on > > > image recompressing, no network I/O was happening while the image > > > recompress was busy number crunching. > > > > > > The low-end EPIA has no SSE flags. Recompiling with -O3 made almost no > > > difference. Limiting simultaneous connections to 1 or 2 actually made > > > things a little better... as it made more CPU resources available for > > > image recompress. > > > > > > Anyhow, the main culprit appears to be the dual recompression of JP2 > > > into JPEG and PNG in image.c > > > > > > This is the ziproxy debug log, with a little extra added to show some > > > of the flags, before and after the 'decision logic' - > > > > > > [10036] Starting image decompression... > > > [10036] Image parms (non-palette) -- w: 600, h: 900, bpp: 3 > > > [10036] Deciding image compression strategy... > > > [10036] Init flags: has_transparency=0 try_lossless=1 try_lossy=1 > > > [10036] Final flags: has_transparency=0 try_lossless=1 try_lossy=1 > > > [10036] Strategy defined. Continuing... > > > [10036] Attempting JPEG compression... > > > [10036] Attempting PNG compression... > > > [10036] Compression return codes -- JP2K:33 JPEG:0 PNG:0 > > > [10036] Image modification/compression took 21537422 us. > > > [10036] and returned 0. > > > > > > As you can see it took 21seconds to complete the recompress for a > > > 600x900 JP2 image (this is with MaxConnections=1) > > > > > > What I've done is added an extra bit of logic to image.c - > > > > > > If source is JP2, has no transparency, the target is JPEG, and lossy > > > encode (PNG) is still enabled... then turn lossy off! > > > > > > > > > This has returned my client ziproxy to it's normal speed (big images > > > taking 4-5 seconds), increased MaxConnections to my "normal" 6. > > > > > > > > > I don't think this is a solution for everybody, and you may have a > > > better idea about when it is really appropriate to do a PNG > > > recompress? > > > > > > > > > image.c in compress_image() > > > > > > #ifdef JP2K > > > if ((source_type == IMG_JP2K) && (has_transparency == 0) && > > > (try_lossy == 0)) > > > try_lossless = 0; > > > if ((source_type == IMG_JP2K) && (has_transparency != 0) && > > > (target_lossy == IMG_JP2K)) > > > try_lossless = 0; > > > if ((source_type == IMG_JP2K) && (has_transparency == 0) && > > > (target_lossy = IMG_JPEG) && (try_lossy == 1)) /* Alf 20100620 */ > > > try_lossless = 0; > > > #endif > > > > > > > > > > > > My debug output looks like this... > > > > > > [10070] Starting image decompression... > > > [10070] Image parms (non-palette) -- w: 600, h: 900, bpp: 3 > > > [10070] Deciding image compression strategy... > > > [10070] Init flags: has_transparency=0 try_lossless=1 try_lossy=1 > > > [10070] Final flags: has_transparency=0 try_lossless=0 try_lossy=1 > > > [10070] Strategy defined. Continuing... > > > [10070] Attempting JPEG compression... > > > [10070] Compression return codes -- JP2K:33 JPEG:0 PNG:33 > > > [10070] Image modification/compression took 3488304 us. > > > [10070] and returned 0. > > > > > > So from 21seconds to 3seconds. > > > > > > Cheers, Andrew. > > > > ------------------------------------------------------------------------- > >----- ThinkGeek and WIRED's GeekDad team up for the Ultimate > > GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the > > lucky parental unit. See the prize list and enter to win: > > http://p.sf.net/sfu/thinkgeek-promo > > _______________________________________________ > > Ziproxy-devel mailing list > > Zip...@li... > > https://lists.sourceforge.net/lists/listinfo/ziproxy-devel |