Re: [Ziproxy-devel] ziproxy 3.1.x CPU load problem... patch to not do PNG recompress.

[Andrew L. <al...@gm...>]

This particular ziproxy machine has an EPIA Eden ESP 600 "Samuel 2" (600MHz,
64Kb L2 Cache). Which is over 5 years old. It's on the lean-side, but it
still grunts along, fan-less system & consuming only 7-10Watts.

cpu MHz    : 601.471
cache size : 64 KB
flags        : fpu de tsc msr cx8 mtrr pge mmx 3dnow up

I'll upgrade it eventually!

I've implemented those DCT changes you suggested, not sure it's had much
affect.
Probably need to set up some test pages to compare/benchmark.

Andrew


On 21 June 2010 04:11, Daniel Mealha Cabrita <da...@gm...> wrote:

>
> That's peculiar. I didn't realize that libpng could be so heavy.
> AFAIR libpng has some kind of fuzzy analysis to guess which is the best
> compression strategy -- perhaps that's the reason.
>
> I will collect some compression statistics later and see what's the best to
> do
> about.
>
>
> Parallel to this, it seems that your CPU is one of the old models with a
> rather slow FPU (how much L2 cache does it have?).
> While keeping the changes you've made, could you try replacing the two
> occurences of:
> cinfo.dct_method = JDCT_FLOAT;
> to
> cinfo.dct_method = JDCT_ISLOW;
> And see if it improves performance?
>
>
> Em Sunday 20 June 2010, Andrew Leahy escreveu:
> > Dan, I've done a little digging to find why image 'decompression' (JP2
> > to JPEG) has been slow since 3.1.x on my tiny EPIA box.
> >
> > Things were slow enough that the 90second timeout was sometimes being
> > hit with a maximum of 4 simultaneous connections. CPU Load would sit
> > above 4 (~25% for each ziproxy). The machine was CPU flatlining on
> > image recompressing, no network I/O was happening while the image
> > recompress was busy number crunching.
> >
> > The low-end EPIA has no SSE flags. Recompiling with -O3 made almost no
> > difference. Limiting simultaneous connections to 1 or 2 actually made
> > things a little better... as it made more CPU resources available for
> > image recompress.
> >
> > Anyhow, the main culprit appears to be the dual recompression of JP2
> > into JPEG and PNG in image.c
> >
> > This is the ziproxy debug log, with a little extra added to show some
> > of the flags, before and after the 'decision logic' -
> >
> > [10036] Starting image decompression...
> > [10036] Image parms (non-palette) -- w: 600, h: 900, bpp: 3
> > [10036] Deciding image compression strategy...
> > [10036] Init flags: has_transparency=0 try_lossless=1 try_lossy=1
> > [10036] Final flags: has_transparency=0 try_lossless=1 try_lossy=1
> > [10036] Strategy defined. Continuing...
> > [10036] Attempting JPEG compression...
> > [10036] Attempting PNG compression...
> > [10036] Compression return codes -- JP2K:33 JPEG:0 PNG:0
> > [10036] Image modification/compression took 21537422 us.
> > [10036]   and returned 0.
> >
> > As you can see it took 21seconds to complete the recompress for a
> > 600x900 JP2 image (this is with  MaxConnections=1)
> >
> > What I've done is added an extra bit of logic to image.c -
> >
> > If source is JP2, has no transparency, the target is JPEG, and lossy
> > encode (PNG) is still enabled... then turn lossy off!
> >
> >
> > This has returned my client ziproxy to it's normal speed (big images
> > taking 4-5 seconds), increased MaxConnections to my "normal" 6.
> >
> >
> > I don't think this is a solution for everybody, and you may have a
> > better idea about when it is really appropriate to do a PNG
> > recompress?
> >
> >
> > image.c in compress_image()
> >
> > #ifdef JP2K
> >         if ((source_type == IMG_JP2K) && (has_transparency == 0) &&
> > (try_lossy == 0))
> >                 try_lossless = 0;
> >         if ((source_type == IMG_JP2K) && (has_transparency != 0) &&
> > (target_lossy == IMG_JP2K))
> >                 try_lossless = 0;
> >         if ((source_type == IMG_JP2K) && (has_transparency == 0) &&
> > (target_lossy = IMG_JPEG) && (try_lossy == 1)) /* Alf 20100620 */
> >                 try_lossless = 0;
> > #endif
> >
> >
> >
> > My debug output looks like this...
> >
> > [10070] Starting image decompression...
> > [10070] Image parms (non-palette) -- w: 600, h: 900, bpp: 3
> > [10070] Deciding image compression strategy...
> > [10070] Init flags: has_transparency=0 try_lossless=1 try_lossy=1
> > [10070] Final flags: has_transparency=0 try_lossless=0 try_lossy=1
> > [10070] Strategy defined. Continuing...
> > [10070] Attempting JPEG compression...
> > [10070] Compression return codes -- JP2K:33 JPEG:0 PNG:33
> > [10070] Image modification/compression took 3488304 us.
> > [10070]   and returned 0.
> >
> > So from 21seconds to 3seconds.
> >
> > Cheers, Andrew.
>
>
>
>
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit.  See the prize list and enter to win:
> http://p.sf.net/sfu/thinkgeek-promo
> _______________________________________________
> Ziproxy-devel mailing list
> Zip...@li...
> https://lists.sourceforge.net/lists/listinfo/ziproxy-devel
>



-- 
"The simplest way to deal with a need or want is to stop needing or wanting
it." (John Michael Greer)

My household GHG emissions from energy & transport ~4kg CO2e/day or
~1.4tonnes/year
Household daily use of Water 120L, Electricity 4.0kWh, Petrol 1.2L, Gas 0MJ