Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Michael Droettboom - 2011-10-21 15:21 ```On 10/21/2011 09:49 AM, Daniel Hyams wrote: > All sounds reasonable Mike; I do agree that patching the agg source > code is not that desirable; I was operating under the (incorrect) > assumption that most, if not all, backends used straight alpha. > > I'll certainly test the patch tonight, but I can only test it under > wxAgg reasonably, which is one of the backends that expects straight > alpha as far as I know. > > The "loss in picture detail" comment was just discomfort with the fact > that once the alphas are premultiplied in, there is not an exact > reverse transformation to get your original color back. In googling > around to better explain here, I found this, which is a much more in > depth and better explanation than what I would have come up with: > > http://www.quasimondo.com/archives/000665.php > > The crux is here: > >> An example: when you set the alpha value of a pixel to 16 all color values will be multiplied with >> a factor of 16/256 = 0.0625. So a gray pixel of 128 will become 128 * 0.0625 = 8, a darker pixel >> of 64 will become 64 * 0.0625 = 4. But a slightly lighter pixel of maybe 67 will become 67 * >> 0.0625 = 4.1875 - yet there are no decimals in integer pixels which means it will also become >> 4. The effect that you will get posterization - setting your alpha channel to 8 means that you >> also reduce your color channels to 8 levels, this means instead = 256*256*256 different colors >> you will end up with a maximum of 8*8*8 = 512 different colors. > One might argue that the loss of color information is not that > crucial, because for very low alpha (where the problem is most > pronounced), the image is almost invisible anyway... so it won't > matter. That's true, but what if I want to (for whatever reason) take > the image's pixels again, before draw, and boost the alpha up again? > What I'll get is a posterized mess. So, I'm still of the opinion that > patching agg in this situation might be the best solution to this. > This way, straight alphas are used throughout, and for backends that > require premultiplied alpha, the alpha can be premultiplied in at the > latest possible moment. Thanks for clarifying. I understand what you're saying now. I think what we want to do is store the unmultiplied alpha as a "canonical" version of the image, and premultiply a copy (or use some C++ iterator magic to avoid the copy) right before sending it off to Agg. Then the alpha can be fully "tweakable" at runtime. I'll try to tackle this problem, as well as the problem that set_alpha simply doesn't work, at the same time when I get a chance (or patches are always welcome, of course :). Mike > Thanks for all of the help with this Mike, > > Daniel > > On Fri, Oct 21, 2011 at 9:31 AM, Michael Droettboom wrote: >> Thanks for looking into this deeper. >> >> Agg requires image buffers to be premultiplied, as described in the third >> bullet point here. (It's not exactly clear, to say the least, but that's >> what I take it to mean, and also from reading the code). >> >> http://www.antigrain.com/news/release_notes/v22.agdoc.html >> >> The bug is that in _image.cpp the input buffers are not declared as >> premultiplied in _image.cpp. Arguably it is a bug that agg doesn't reject >> filtering unmultiplied images, since the note states that the assumption is >> that they are premultiplied by the time they get to the filters. >> >> I have attached a patch that fixes this. Would you mind testing it and let >> me know how it works for you? >> >> On 10/20/2011 10:29 PM, Daniel Hyams wrote: >>> I've looked all over the place through both the Python and C code, and >>> I don't see any premultiplication of alphas at any stage before the >>> pixels are passed off to agg, and neither can I find any place where >>> the alphas are "unmultiplied" on the way back from agg to the backend >>> for rendering. >> matplotlib's support for alpha blending of images is basically by accident, >> so it's not surprising the details aren't right. >> >> I think it is a bug that after reading images in we don't premultiply them >> before sending them to Agg. That bug has existed for a long time in >> matplotlib because no one is really using alpha images a great deal. >> (Masked images, yes, but that implies alpha is strictly 0 or 255 and thus >> these issues don't come into play.) >> >> Unmultiplying is not always necessary. Many of the GUI backends also expect >> premultiplied alpha (Qt for example). However, there is certainly a bug in >> writing PNG files (where the file format specifies unmultiplied). >> >>> It's very possible that I missed it, but I would have to miss it in >>> two places (premultiply and the unmultiply). It looks to me like the >>> output from agg ends up getting passed on directly to the renderer, >>> which as far as I know, just uses straight alpha. The WxAgg renderer, >>> for example, just creates a wx.Bitmap out of the pixels and blits it. >>> Which means that any image going through agg's filters will not be >>> correct if it has any pixels with alpha != 0 or != 255. >>> >>> [Using PIL images because they are simple to talk about...but the PIL >>> image could alternatively be an image.py image] >>> >>> As far as I can tell, the image pixels current go through a pipeline >>> like the following: >>> >>> [1] PIL Image -> _image image -> agg operations -> modified and/or >>> resized _image image -> renderer >>> >>> If agg expects premultiplied alpha, the procedure should look something >>> like: >>> >>> [2] PIL Image -> _image image -> premultiply alphas ->agg options -> >>> unmultiply alphas -> modified and/or resized _image image -> renderer >>> >>> I personally don't like pipeline [2] because picture detail is lost in >>> the "unmultiply alphas" stage. Better to use straight alpha all the >>> way through. >> I think what needed is: >> >> [3] PIL Image (or _png.cpp) -> premultiply alphas -> _image image -> agg >> options -> >> -> modified and/or resized _image image -> renderer -> (unmultiply >> alphas)? -> GUI library >> >> That is -- all image data should be kept premultiplied internally in all >> buffers for efficiency and because this is what Agg is designed for. >> >> Can you explain what you mean by "picture detail is lost in the unmultiply >> alphas stage". There is the usual problem that by premultiplying you lose >> any color data where alpha = 0 (and you lose resolution everywhere else, but >> not resolution you can actually see after compositing). >>> So long as matplotlib is using only a subset of agg algorithms that >>> work no matter whether the alphas are premultiplied or not, I would >>> think that the most reasonable route was the one that I took; to >>> always pass straight alphas (sticking with pipeline [1]), and modify >>> the agg source slightly to fit matplotlib's approach (i.e., remove the >>> clipping there). >> I'd be really wary of modifying agg like this. Those things become hard to >> maintain. I think this instead a bug in matplotlib and should be fixed >> there. >> >> I've put an issue in the issue tracker here: >> >> https://github.com/matplotlib/matplotlib/issues/545 >> >> Cheers, >> Mike >> >>> I hope that I'm not way off base (I have a sneaking feeling that I am >>> :O ), and hope this helps. I've verified on both Linux and Windows >>> that removing the alpha-clip lines from agg_span_image_filter_rgba.h, >>> rebuilding matplotlib, and replacing _image.so/_image.pyd and >>> _backend_agg.so/_backend_agg.pyd does the trick (along with passing >>> straight alphas). So far, I've seen no ill effects on any of my >>> plots, but I'm also not in a position to run the pixel-by-pixel >>> comparison matplotlib tests. >>> >>> >>> On Wed, Oct 19, 2011 at 7:26 PM, Daniel Hyams wrote: >>>> There has to be something else in play here. I'll try to keep this >>>> short, but the summary is this: I can get the transparency to look >>>> right, but only if 1) I put "straight" alpha in image, not >>>> premultiplied, and 2) I hack agg to remove specificially every >>>> instance of the lines of code that you refer to above. >>>> >>>> Why this is, I don't know. Hopefully I'm still misusing something. >>>> However, it behaves as if the clipping of alpha in the agg library is >>>> corrupting the alpha channel. I also submit that I could have broken >>>> some other transparency capabilities of matplotlib, because I don't >>>> know what other routines use what I hacked....I did check a few >>>> transparent polygons and such though, and everything seemed to be >>>> fine. >>>> >>>> I know that the agg library has been around for quite a long time, so >>>> that also means that such a basic bug is unlikely. >>>> >>>> I've reattached the (slightly modified) script that reproduces the >>>> problem, along with a sample image that it uses. The only change to >>>> the script is right at the top, where a different image is read, a >>>> quick statement is placed to add an alpha channel if there is not >>>> already one, and I'm attempting to use premultiplied alphas. I've >>>> also attached a screenshot of the output. Notice that in this case, >>>> both "transparent" images look wrong. >>>> >>>> Now, if I 1) hack agg to remove the alpha clipping, and 2) modify the >>>> one line in the attached python script so that I use straight alpha, >>>> everything looks right. Specifically, I removed every instance of the >>>> code below from xxxx, rebuilt all of the matplotlib .so's, and >>>> specifically replaced _image.so and _backend_agg.so in my matplotlib >>>> distribution. >>>> >>>> if(fg[order_type::A]> base_mask) fg[order_type::A] >>>> = base_mask; >>>> if(fg[order_type::R]> fg[order_type::A]) >>>> fg[order_type::R] = fg[order_type::A]; >>>> if(fg[order_type::G]> fg[order_type::A]) >>>> fg[order_type::G] = fg[order_type::A]; >>>> if(fg[order_type::B]> fg[order_type::A]) >>>> fg[order_type::B] = fg[order_type::A]; >>>> >>>> >>>> >>>> >>>> On Wed, Oct 19, 2011 at 2:34 PM, Daniel Hyams wrote: >>>>> Ah, thanks so much Michael! That explanation helps a great deal; I >>>>> was always considering things in "straight alpha" format, not even >>>>> knowing that there was alternative. >>>>> >>>>> I'll play with this tonight; I don't see any problem getting the thing >>>>> working, though, now that I know what agg expects to see... >>>>> >>>>> And yes, alpha support in the image class would be very helpful ;) >>>>> >>>>> On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom >>>>> wrote: >>>>>> You are right that Agg is doing the resizing here. Agg expects >>>>>> premultiplied alpha. See [1] for information about what that means. >>>>>> >>>>>> [1] http://en.wikipedia.org/wiki/Alpha_compositing >>>>>> >>>>>> After Agg interpolates the pixel values, to prevent oversaturation it >>>>>> truncates all values to be less than alpha (which makes sense if >>>>>> everything >>>>>> is assumed to be premultiplied alpha). Arguably, the bug here is that >>>>>> nearest neighbor (which doesn't have to do any blending) doesn't >>>>>> perform the >>>>>> truncation step -- then both would look "wrong". >>>>>> >>>>>> It happens in this code snippet in span_image_filter_rgba: (base_mask >>>>>> is >>>>>> 255) >>>>>> >>>>>> if(fg[order_type::A]> base_mask) >>>>>> fg[order_type::A] >>>>>> = base_mask; >>>>>> if(fg[order_type::R]> fg[order_type::A]) >>>>>> fg[order_type::R] >>>>>> = fg[order_type::A]; >>>>>> if(fg[order_type::G]> fg[order_type::A]) >>>>>> fg[order_type::G] >>>>>> = fg[order_type::A]; >>>>>> if(fg[order_type::B]> fg[order_type::A]) >>>>>> fg[order_type::B] >>>>>> = fg[order_type::A]; >>>>>> >>>>>> So, the solution to make a partially transparent image is to not do: >>>>>> >>>>>> pix[:,:,3] = 127 >>>>>> >>>>>> but instead, do >>>>>> >>>>>> pix *= 0.5 >>>>>> >>>>>> Of course, the real fix here is to support alpha blending properly in >>>>>> the >>>>>> image class, then the user wouldn't have to deal with such details. A >>>>>> bug >>>>>> should probably be filed in the matplotlib issue tracker for this. >>>>>> >>>>>> Mike >>>>>> >>>>>> On 10/19/2011 12:23 PM, Daniel Hyams wrote: >>>>>> >>>>>> [Sorry, I keep getting tripped up with HTML mail....resent in ascii, >>>>>> and resaved one of the attachment png's to make it smaller.] >>>>>> >>>>>> >>>>>> Example script attached (PIL required). Basically, if I impose a >>>>>> specific value into an image's alpha channel and use any interpolation >>>>>> scheme other than 'nearest', there appears gray all where the figure >>>>>> didn't have any color to begin with. I've also attached a screenshot >>>>>> of the output of the script on my machine. >>>>>> >>>>>> Hopefully I'm doing something wrongly? >>>>>> >>>>>> I chased the problem and managed to hack in a solution that fixes the >>>>>> problem, but it's extremely inefficient...basically, in matplotlib's >>>>>> image.py, routine BboxImage.make_image, you can create two images >>>>>> there....one with no alpha channel (call it imRGB) and one with (call >>>>>> it imRGBA). Go through all of the routine, doing exactly the same >>>>>> things to both of the images *except* for the interpolation, which is >>>>>> set to 'nearest' for imRGBA. Then, rip the colors out of imRGB, the >>>>>> alpha channel off of imRGBA, and put them together....go through all >>>>>> of the routine again with this composited image, and it works. I >>>>>> know...I told you it was bad ;) >>>>>> >>>>>> The problem seems to be in the "resize" call in that routine...resize, >>>>>> which calls into C code, does not appear to handle things correctly >>>>>> when the alpha is anything other than 255's across the board. It >>>>>> might be a problem in the agg routines, but hopefully it is just maybe >>>>>> a misuse of the agg routines. >>>>>> >>>>>> The behavior seems to be backend independent as far as I could test (I >>>>>> tried with wxagg and tk backends). I am using mpl 1.0.0 on Windows if >>>>>> it matters. >>>>>> >>>>>> >>>>>> -- >>>>>> Daniel Hyams >>>>>> dhyams@... >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> All the data continuously generated in your IT infrastructure contains >>>>>> a >>>>>> definitive record of customers, application performance, security >>>>>> threats, fraudulent activity and more. Splunk takes this data and makes >>>>>> sense of it. Business sense. IT sense. Common sense. >>>>>> http://p.sf.net/sfu/splunk-d2d-oct >>>>>> >>>>>> _______________________________________________ >>>>>> Matplotlib-users mailing list >>>>>> Matplotlib-users@... >>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> The demand for IT networking professionals continues to grow, and the >>>>>> demand for specialized networking skills is growing even more rapidly. >>>>>> Take a complimentary Learning@... Self-Assessment and learn >>>>>> about Cisco certifications, training, and career opportunities. >>>>>> http://p.sf.net/sfu/cisco-dev2dev >>>>>> _______________________________________________ >>>>>> Matplotlib-users mailing list >>>>>> Matplotlib-users@... >>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Daniel Hyams >>>>> dhyams@... >>>> >>>> -- >>>> Daniel Hyams >>>> dhyams@... >>>> >>> >> >> ------------------------------------------------------------------------------ >> The demand for IT networking professionals continues to grow, and the >> demand for specialized networking skills is growing even more rapidly. >> Take a complimentary Learning@... Self-Assessment and learn >> about Cisco certifications, training, and career opportunities. >> http://p.sf.net/sfu/cisco-dev2dev >> _______________________________________________ >> Matplotlib-users mailing list >> Matplotlib-users@... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >> >> > > ```

[Matplotlib-users] strange behavior of images when they have an alpha channel [resend] Daniel Hyams <dhyams@gm...>
 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Michael Droettboom - 2011-10-19 18:16 Attachments: Message as HTML ```You are right that Agg is doing the resizing here. Agg expects premultiplied alpha. See [1] for information about what that means. [1] http://en.wikipedia.org/wiki/Alpha_compositing After Agg interpolates the pixel values, to prevent oversaturation it truncates all values to be less than alpha (which makes sense if everything is assumed to be premultiplied alpha). Arguably, the bug here is that nearest neighbor (which doesn't have to do any blending) doesn't perform the truncation step -- then both would look "wrong". It happens in this code snippet in span_image_filter_rgba: (base_mask is 255) if(fg[order_type::A] > base_mask) fg[order_type::A] = base_mask; if(fg[order_type::R] > fg[order_type::A]) fg[order_type::R] = fg[order_type::A]; if(fg[order_type::G] > fg[order_type::A]) fg[order_type::G] = fg[order_type::A]; if(fg[order_type::B] > fg[order_type::A]) fg[order_type::B] = fg[order_type::A]; So, the solution to make a partially transparent image is to not do: pix[:,:,3] = 127 but instead, do pix *= 0.5 Of course, the real fix here is to support alpha blending properly in the image class, then the user wouldn't have to deal with such details. A bug should probably be filed in the matplotlib issue tracker for this. Mike On 10/19/2011 12:23 PM, Daniel Hyams wrote: > [Sorry, I keep getting tripped up with HTML mail....resent in ascii, > and resaved one of the attachment png's to make it smaller.] > > > Example script attached (PIL required). Basically, if I impose a > specific value into an image's alpha channel and use any interpolation > scheme other than 'nearest', there appears gray all where the figure > didn't have any color to begin with. I've also attached a screenshot > of the output of the script on my machine. > > Hopefully I'm doing something wrongly? > > I chased the problem and managed to hack in a solution that fixes the > problem, but it's extremely inefficient...basically, in matplotlib's > image.py, routine BboxImage.make_image, you can create two images > there....one with no alpha channel (call it imRGB) and one with (call > it imRGBA). Go through all of the routine, doing exactly the same > things to both of the images *except* for the interpolation, which is > set to 'nearest' for imRGBA. Then, rip the colors out of imRGB, the > alpha channel off of imRGBA, and put them together....go through all > of the routine again with this composited image, and it works. I > know...I told you it was bad ;) > > The problem seems to be in the "resize" call in that routine...resize, > which calls into C code, does not appear to handle things correctly > when the alpha is anything other than 255's across the board. It > might be a problem in the agg routines, but hopefully it is just maybe > a misuse of the agg routines. > > The behavior seems to be backend independent as far as I could test (I > tried with wxagg and tk backends). I am using mpl 1.0.0 on Windows if > it matters. > > > -- > Daniel Hyams > dhyams@... > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2d-oct > > > _______________________________________________ > Matplotlib-users mailing list > Matplotlib-users@... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Daniel Hyams - 2011-10-19 18:37 ```Ah, thanks so much Michael! That explanation helps a great deal; I was always considering things in "straight alpha" format, not even knowing that there was alternative. I'll play with this tonight; I don't see any problem getting the thing working, though, now that I know what agg expects to see... And yes, alpha support in the image class would be very helpful ;) On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom wrote: > You are right that Agg is doing the resizing here.  Agg expects > premultiplied alpha.  See [1] for information about what that means. > > [1] http://en.wikipedia.org/wiki/Alpha_compositing > > After Agg interpolates the pixel values, to prevent oversaturation it > truncates all values to be less than alpha (which makes sense if everything > is assumed to be premultiplied alpha).  Arguably, the bug here is that > nearest neighbor (which doesn't have to do any blending) doesn't perform the > truncation step -- then both would look "wrong". > > It happens in this code snippet in span_image_filter_rgba: (base_mask is > 255) > >                 if(fg[order_type::A] > base_mask)         fg[order_type::A] > = base_mask; >                 if(fg[order_type::R] > fg[order_type::A]) fg[order_type::R] > = fg[order_type::A]; >                 if(fg[order_type::G] > fg[order_type::A]) fg[order_type::G] > = fg[order_type::A]; >                 if(fg[order_type::B] > fg[order_type::A]) fg[order_type::B] > = fg[order_type::A]; > > So, the solution to make a partially transparent image is to not do: > >     pix[:,:,3] = 127 > > but instead, do > >     pix *= 0.5 > > Of course, the real fix here is to support alpha blending properly in the > image class, then the user wouldn't have to deal with such details.  A bug > should probably be filed in the matplotlib issue tracker for this. > > Mike > > On 10/19/2011 12:23 PM, Daniel Hyams wrote: > > [Sorry, I keep getting tripped up with HTML mail....resent in ascii, > and resaved one of the attachment png's to make it smaller.] > > > Example script attached (PIL required).  Basically, if I impose a > specific value into an image's alpha channel and use any interpolation > scheme other than 'nearest', there appears gray all where the figure > didn't have any color to begin with.   I've also attached a screenshot > of the output of the script on my machine. > > Hopefully I'm doing something wrongly? > > I chased the problem and managed to hack in a solution that fixes the > problem, but it's extremely inefficient...basically, in matplotlib's > image.py, routine BboxImage.make_image, you can create two images > there....one with no alpha channel (call it imRGB) and one with (call > it imRGBA).  Go through all of the routine, doing exactly the same > things to both of the images *except* for the interpolation, which is > set to 'nearest' for imRGBA.  Then, rip the colors out of imRGB, the > alpha channel off of imRGBA, and put them together....go through all > of the routine again with this composited image, and it works.  I > know...I told you it was bad ;) > > The problem seems to be in the "resize" call in that routine...resize, > which calls into C code, does not appear to handle things correctly > when the alpha is anything other than 255's across the board.  It > might be a problem in the agg routines, but hopefully it is just maybe > a misuse of the agg routines. > > The behavior seems to be backend independent as far as I could test (I > tried with wxagg and tk backends).  I am using mpl 1.0.0 on Windows if > it matters. > > > -- > Daniel Hyams > dhyams@... > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2d-oct > > _______________________________________________ > Matplotlib-users mailing list > Matplotlib-users@... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > > ------------------------------------------------------------------------------ > The demand for IT networking professionals continues to grow, and the > demand for specialized networking skills is growing even more rapidly. > Take a complimentary Learning@... Self-Assessment and learn > about Cisco certifications, training, and career opportunities. > http://p.sf.net/sfu/cisco-dev2dev > _______________________________________________ > Matplotlib-users mailing list > Matplotlib-users@... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- Daniel Hyams dhyams@... ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Daniel Hyams - 2011-10-19 23:27 Attachments: image_alpha_bug.py      pagesw.png      premultiplied_alpha_orig_libs.png      straight_alpha_orig_libs.png      straight_alpha_hacked_libs.png ```There has to be something else in play here.  I'll try to keep this short, but the summary is this: I can get the transparency to look right, but only if 1) I put "straight" alpha in image, not premultiplied, and 2) I hack agg to remove specificially every instance of the lines of code that you refer to above. Why this is, I don't know. Hopefully I'm still misusing something. However, it behaves as if the clipping of alpha in the agg library is corrupting the alpha channel.  I also submit that I could have broken some other transparency capabilities of matplotlib, because I don't know what other routines use what I hacked....I did check a few transparent polygons and such though, and everything seemed to be fine. I know that the agg library has been around for quite a long time, so that also means that such a basic bug is unlikely. I've reattached the (slightly modified) script that reproduces the problem, along with a sample image that it uses. The only change to the script is right at the top, where a different image is read, a quick statement is placed to add an alpha channel if there is not already one, and I'm attempting to use premultiplied alphas. I've also attached a screenshot of the output. Notice that in this case, both "transparent" images look wrong. Now, if I 1) hack agg to remove the alpha clipping, and 2) modify the one line in the attached python script so that I use straight alpha, everything looks right. Specifically, I removed every instance of the code below from xxxx, rebuilt all of the matplotlib .so's, and specifically replaced _image.so and _backend_agg.so in my matplotlib distribution. if(fg[order_type::A] > base_mask) fg[order_type::A] = base_mask; if(fg[order_type::R] > fg[order_type::A]) fg[order_type::R] = fg[order_type::A]; if(fg[order_type::G] > fg[order_type::A]) fg[order_type::G] = fg[order_type::A]; if(fg[order_type::B] > fg[order_type::A]) fg[order_type::B] = fg[order_type::A]; On Wed, Oct 19, 2011 at 2:34 PM, Daniel Hyams wrote: > > Ah, thanks so much Michael!  That explanation helps a great deal; I > was always considering things in "straight alpha" format, not even > knowing that there was alternative. > > I'll play with this tonight; I don't see any problem getting the thing > working, though, now that I know what agg expects to see... > > And yes, alpha support in the image class would be very helpful ;) > > On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom wrote: > > You are right that Agg is doing the resizing here.  Agg expects > > premultiplied alpha.  See [1] for information about what that means. > > > > [1] http://en.wikipedia.org/wiki/Alpha_compositing > > > > After Agg interpolates the pixel values, to prevent oversaturation it > > truncates all values to be less than alpha (which makes sense if everything > > is assumed to be premultiplied alpha).  Arguably, the bug here is that > > nearest neighbor (which doesn't have to do any blending) doesn't perform the > > truncation step -- then both would look "wrong". > > > > It happens in this code snippet in span_image_filter_rgba: (base_mask is > > 255) > > > >                 if(fg[order_type::A] > base_mask)         fg[order_type::A] > > = base_mask; > >                 if(fg[order_type::R] > fg[order_type::A]) fg[order_type::R] > > = fg[order_type::A]; > >                 if(fg[order_type::G] > fg[order_type::A]) fg[order_type::G] > > = fg[order_type::A]; > >                 if(fg[order_type::B] > fg[order_type::A]) fg[order_type::B] > > = fg[order_type::A]; > > > > So, the solution to make a partially transparent image is to not do: > > > >     pix[:,:,3] = 127 > > > > but instead, do > > > >     pix *= 0.5 > > > > Of course, the real fix here is to support alpha blending properly in the > > image class, then the user wouldn't have to deal with such details.  A bug > > should probably be filed in the matplotlib issue tracker for this. > > > > Mike > > > > On 10/19/2011 12:23 PM, Daniel Hyams wrote: > > > > [Sorry, I keep getting tripped up with HTML mail....resent in ascii, > > and resaved one of the attachment png's to make it smaller.] > > > > > > Example script attached (PIL required).  Basically, if I impose a > > specific value into an image's alpha channel and use any interpolation > > scheme other than 'nearest', there appears gray all where the figure > > didn't have any color to begin with.   I've also attached a screenshot > > of the output of the script on my machine. > > > > Hopefully I'm doing something wrongly? > > > > I chased the problem and managed to hack in a solution that fixes the > > problem, but it's extremely inefficient...basically, in matplotlib's > > image.py, routine BboxImage.make_image, you can create two images > > there....one with no alpha channel (call it imRGB) and one with (call > > it imRGBA).  Go through all of the routine, doing exactly the same > > things to both of the images *except* for the interpolation, which is > > set to 'nearest' for imRGBA.  Then, rip the colors out of imRGB, the > > alpha channel off of imRGBA, and put them together....go through all > > of the routine again with this composited image, and it works.  I > > know...I told you it was bad ;) > > > > The problem seems to be in the "resize" call in that routine...resize, > > which calls into C code, does not appear to handle things correctly > > when the alpha is anything other than 255's across the board.  It > > might be a problem in the agg routines, but hopefully it is just maybe > > a misuse of the agg routines. > > > > The behavior seems to be backend independent as far as I could test (I > > tried with wxagg and tk backends).  I am using mpl 1.0.0 on Windows if > > it matters. > > > > > > -- > > Daniel Hyams > > dhyams@... > > > > ------------------------------------------------------------------------------ > > All the data continuously generated in your IT infrastructure contains a > > definitive record of customers, application performance, security > > threats, fraudulent activity and more. Splunk takes this data and makes > > sense of it. Business sense. IT sense. Common sense. > > http://p.sf.net/sfu/splunk-d2d-oct > > > > _______________________________________________ > > Matplotlib-users mailing list > > Matplotlib-users@... > > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > > > > > ------------------------------------------------------------------------------ > > The demand for IT networking professionals continues to grow, and the > > demand for specialized networking skills is growing even more rapidly. > > Take a complimentary Learning@... Self-Assessment and learn > > about Cisco certifications, training, and career opportunities. > > http://p.sf.net/sfu/cisco-dev2dev > > _______________________________________________ > > Matplotlib-users mailing list > > Matplotlib-users@... > > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > > > > > > > -- > Daniel Hyams > dhyams@... -- Daniel Hyams dhyams@... ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Daniel Hyams - 2011-10-21 02:29 ```I've looked all over the place through both the Python and C code, and I don't see any premultiplication of alphas at any stage before the pixels are passed off to agg, and neither can I find any place where the alphas are "unmultiplied" on the way back from agg to the backend for rendering. It's very possible that I missed it, but I would have to miss it in two places (premultiply and the unmultiply). It looks to me like the output from agg ends up getting passed on directly to the renderer, which as far as I know, just uses straight alpha. The WxAgg renderer, for example, just creates a wx.Bitmap out of the pixels and blits it. Which means that any image going through agg's filters will not be correct if it has any pixels with alpha != 0 or != 255. [Using PIL images because they are simple to talk about...but the PIL image could alternatively be an image.py image] As far as I can tell, the image pixels current go through a pipeline like the following: [1] PIL Image -> _image image -> agg operations -> modified and/or resized _image image -> renderer If agg expects premultiplied alpha, the procedure should look something like: [2] PIL Image -> _image image -> premultiply alphas ->agg options -> unmultiply alphas -> modified and/or resized _image image -> renderer I personally don't like pipeline [2] because picture detail is lost in the "unmultiply alphas" stage. Better to use straight alpha all the way through. So long as matplotlib is using only a subset of agg algorithms that work no matter whether the alphas are premultiplied or not, I would think that the most reasonable route was the one that I took; to always pass straight alphas (sticking with pipeline [1]), and modify the agg source slightly to fit matplotlib's approach (i.e., remove the clipping there). I hope that I'm not way off base (I have a sneaking feeling that I am :O ), and hope this helps. I've verified on both Linux and Windows that removing the alpha-clip lines from agg_span_image_filter_rgba.h, rebuilding matplotlib, and replacing _image.so/_image.pyd and _backend_agg.so/_backend_agg.pyd does the trick (along with passing straight alphas). So far, I've seen no ill effects on any of my plots, but I'm also not in a position to run the pixel-by-pixel comparison matplotlib tests. On Wed, Oct 19, 2011 at 7:26 PM, Daniel Hyams wrote: > There has to be something else in play here.  I'll try to keep this > short, but the summary is this: I can get the transparency to look > right, but only if 1) I put "straight" alpha in image, not > premultiplied, and 2) I hack agg to remove specificially every > instance of the lines of code that you refer to above. > > Why this is, I don't know.  Hopefully I'm still misusing something. > However, it behaves as if the clipping of alpha in the agg library is > corrupting the alpha channel.  I also submit that I could have broken > some other transparency capabilities of matplotlib, because I don't > know what other routines use what I hacked....I did check a few > transparent polygons and such though, and everything seemed to be > fine. > > I know that the agg library has been around for quite a long time, so > that also means that such a basic bug is unlikely. > > I've reattached the (slightly modified) script that reproduces the > problem, along with a sample image that it uses.  The only change to > the script is right at the top, where a different image is read, a > quick statement is placed to add an alpha channel if there is not > already one, and I'm attempting to use premultiplied alphas.  I've > also attached a screenshot of the output.  Notice that in this case, > both "transparent" images look wrong. > > Now, if I 1) hack agg to remove the alpha clipping, and 2) modify the > one line in the attached python script so that I use straight alpha, > everything looks right.  Specifically, I removed every instance of the > code below from xxxx, rebuilt all of the matplotlib .so's, and > specifically replaced _image.so and _backend_agg.so in my matplotlib > distribution. > >           if(fg[order_type::A] > base_mask)         fg[order_type::A] > = base_mask; >                if(fg[order_type::R] > fg[order_type::A]) > fg[order_type::R] = fg[order_type::A]; >                if(fg[order_type::G] > fg[order_type::A]) > fg[order_type::G] = fg[order_type::A]; >                if(fg[order_type::B] > fg[order_type::A]) > fg[order_type::B] = fg[order_type::A]; > > > > > On Wed, Oct 19, 2011 at 2:34 PM, Daniel Hyams wrote: >> >> Ah, thanks so much Michael!  That explanation helps a great deal; I >> was always considering things in "straight alpha" format, not even >> knowing that there was alternative. >> >> I'll play with this tonight; I don't see any problem getting the thing >> working, though, now that I know what agg expects to see... >> >> And yes, alpha support in the image class would be very helpful ;) >> >> On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom wrote: >> > You are right that Agg is doing the resizing here.  Agg expects >> > premultiplied alpha.  See [1] for information about what that means. >> > >> > [1] http://en.wikipedia.org/wiki/Alpha_compositing >> > >> > After Agg interpolates the pixel values, to prevent oversaturation it >> > truncates all values to be less than alpha (which makes sense if everything >> > is assumed to be premultiplied alpha).  Arguably, the bug here is that >> > nearest neighbor (which doesn't have to do any blending) doesn't perform the >> > truncation step -- then both would look "wrong". >> > >> > It happens in this code snippet in span_image_filter_rgba: (base_mask is >> > 255) >> > >> >                 if(fg[order_type::A] > base_mask)         fg[order_type::A] >> > = base_mask; >> >                 if(fg[order_type::R] > fg[order_type::A]) fg[order_type::R] >> > = fg[order_type::A]; >> >                 if(fg[order_type::G] > fg[order_type::A]) fg[order_type::G] >> > = fg[order_type::A]; >> >                 if(fg[order_type::B] > fg[order_type::A]) fg[order_type::B] >> > = fg[order_type::A]; >> > >> > So, the solution to make a partially transparent image is to not do: >> > >> >     pix[:,:,3] = 127 >> > >> > but instead, do >> > >> >     pix *= 0.5 >> > >> > Of course, the real fix here is to support alpha blending properly in the >> > image class, then the user wouldn't have to deal with such details.  A bug >> > should probably be filed in the matplotlib issue tracker for this. >> > >> > Mike >> > >> > On 10/19/2011 12:23 PM, Daniel Hyams wrote: >> > >> > [Sorry, I keep getting tripped up with HTML mail....resent in ascii, >> > and resaved one of the attachment png's to make it smaller.] >> > >> > >> > Example script attached (PIL required).  Basically, if I impose a >> > specific value into an image's alpha channel and use any interpolation >> > scheme other than 'nearest', there appears gray all where the figure >> > didn't have any color to begin with.   I've also attached a screenshot >> > of the output of the script on my machine. >> > >> > Hopefully I'm doing something wrongly? >> > >> > I chased the problem and managed to hack in a solution that fixes the >> > problem, but it's extremely inefficient...basically, in matplotlib's >> > image.py, routine BboxImage.make_image, you can create two images >> > there....one with no alpha channel (call it imRGB) and one with (call >> > it imRGBA).  Go through all of the routine, doing exactly the same >> > things to both of the images *except* for the interpolation, which is >> > set to 'nearest' for imRGBA.  Then, rip the colors out of imRGB, the >> > alpha channel off of imRGBA, and put them together....go through all >> > of the routine again with this composited image, and it works.  I >> > know...I told you it was bad ;) >> > >> > The problem seems to be in the "resize" call in that routine...resize, >> > which calls into C code, does not appear to handle things correctly >> > when the alpha is anything other than 255's across the board.  It >> > might be a problem in the agg routines, but hopefully it is just maybe >> > a misuse of the agg routines. >> > >> > The behavior seems to be backend independent as far as I could test (I >> > tried with wxagg and tk backends).  I am using mpl 1.0.0 on Windows if >> > it matters. >> > >> > >> > -- >> > Daniel Hyams >> > dhyams@... >> > >> > ------------------------------------------------------------------------------ >> > All the data continuously generated in your IT infrastructure contains a >> > definitive record of customers, application performance, security >> > threats, fraudulent activity and more. Splunk takes this data and makes >> > sense of it. Business sense. IT sense. Common sense. >> > http://p.sf.net/sfu/splunk-d2d-oct >> > >> > _______________________________________________ >> > Matplotlib-users mailing list >> > Matplotlib-users@... >> > https://lists.sourceforge.net/lists/listinfo/matplotlib-users >> > >> > >> > ------------------------------------------------------------------------------ >> > The demand for IT networking professionals continues to grow, and the >> > demand for specialized networking skills is growing even more rapidly. >> > Take a complimentary Learning@... Self-Assessment and learn >> > about Cisco certifications, training, and career opportunities. >> > http://p.sf.net/sfu/cisco-dev2dev >> > _______________________________________________ >> > Matplotlib-users mailing list >> > Matplotlib-users@... >> > https://lists.sourceforge.net/lists/listinfo/matplotlib-users >> > >> > >> >> >> >> -- >> Daniel Hyams >> dhyams@... > > > > -- > Daniel Hyams > dhyams@... > -- Daniel Hyams dhyams@... ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Michael Droettboom - 2011-10-21 13:31 Attachments: image_pre.diff ```Thanks for looking into this deeper. Agg requires image buffers to be premultiplied, as described in the third bullet point here. (It's not exactly clear, to say the least, but that's what I take it to mean, and also from reading the code). http://www.antigrain.com/news/release_notes/v22.agdoc.html The bug is that in _image.cpp the input buffers are not declared as premultiplied in _image.cpp. Arguably it is a bug that agg doesn't reject filtering unmultiplied images, since the note states that the assumption is that they are premultiplied by the time they get to the filters. I have attached a patch that fixes this. Would you mind testing it and let me know how it works for you? On 10/20/2011 10:29 PM, Daniel Hyams wrote: > I've looked all over the place through both the Python and C code, and > I don't see any premultiplication of alphas at any stage before the > pixels are passed off to agg, and neither can I find any place where > the alphas are "unmultiplied" on the way back from agg to the backend > for rendering. matplotlib's support for alpha blending of images is basically by accident, so it's not surprising the details aren't right. I think it is a bug that after reading images in we don't premultiply them before sending them to Agg. That bug has existed for a long time in matplotlib because no one is really using alpha images a great deal. (Masked images, yes, but that implies alpha is strictly 0 or 255 and thus these issues don't come into play.) Unmultiplying is not always necessary. Many of the GUI backends also expect premultiplied alpha (Qt for example). However, there is certainly a bug in writing PNG files (where the file format specifies unmultiplied). > It's very possible that I missed it, but I would have to miss it in > two places (premultiply and the unmultiply). It looks to me like the > output from agg ends up getting passed on directly to the renderer, > which as far as I know, just uses straight alpha. The WxAgg renderer, > for example, just creates a wx.Bitmap out of the pixels and blits it. > Which means that any image going through agg's filters will not be > correct if it has any pixels with alpha != 0 or != 255. > > [Using PIL images because they are simple to talk about...but the PIL > image could alternatively be an image.py image] > > As far as I can tell, the image pixels current go through a pipeline > like the following: > > [1] PIL Image -> _image image -> agg operations -> modified and/or > resized _image image -> renderer > > If agg expects premultiplied alpha, the procedure should look something like: > > [2] PIL Image -> _image image -> premultiply alphas ->agg options -> > unmultiply alphas -> modified and/or resized _image image -> renderer > > I personally don't like pipeline [2] because picture detail is lost in > the "unmultiply alphas" stage. Better to use straight alpha all the > way through. I think what needed is: [3] PIL Image (or _png.cpp) -> premultiply alphas -> _image image -> agg options -> -> modified and/or resized _image image -> renderer -> (unmultiply alphas)? -> GUI library That is -- all image data should be kept premultiplied internally in all buffers for efficiency and because this is what Agg is designed for. Can you explain what you mean by "picture detail is lost in the unmultiply alphas stage". There is the usual problem that by premultiplying you lose any color data where alpha = 0 (and you lose resolution everywhere else, but not resolution you can actually see after compositing). > So long as matplotlib is using only a subset of agg algorithms that > work no matter whether the alphas are premultiplied or not, I would > think that the most reasonable route was the one that I took; to > always pass straight alphas (sticking with pipeline [1]), and modify > the agg source slightly to fit matplotlib's approach (i.e., remove the > clipping there). I'd be really wary of modifying agg like this. Those things become hard to maintain. I think this instead a bug in matplotlib and should be fixed there. I've put an issue in the issue tracker here: https://github.com/matplotlib/matplotlib/issues/545 Cheers, Mike > I hope that I'm not way off base (I have a sneaking feeling that I am > :O ), and hope this helps. I've verified on both Linux and Windows > that removing the alpha-clip lines from agg_span_image_filter_rgba.h, > rebuilding matplotlib, and replacing _image.so/_image.pyd and > _backend_agg.so/_backend_agg.pyd does the trick (along with passing > straight alphas). So far, I've seen no ill effects on any of my > plots, but I'm also not in a position to run the pixel-by-pixel > comparison matplotlib tests. > > > On Wed, Oct 19, 2011 at 7:26 PM, Daniel Hyams wrote: >> There has to be something else in play here. I'll try to keep this >> short, but the summary is this: I can get the transparency to look >> right, but only if 1) I put "straight" alpha in image, not >> premultiplied, and 2) I hack agg to remove specificially every >> instance of the lines of code that you refer to above. >> >> Why this is, I don't know. Hopefully I'm still misusing something. >> However, it behaves as if the clipping of alpha in the agg library is >> corrupting the alpha channel. I also submit that I could have broken >> some other transparency capabilities of matplotlib, because I don't >> know what other routines use what I hacked....I did check a few >> transparent polygons and such though, and everything seemed to be >> fine. >> >> I know that the agg library has been around for quite a long time, so >> that also means that such a basic bug is unlikely. >> >> I've reattached the (slightly modified) script that reproduces the >> problem, along with a sample image that it uses. The only change to >> the script is right at the top, where a different image is read, a >> quick statement is placed to add an alpha channel if there is not >> already one, and I'm attempting to use premultiplied alphas. I've >> also attached a screenshot of the output. Notice that in this case, >> both "transparent" images look wrong. >> >> Now, if I 1) hack agg to remove the alpha clipping, and 2) modify the >> one line in the attached python script so that I use straight alpha, >> everything looks right. Specifically, I removed every instance of the >> code below from xxxx, rebuilt all of the matplotlib .so's, and >> specifically replaced _image.so and _backend_agg.so in my matplotlib >> distribution. >> >> if(fg[order_type::A]> base_mask) fg[order_type::A] >> = base_mask; >> if(fg[order_type::R]> fg[order_type::A]) >> fg[order_type::R] = fg[order_type::A]; >> if(fg[order_type::G]> fg[order_type::A]) >> fg[order_type::G] = fg[order_type::A]; >> if(fg[order_type::B]> fg[order_type::A]) >> fg[order_type::B] = fg[order_type::A]; >> >> >> >> >> On Wed, Oct 19, 2011 at 2:34 PM, Daniel Hyams wrote: >>> Ah, thanks so much Michael! That explanation helps a great deal; I >>> was always considering things in "straight alpha" format, not even >>> knowing that there was alternative. >>> >>> I'll play with this tonight; I don't see any problem getting the thing >>> working, though, now that I know what agg expects to see... >>> >>> And yes, alpha support in the image class would be very helpful ;) >>> >>> On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom wrote: >>>> You are right that Agg is doing the resizing here. Agg expects >>>> premultiplied alpha. See [1] for information about what that means. >>>> >>>> [1] http://en.wikipedia.org/wiki/Alpha_compositing >>>> >>>> After Agg interpolates the pixel values, to prevent oversaturation it >>>> truncates all values to be less than alpha (which makes sense if everything >>>> is assumed to be premultiplied alpha). Arguably, the bug here is that >>>> nearest neighbor (which doesn't have to do any blending) doesn't perform the >>>> truncation step -- then both would look "wrong". >>>> >>>> It happens in this code snippet in span_image_filter_rgba: (base_mask is >>>> 255) >>>> >>>> if(fg[order_type::A]> base_mask) fg[order_type::A] >>>> = base_mask; >>>> if(fg[order_type::R]> fg[order_type::A]) fg[order_type::R] >>>> = fg[order_type::A]; >>>> if(fg[order_type::G]> fg[order_type::A]) fg[order_type::G] >>>> = fg[order_type::A]; >>>> if(fg[order_type::B]> fg[order_type::A]) fg[order_type::B] >>>> = fg[order_type::A]; >>>> >>>> So, the solution to make a partially transparent image is to not do: >>>> >>>> pix[:,:,3] = 127 >>>> >>>> but instead, do >>>> >>>> pix *= 0.5 >>>> >>>> Of course, the real fix here is to support alpha blending properly in the >>>> image class, then the user wouldn't have to deal with such details. A bug >>>> should probably be filed in the matplotlib issue tracker for this. >>>> >>>> Mike >>>> >>>> On 10/19/2011 12:23 PM, Daniel Hyams wrote: >>>> >>>> [Sorry, I keep getting tripped up with HTML mail....resent in ascii, >>>> and resaved one of the attachment png's to make it smaller.] >>>> >>>> >>>> Example script attached (PIL required). Basically, if I impose a >>>> specific value into an image's alpha channel and use any interpolation >>>> scheme other than 'nearest', there appears gray all where the figure >>>> didn't have any color to begin with. I've also attached a screenshot >>>> of the output of the script on my machine. >>>> >>>> Hopefully I'm doing something wrongly? >>>> >>>> I chased the problem and managed to hack in a solution that fixes the >>>> problem, but it's extremely inefficient...basically, in matplotlib's >>>> image.py, routine BboxImage.make_image, you can create two images >>>> there....one with no alpha channel (call it imRGB) and one with (call >>>> it imRGBA). Go through all of the routine, doing exactly the same >>>> things to both of the images *except* for the interpolation, which is >>>> set to 'nearest' for imRGBA. Then, rip the colors out of imRGB, the >>>> alpha channel off of imRGBA, and put them together....go through all >>>> of the routine again with this composited image, and it works. I >>>> know...I told you it was bad ;) >>>> >>>> The problem seems to be in the "resize" call in that routine...resize, >>>> which calls into C code, does not appear to handle things correctly >>>> when the alpha is anything other than 255's across the board. It >>>> might be a problem in the agg routines, but hopefully it is just maybe >>>> a misuse of the agg routines. >>>> >>>> The behavior seems to be backend independent as far as I could test (I >>>> tried with wxagg and tk backends). I am using mpl 1.0.0 on Windows if >>>> it matters. >>>> >>>> >>>> -- >>>> Daniel Hyams >>>> dhyams@... >>>> >>>> ------------------------------------------------------------------------------ >>>> All the data continuously generated in your IT infrastructure contains a >>>> definitive record of customers, application performance, security >>>> threats, fraudulent activity and more. Splunk takes this data and makes >>>> sense of it. Business sense. IT sense. Common sense. >>>> http://p.sf.net/sfu/splunk-d2d-oct >>>> >>>> _______________________________________________ >>>> Matplotlib-users mailing list >>>> Matplotlib-users@... >>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> The demand for IT networking professionals continues to grow, and the >>>> demand for specialized networking skills is growing even more rapidly. >>>> Take a complimentary Learning@... Self-Assessment and learn >>>> about Cisco certifications, training, and career opportunities. >>>> http://p.sf.net/sfu/cisco-dev2dev >>>> _______________________________________________ >>>> Matplotlib-users mailing list >>>> Matplotlib-users@... >>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>> >>>> >>> >>> >>> -- >>> Daniel Hyams >>> dhyams@... >> >> >> -- >> Daniel Hyams >> dhyams@... >> > > ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Daniel Hyams - 2011-10-21 13:50 ```All sounds reasonable Mike; I do agree that patching the agg source code is not that desirable; I was operating under the (incorrect) assumption that most, if not all, backends used straight alpha. I'll certainly test the patch tonight, but I can only test it under wxAgg reasonably, which is one of the backends that expects straight alpha as far as I know. The "loss in picture detail" comment was just discomfort with the fact that once the alphas are premultiplied in, there is not an exact reverse transformation to get your original color back. In googling around to better explain here, I found this, which is a much more in depth and better explanation than what I would have come up with: http://www.quasimondo.com/archives/000665.php The crux is here: >An example: when you set the alpha value of a pixel to 16 all color values will be multiplied with >a factor of 16/256 = 0.0625. So a gray pixel of 128 will become 128 * 0.0625 = 8, a darker pixel >of 64 will become 64 * 0.0625 = 4. But a slightly lighter pixel of maybe 67 will become 67 * > 0.0625 = 4.1875 - yet there are no decimals in integer pixels which means it will also become > 4. The effect that you will get posterization - setting your alpha channel to 8 means that you > also reduce your color channels to 8 levels, this means instead = 256*256*256 different colors > you will end up with a maximum of 8*8*8 = 512 different colors. One might argue that the loss of color information is not that crucial, because for very low alpha (where the problem is most pronounced), the image is almost invisible anyway... so it won't matter. That's true, but what if I want to (for whatever reason) take the image's pixels again, before draw, and boost the alpha up again? What I'll get is a posterized mess. So, I'm still of the opinion that patching agg in this situation might be the best solution to this. This way, straight alphas are used throughout, and for backends that require premultiplied alpha, the alpha can be premultiplied in at the latest possible moment. Thanks for all of the help with this Mike, Daniel On Fri, Oct 21, 2011 at 9:31 AM, Michael Droettboom wrote: > Thanks for looking into this deeper. > > Agg requires image buffers to be premultiplied, as described in the third > bullet point here.  (It's not exactly clear, to say the least, but that's > what I take it to mean, and also from reading the code). > > http://www.antigrain.com/news/release_notes/v22.agdoc.html > > The bug is that in _image.cpp the input buffers are not declared as > premultiplied in _image.cpp.  Arguably it is a bug that agg doesn't reject > filtering unmultiplied images, since the note states that the assumption is > that they are premultiplied by the time they get to the filters. > > I have attached a patch that fixes this.  Would you mind testing it and let > me know how it works for you? > > On 10/20/2011 10:29 PM, Daniel Hyams wrote: >> >> I've looked all over the place through both the Python and C code, and >> I don't see any premultiplication of alphas at any stage before the >> pixels are passed off to agg, and neither can I find any place where >> the alphas are "unmultiplied" on the way back from agg to the backend >> for rendering. > > matplotlib's support for alpha blending of images is basically by accident, > so it's not surprising the details aren't right. > > I think it is a bug that after reading images in we don't premultiply them > before sending them to Agg.  That bug has existed for a long time in > matplotlib because no one is really using alpha images a great deal. >  (Masked images, yes, but that implies alpha is strictly 0 or 255 and thus > these issues don't come into play.) > > Unmultiplying is not always necessary.  Many of the GUI backends also expect > premultiplied alpha (Qt for example).  However, there is certainly a bug in > writing PNG files (where the file format specifies unmultiplied). > >> It's very possible that I missed it, but I would have to miss it in >> two places (premultiply and the unmultiply).  It looks to me like the >> output from agg ends up getting passed on directly to the renderer, >> which as far as I know, just uses straight alpha. The WxAgg renderer, >> for example, just creates a wx.Bitmap out of the pixels and blits it. >> Which means that any image going through agg's filters will not be >> correct if it has any pixels with alpha != 0 or != 255. >> >> [Using PIL images because they are simple to talk about...but the PIL >> image could alternatively be an image.py image] >> >> As far as I can tell, the image pixels current go through a pipeline >> like the following: >> >> [1] PIL Image ->  _image image ->  agg operations ->  modified and/or >> resized _image image ->  renderer >> >> If agg expects premultiplied alpha, the procedure should look something >> like: >> >> [2] PIL Image ->  _image image ->  premultiply alphas ->agg options -> >> unmultiply alphas ->  modified and/or resized _image image ->  renderer >> >> I personally don't like pipeline [2] because picture detail is lost in >> the "unmultiply alphas" stage.  Better to use straight alpha all the >> way through. > > I think what needed is: > > [3] PIL Image (or _png.cpp) ->  premultiply alphas ->  _image image ->  agg > options -> > ->  modified and/or resized _image image ->  renderer ->  (unmultiply > alphas)? ->  GUI library > > That is -- all image data should be kept premultiplied internally in all > buffers for efficiency and because this is what Agg is designed for. > > Can you explain what you mean by "picture detail is lost in the unmultiply > alphas stage".  There is the usual problem that by premultiplying you lose > any color data where alpha = 0 (and you lose resolution everywhere else, but > not resolution you can actually see after compositing). >> >> So long as matplotlib is using only a subset of agg algorithms that >> work no matter whether the alphas are premultiplied or not, I would >> think that the most reasonable route was the one that I took; to >> always pass straight alphas (sticking with pipeline [1]), and modify >> the agg source slightly to fit matplotlib's approach (i.e., remove the >> clipping there). > > I'd be really wary of modifying agg like this.  Those things become hard to > maintain.  I think this instead a bug in matplotlib and should be fixed > there. > > I've put an issue in the issue tracker here: > > https://github.com/matplotlib/matplotlib/issues/545 > > Cheers, > Mike > >> I hope that I'm not way off base (I have a sneaking feeling that I am >> :O ), and hope this helps.  I've verified on both Linux and Windows >> that removing the alpha-clip lines from agg_span_image_filter_rgba.h, >> rebuilding matplotlib, and replacing _image.so/_image.pyd and >> _backend_agg.so/_backend_agg.pyd does the trick (along with passing >> straight alphas).  So far, I've seen no ill effects on any of my >> plots, but I'm also not in a position to run the pixel-by-pixel >> comparison matplotlib tests. >> >> >> On Wed, Oct 19, 2011 at 7:26 PM, Daniel Hyams  wrote: >>> >>> There has to be something else in play here.  I'll try to keep this >>> short, but the summary is this: I can get the transparency to look >>> right, but only if 1) I put "straight" alpha in image, not >>> premultiplied, and 2) I hack agg to remove specificially every >>> instance of the lines of code that you refer to above. >>> >>> Why this is, I don't know.  Hopefully I'm still misusing something. >>> However, it behaves as if the clipping of alpha in the agg library is >>> corrupting the alpha channel.  I also submit that I could have broken >>> some other transparency capabilities of matplotlib, because I don't >>> know what other routines use what I hacked....I did check a few >>> transparent polygons and such though, and everything seemed to be >>> fine. >>> >>> I know that the agg library has been around for quite a long time, so >>> that also means that such a basic bug is unlikely. >>> >>> I've reattached the (slightly modified) script that reproduces the >>> problem, along with a sample image that it uses.  The only change to >>> the script is right at the top, where a different image is read, a >>> quick statement is placed to add an alpha channel if there is not >>> already one, and I'm attempting to use premultiplied alphas.  I've >>> also attached a screenshot of the output.  Notice that in this case, >>> both "transparent" images look wrong. >>> >>> Now, if I 1) hack agg to remove the alpha clipping, and 2) modify the >>> one line in the attached python script so that I use straight alpha, >>> everything looks right.  Specifically, I removed every instance of the >>> code below from xxxx, rebuilt all of the matplotlib .so's, and >>> specifically replaced _image.so and _backend_agg.so in my matplotlib >>> distribution. >>> >>>           if(fg[order_type::A]>  base_mask)         fg[order_type::A] >>> = base_mask; >>>                if(fg[order_type::R]>  fg[order_type::A]) >>> fg[order_type::R] = fg[order_type::A]; >>>                if(fg[order_type::G]>  fg[order_type::A]) >>> fg[order_type::G] = fg[order_type::A]; >>>                if(fg[order_type::B]>  fg[order_type::A]) >>> fg[order_type::B] = fg[order_type::A]; >>> >>> >>> >>> >>> On Wed, Oct 19, 2011 at 2:34 PM, Daniel Hyams  wrote: >>>> >>>> Ah, thanks so much Michael!  That explanation helps a great deal; I >>>> was always considering things in "straight alpha" format, not even >>>> knowing that there was alternative. >>>> >>>> I'll play with this tonight; I don't see any problem getting the thing >>>> working, though, now that I know what agg expects to see... >>>> >>>> And yes, alpha support in the image class would be very helpful ;) >>>> >>>> On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom >>>>  wrote: >>>>> >>>>> You are right that Agg is doing the resizing here.  Agg expects >>>>> premultiplied alpha.  See [1] for information about what that means. >>>>> >>>>> [1] http://en.wikipedia.org/wiki/Alpha_compositing >>>>> >>>>> After Agg interpolates the pixel values, to prevent oversaturation it >>>>> truncates all values to be less than alpha (which makes sense if >>>>> everything >>>>> is assumed to be premultiplied alpha).  Arguably, the bug here is that >>>>> nearest neighbor (which doesn't have to do any blending) doesn't >>>>> perform the >>>>> truncation step -- then both would look "wrong". >>>>> >>>>> It happens in this code snippet in span_image_filter_rgba: (base_mask >>>>> is >>>>> 255) >>>>> >>>>>                 if(fg[order_type::A]>  base_mask) >>>>> fg[order_type::A] >>>>> = base_mask; >>>>>                 if(fg[order_type::R]>  fg[order_type::A]) >>>>> fg[order_type::R] >>>>> = fg[order_type::A]; >>>>>                 if(fg[order_type::G]>  fg[order_type::A]) >>>>> fg[order_type::G] >>>>> = fg[order_type::A]; >>>>>                 if(fg[order_type::B]>  fg[order_type::A]) >>>>> fg[order_type::B] >>>>> = fg[order_type::A]; >>>>> >>>>> So, the solution to make a partially transparent image is to not do: >>>>> >>>>>     pix[:,:,3] = 127 >>>>> >>>>> but instead, do >>>>> >>>>>     pix *= 0.5 >>>>> >>>>> Of course, the real fix here is to support alpha blending properly in >>>>> the >>>>> image class, then the user wouldn't have to deal with such details.  A >>>>> bug >>>>> should probably be filed in the matplotlib issue tracker for this. >>>>> >>>>> Mike >>>>> >>>>> On 10/19/2011 12:23 PM, Daniel Hyams wrote: >>>>> >>>>> [Sorry, I keep getting tripped up with HTML mail....resent in ascii, >>>>> and resaved one of the attachment png's to make it smaller.] >>>>> >>>>> >>>>> Example script attached (PIL required).  Basically, if I impose a >>>>> specific value into an image's alpha channel and use any interpolation >>>>> scheme other than 'nearest', there appears gray all where the figure >>>>> didn't have any color to begin with.   I've also attached a screenshot >>>>> of the output of the script on my machine. >>>>> >>>>> Hopefully I'm doing something wrongly? >>>>> >>>>> I chased the problem and managed to hack in a solution that fixes the >>>>> problem, but it's extremely inefficient...basically, in matplotlib's >>>>> image.py, routine BboxImage.make_image, you can create two images >>>>> there....one with no alpha channel (call it imRGB) and one with (call >>>>> it imRGBA).  Go through all of the routine, doing exactly the same >>>>> things to both of the images *except* for the interpolation, which is >>>>> set to 'nearest' for imRGBA.  Then, rip the colors out of imRGB, the >>>>> alpha channel off of imRGBA, and put them together....go through all >>>>> of the routine again with this composited image, and it works.  I >>>>> know...I told you it was bad ;) >>>>> >>>>> The problem seems to be in the "resize" call in that routine...resize, >>>>> which calls into C code, does not appear to handle things correctly >>>>> when the alpha is anything other than 255's across the board.  It >>>>> might be a problem in the agg routines, but hopefully it is just maybe >>>>> a misuse of the agg routines. >>>>> >>>>> The behavior seems to be backend independent as far as I could test (I >>>>> tried with wxagg and tk backends).  I am using mpl 1.0.0 on Windows if >>>>> it matters. >>>>> >>>>> >>>>> -- >>>>> Daniel Hyams >>>>> dhyams@... >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> All the data continuously generated in your IT infrastructure contains >>>>> a >>>>> definitive record of customers, application performance, security >>>>> threats, fraudulent activity and more. Splunk takes this data and makes >>>>> sense of it. Business sense. IT sense. Common sense. >>>>> http://p.sf.net/sfu/splunk-d2d-oct >>>>> >>>>> _______________________________________________ >>>>> Matplotlib-users mailing list >>>>> Matplotlib-users@... >>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> The demand for IT networking professionals continues to grow, and the >>>>> demand for specialized networking skills is growing even more rapidly. >>>>> Take a complimentary Learning@... Self-Assessment and learn >>>>> about Cisco certifications, training, and career opportunities. >>>>> http://p.sf.net/sfu/cisco-dev2dev >>>>> _______________________________________________ >>>>> Matplotlib-users mailing list >>>>> Matplotlib-users@... >>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>>> >>>>> >>>> >>>> >>>> -- >>>> Daniel Hyams >>>> dhyams@... >>> >>> >>> -- >>> Daniel Hyams >>> dhyams@... >>> >> >> > > > ------------------------------------------------------------------------------ > The demand for IT networking professionals continues to grow, and the > demand for specialized networking skills is growing even more rapidly. > Take a complimentary Learning@... Self-Assessment and learn > about Cisco certifications, training, and career opportunities. > http://p.sf.net/sfu/cisco-dev2dev > _______________________________________________ > Matplotlib-users mailing list > Matplotlib-users@... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- Daniel Hyams dhyams@... ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Michael Droettboom - 2011-10-21 15:21 ```On 10/21/2011 09:49 AM, Daniel Hyams wrote: > All sounds reasonable Mike; I do agree that patching the agg source > code is not that desirable; I was operating under the (incorrect) > assumption that most, if not all, backends used straight alpha. > > I'll certainly test the patch tonight, but I can only test it under > wxAgg reasonably, which is one of the backends that expects straight > alpha as far as I know. > > The "loss in picture detail" comment was just discomfort with the fact > that once the alphas are premultiplied in, there is not an exact > reverse transformation to get your original color back. In googling > around to better explain here, I found this, which is a much more in > depth and better explanation than what I would have come up with: > > http://www.quasimondo.com/archives/000665.php > > The crux is here: > >> An example: when you set the alpha value of a pixel to 16 all color values will be multiplied with >> a factor of 16/256 = 0.0625. So a gray pixel of 128 will become 128 * 0.0625 = 8, a darker pixel >> of 64 will become 64 * 0.0625 = 4. But a slightly lighter pixel of maybe 67 will become 67 * >> 0.0625 = 4.1875 - yet there are no decimals in integer pixels which means it will also become >> 4. The effect that you will get posterization - setting your alpha channel to 8 means that you >> also reduce your color channels to 8 levels, this means instead = 256*256*256 different colors >> you will end up with a maximum of 8*8*8 = 512 different colors. > One might argue that the loss of color information is not that > crucial, because for very low alpha (where the problem is most > pronounced), the image is almost invisible anyway... so it won't > matter. That's true, but what if I want to (for whatever reason) take > the image's pixels again, before draw, and boost the alpha up again? > What I'll get is a posterized mess. So, I'm still of the opinion that > patching agg in this situation might be the best solution to this. > This way, straight alphas are used throughout, and for backends that > require premultiplied alpha, the alpha can be premultiplied in at the > latest possible moment. Thanks for clarifying. I understand what you're saying now. I think what we want to do is store the unmultiplied alpha as a "canonical" version of the image, and premultiply a copy (or use some C++ iterator magic to avoid the copy) right before sending it off to Agg. Then the alpha can be fully "tweakable" at runtime. I'll try to tackle this problem, as well as the problem that set_alpha simply doesn't work, at the same time when I get a chance (or patches are always welcome, of course :). Mike > Thanks for all of the help with this Mike, > > Daniel > > On Fri, Oct 21, 2011 at 9:31 AM, Michael Droettboom wrote: >> Thanks for looking into this deeper. >> >> Agg requires image buffers to be premultiplied, as described in the third >> bullet point here. (It's not exactly clear, to say the least, but that's >> what I take it to mean, and also from reading the code). >> >> http://www.antigrain.com/news/release_notes/v22.agdoc.html >> >> The bug is that in _image.cpp the input buffers are not declared as >> premultiplied in _image.cpp. Arguably it is a bug that agg doesn't reject >> filtering unmultiplied images, since the note states that the assumption is >> that they are premultiplied by the time they get to the filters. >> >> I have attached a patch that fixes this. Would you mind testing it and let >> me know how it works for you? >> >> On 10/20/2011 10:29 PM, Daniel Hyams wrote: >>> I've looked all over the place through both the Python and C code, and >>> I don't see any premultiplication of alphas at any stage before the >>> pixels are passed off to agg, and neither can I find any place where >>> the alphas are "unmultiplied" on the way back from agg to the backend >>> for rendering. >> matplotlib's support for alpha blending of images is basically by accident, >> so it's not surprising the details aren't right. >> >> I think it is a bug that after reading images in we don't premultiply them >> before sending them to Agg. That bug has existed for a long time in >> matplotlib because no one is really using alpha images a great deal. >> (Masked images, yes, but that implies alpha is strictly 0 or 255 and thus >> these issues don't come into play.) >> >> Unmultiplying is not always necessary. Many of the GUI backends also expect >> premultiplied alpha (Qt for example). However, there is certainly a bug in >> writing PNG files (where the file format specifies unmultiplied). >> >>> It's very possible that I missed it, but I would have to miss it in >>> two places (premultiply and the unmultiply). It looks to me like the >>> output from agg ends up getting passed on directly to the renderer, >>> which as far as I know, just uses straight alpha. The WxAgg renderer, >>> for example, just creates a wx.Bitmap out of the pixels and blits it. >>> Which means that any image going through agg's filters will not be >>> correct if it has any pixels with alpha != 0 or != 255. >>> >>> [Using PIL images because they are simple to talk about...but the PIL >>> image could alternatively be an image.py image] >>> >>> As far as I can tell, the image pixels current go through a pipeline >>> like the following: >>> >>> [1] PIL Image -> _image image -> agg operations -> modified and/or >>> resized _image image -> renderer >>> >>> If agg expects premultiplied alpha, the procedure should look something >>> like: >>> >>> [2] PIL Image -> _image image -> premultiply alphas ->agg options -> >>> unmultiply alphas -> modified and/or resized _image image -> renderer >>> >>> I personally don't like pipeline [2] because picture detail is lost in >>> the "unmultiply alphas" stage. Better to use straight alpha all the >>> way through. >> I think what needed is: >> >> [3] PIL Image (or _png.cpp) -> premultiply alphas -> _image image -> agg >> options -> >> -> modified and/or resized _image image -> renderer -> (unmultiply >> alphas)? -> GUI library >> >> That is -- all image data should be kept premultiplied internally in all >> buffers for efficiency and because this is what Agg is designed for. >> >> Can you explain what you mean by "picture detail is lost in the unmultiply >> alphas stage". There is the usual problem that by premultiplying you lose >> any color data where alpha = 0 (and you lose resolution everywhere else, but >> not resolution you can actually see after compositing). >>> So long as matplotlib is using only a subset of agg algorithms that >>> work no matter whether the alphas are premultiplied or not, I would >>> think that the most reasonable route was the one that I took; to >>> always pass straight alphas (sticking with pipeline [1]), and modify >>> the agg source slightly to fit matplotlib's approach (i.e., remove the >>> clipping there). >> I'd be really wary of modifying agg like this. Those things become hard to >> maintain. I think this instead a bug in matplotlib and should be fixed >> there. >> >> I've put an issue in the issue tracker here: >> >> https://github.com/matplotlib/matplotlib/issues/545 >> >> Cheers, >> Mike >> >>> I hope that I'm not way off base (I have a sneaking feeling that I am >>> :O ), and hope this helps. I've verified on both Linux and Windows >>> that removing the alpha-clip lines from agg_span_image_filter_rgba.h, >>> rebuilding matplotlib, and replacing _image.so/_image.pyd and >>> _backend_agg.so/_backend_agg.pyd does the trick (along with passing >>> straight alphas). So far, I've seen no ill effects on any of my >>> plots, but I'm also not in a position to run the pixel-by-pixel >>> comparison matplotlib tests. >>> >>> >>> On Wed, Oct 19, 2011 at 7:26 PM, Daniel Hyams wrote: >>>> There has to be something else in play here. I'll try to keep this >>>> short, but the summary is this: I can get the transparency to look >>>> right, but only if 1) I put "straight" alpha in image, not >>>> premultiplied, and 2) I hack agg to remove specificially every >>>> instance of the lines of code that you refer to above. >>>> >>>> Why this is, I don't know. Hopefully I'm still misusing something. >>>> However, it behaves as if the clipping of alpha in the agg library is >>>> corrupting the alpha channel. I also submit that I could have broken >>>> some other transparency capabilities of matplotlib, because I don't >>>> know what other routines use what I hacked....I did check a few >>>> transparent polygons and such though, and everything seemed to be >>>> fine. >>>> >>>> I know that the agg library has been around for quite a long time, so >>>> that also means that such a basic bug is unlikely. >>>> >>>> I've reattached the (slightly modified) script that reproduces the >>>> problem, along with a sample image that it uses. The only change to >>>> the script is right at the top, where a different image is read, a >>>> quick statement is placed to add an alpha channel if there is not >>>> already one, and I'm attempting to use premultiplied alphas. I've >>>> also attached a screenshot of the output. Notice that in this case, >>>> both "transparent" images look wrong. >>>> >>>> Now, if I 1) hack agg to remove the alpha clipping, and 2) modify the >>>> one line in the attached python script so that I use straight alpha, >>>> everything looks right. Specifically, I removed every instance of the >>>> code below from xxxx, rebuilt all of the matplotlib .so's, and >>>> specifically replaced _image.so and _backend_agg.so in my matplotlib >>>> distribution. >>>> >>>> if(fg[order_type::A]> base_mask) fg[order_type::A] >>>> = base_mask; >>>> if(fg[order_type::R]> fg[order_type::A]) >>>> fg[order_type::R] = fg[order_type::A]; >>>> if(fg[order_type::G]> fg[order_type::A]) >>>> fg[order_type::G] = fg[order_type::A]; >>>> if(fg[order_type::B]> fg[order_type::A]) >>>> fg[order_type::B] = fg[order_type::A]; >>>> >>>> >>>> >>>> >>>> On Wed, Oct 19, 2011 at 2:34 PM, Daniel Hyams wrote: >>>>> Ah, thanks so much Michael! That explanation helps a great deal; I >>>>> was always considering things in "straight alpha" format, not even >>>>> knowing that there was alternative. >>>>> >>>>> I'll play with this tonight; I don't see any problem getting the thing >>>>> working, though, now that I know what agg expects to see... >>>>> >>>>> And yes, alpha support in the image class would be very helpful ;) >>>>> >>>>> On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom >>>>> wrote: >>>>>> You are right that Agg is doing the resizing here. Agg expects >>>>>> premultiplied alpha. See [1] for information about what that means. >>>>>> >>>>>> [1] http://en.wikipedia.org/wiki/Alpha_compositing >>>>>> >>>>>> After Agg interpolates the pixel values, to prevent oversaturation it >>>>>> truncates all values to be less than alpha (which makes sense if >>>>>> everything >>>>>> is assumed to be premultiplied alpha). Arguably, the bug here is that >>>>>> nearest neighbor (which doesn't have to do any blending) doesn't >>>>>> perform the >>>>>> truncation step -- then both would look "wrong". >>>>>> >>>>>> It happens in this code snippet in span_image_filter_rgba: (base_mask >>>>>> is >>>>>> 255) >>>>>> >>>>>> if(fg[order_type::A]> base_mask) >>>>>> fg[order_type::A] >>>>>> = base_mask; >>>>>> if(fg[order_type::R]> fg[order_type::A]) >>>>>> fg[order_type::R] >>>>>> = fg[order_type::A]; >>>>>> if(fg[order_type::G]> fg[order_type::A]) >>>>>> fg[order_type::G] >>>>>> = fg[order_type::A]; >>>>>> if(fg[order_type::B]> fg[order_type::A]) >>>>>> fg[order_type::B] >>>>>> = fg[order_type::A]; >>>>>> >>>>>> So, the solution to make a partially transparent image is to not do: >>>>>> >>>>>> pix[:,:,3] = 127 >>>>>> >>>>>> but instead, do >>>>>> >>>>>> pix *= 0.5 >>>>>> >>>>>> Of course, the real fix here is to support alpha blending properly in >>>>>> the >>>>>> image class, then the user wouldn't have to deal with such details. A >>>>>> bug >>>>>> should probably be filed in the matplotlib issue tracker for this. >>>>>> >>>>>> Mike >>>>>> >>>>>> On 10/19/2011 12:23 PM, Daniel Hyams wrote: >>>>>> >>>>>> [Sorry, I keep getting tripped up with HTML mail....resent in ascii, >>>>>> and resaved one of the attachment png's to make it smaller.] >>>>>> >>>>>> >>>>>> Example script attached (PIL required). Basically, if I impose a >>>>>> specific value into an image's alpha channel and use any interpolation >>>>>> scheme other than 'nearest', there appears gray all where the figure >>>>>> didn't have any color to begin with. I've also attached a screenshot >>>>>> of the output of the script on my machine. >>>>>> >>>>>> Hopefully I'm doing something wrongly? >>>>>> >>>>>> I chased the problem and managed to hack in a solution that fixes the >>>>>> problem, but it's extremely inefficient...basically, in matplotlib's >>>>>> image.py, routine BboxImage.make_image, you can create two images >>>>>> there....one with no alpha channel (call it imRGB) and one with (call >>>>>> it imRGBA). Go through all of the routine, doing exactly the same >>>>>> things to both of the images *except* for the interpolation, which is >>>>>> set to 'nearest' for imRGBA. Then, rip the colors out of imRGB, the >>>>>> alpha channel off of imRGBA, and put them together....go through all >>>>>> of the routine again with this composited image, and it works. I >>>>>> know...I told you it was bad ;) >>>>>> >>>>>> The problem seems to be in the "resize" call in that routine...resize, >>>>>> which calls into C code, does not appear to handle things correctly >>>>>> when the alpha is anything other than 255's across the board. It >>>>>> might be a problem in the agg routines, but hopefully it is just maybe >>>>>> a misuse of the agg routines. >>>>>> >>>>>> The behavior seems to be backend independent as far as I could test (I >>>>>> tried with wxagg and tk backends). I am using mpl 1.0.0 on Windows if >>>>>> it matters. >>>>>> >>>>>> >>>>>> -- >>>>>> Daniel Hyams >>>>>> dhyams@... >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> All the data continuously generated in your IT infrastructure contains >>>>>> a >>>>>> definitive record of customers, application performance, security >>>>>> threats, fraudulent activity and more. Splunk takes this data and makes >>>>>> sense of it. Business sense. IT sense. Common sense. >>>>>> http://p.sf.net/sfu/splunk-d2d-oct >>>>>> >>>>>> _______________________________________________ >>>>>> Matplotlib-users mailing list >>>>>> Matplotlib-users@... >>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> The demand for IT networking professionals continues to grow, and the >>>>>> demand for specialized networking skills is growing even more rapidly. >>>>>> Take a complimentary Learning@... Self-Assessment and learn >>>>>> about Cisco certifications, training, and career opportunities. >>>>>> http://p.sf.net/sfu/cisco-dev2dev >>>>>> _______________________________________________ >>>>>> Matplotlib-users mailing list >>>>>> Matplotlib-users@... >>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Daniel Hyams >>>>> dhyams@... >>>> >>>> -- >>>> Daniel Hyams >>>> dhyams@... >>>> >>> >> >> ------------------------------------------------------------------------------ >> The demand for IT networking professionals continues to grow, and the >> demand for specialized networking skills is growing even more rapidly. >> Take a complimentary Learning@... Self-Assessment and learn >> about Cisco certifications, training, and career opportunities. >> http://p.sf.net/sfu/cisco-dev2dev >> _______________________________________________ >> Matplotlib-users mailing list >> Matplotlib-users@... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >> >> > > ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Benjamin Root - 2011-10-21 15:48 Attachments: Message as HTML ```On Friday, October 21, 2011, Michael Droettboom wrote: > On 10/21/2011 09:49 AM, Daniel Hyams wrote: >> All sounds reasonable Mike; I do agree that patching the agg source >> code is not that desirable; I was operating under the (incorrect) >> assumption that most, if not all, backends used straight alpha. >> >> I'll certainly test the patch tonight, but I can only test it under >> wxAgg reasonably, which is one of the backends that expects straight >> alpha as far as I know. >> >> The "loss in picture detail" comment was just discomfort with the fact >> that once the alphas are premultiplied in, there is not an exact >> reverse transformation to get your original color back. In googling >> around to better explain here, I found this, which is a much more in >> depth and better explanation than what I would have come up with: >> >> http://www.quasimondo.com/archives/000665.php >> >> The crux is here: >> >>> An example: when you set the alpha value of a pixel to 16 all color values will be multiplied with >>> a factor of 16/256 = 0.0625. So a gray pixel of 128 will become 128 * 0.0625 = 8, a darker pixel >>> of 64 will become 64 * 0.0625 = 4. But a slightly lighter pixel of maybe 67 will become 67 * >>> 0.0625 = 4.1875 - yet there are no decimals in integer pixels which means it will also become >>> 4. The effect that you will get posterization - setting your alpha channel to 8 means that you >>> also reduce your color channels to 8 levels, this means instead = 256*256*256 different colors >>> you will end up with a maximum of 8*8*8 = 512 different colors. >> One might argue that the loss of color information is not that >> crucial, because for very low alpha (where the problem is most >> pronounced), the image is almost invisible anyway... so it won't >> matter. That's true, but what if I want to (for whatever reason) take >> the image's pixels again, before draw, and boost the alpha up again? >> What I'll get is a posterized mess. So, I'm still of the opinion that >> patching agg in this situation might be the best solution to this. >> This way, straight alphas are used throughout, and for backends that >> require premultiplied alpha, the alpha can be premultiplied in at the >> latest possible moment. > > Thanks for clarifying. I understand what you're saying now. I think > what we want to do is store the unmultiplied alpha as a "canonical" > version of the image, and premultiply a copy (or use some C++ iterator > magic to avoid the copy) right before sending it off to Agg. Then the > alpha can be fully "tweakable" at runtime. > > I'll try to tackle this problem, as well as the problem that set_alpha > simply doesn't work, at the same time when I get a chance (or patches > are always welcome, of course :). > > Mike Mike, This idea is sort of along the same vein of the idea that I have been having for unifying the colors framework and allowing for transforms to be performed upon access of the colors. If you think about it, the issue of different backends wanting different information can be dealt with by having a transform that returns pre multiplied alphas and another that returns straight alphas. The backends can then solely the desired transform that it expects. I guess the question is whether the transformation step should be generalized at the python level, or specialized down at the python/c++ boundary. I really wish I had more time this semester to try and come up with a proof of concept. Ben Root P.S. - I had also noticed alpha-blending issues before (and asked about it), but I ended up convincing myself (apparently wrongly) that it was correct. ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Daniel Hyams - 2011-10-21 18:21 ```> Thanks for clarifying.  I understand what you're saying now.  I think > what we want to do is store the unmultiplied alpha as a "canonical" > version of the image, and premultiply a copy (or use some C++ iterator > magic to avoid the copy) right before sending it off to Agg.  Then the > alpha can be fully "tweakable" at runtime. Just for my understanding of the design approach of matplotlib in general, is it implicitly assumed that once pixels have been passed to agg, they are destined for the renderer? No other possible use? And the life span is short? I'm just not very familiar with where _image is used throughout the rest of the code, if at all. As far as the iterator magic, the kicker is that it would have to take place inside of agg; it cannot be on the mpl side; So if I'm understanding correctly, 1) create a copy of the image within resize() 2) premultiply the alpha on the copy 3) let agg do its work, returning its result in a new pixmap (resized) that is premultiplied. 4) toss the extra copy made in (1) 5) in each individual backend's renderer, right before the pixels are rendered, unmultiply the alpha if necessary (as is the case with wxagg). I also remember you making mention of saving pngs above, I suppose there would have to be an unmultiply there as well. i've done some grepping through the code, and don't see that these filter routines are used anywhere except specifically in this resize() function. I can fully understand the desire to comply with agg's spec, but I'm still not convinced that a small patch to agg isn't appropriate here. It's a little odiferous, but won't be hard to maintain given that agg is a very slowly changing library (last release 2006?). Or maybe treat patching agg as short term solution. Looking through the filter algorithms too, it looks like exactly the same ops are done to each channel (RGBA) separately. So (it appears as if, to my untrained eye) that the algorithms are applicable regardless of whether or not the alphas are premultiplied. Further, removing the clip will have very minimal impact on "proper" usage of agg (i.e. sending in premultiplied alpha). > I'll try to tackle this problem, as well as the problem that set_alpha > simply doesn't work, at the same time when I get a chance (or patches > are always welcome, of course :). I have a patch, but you don't like it :D But seriously, I'll take a more careful look at what it would take to implement items 1-5 above; it might not be so bad, but I'm uncomfortable dealing with the "underbelly" of matplotlib, as I'm not familiar with either mpl's design nor the general subject of graphics. ```

 Re: [Matplotlib-users] strange behavior of images when they have an alpha channel [resend] From: Daniel Hyams - 2011-10-22 23:40 ```Michael: I commented on the patch here: https://github.com/matplotlib/matplotlib/issues/545 In short...it works! ```