Need help on Anti-grain geometry for alpha handling

[Coconut]

Hi,

I have a piece of code that was originally written as directly draw lines,
fonts, polygons with anti-aliasing on an image.

But later on, we decided to use layer approach which is the following

1. create a layer, and draw that layer.
2. we can have multiple layers...
3. and then compose all the layers onto a white background image to get a
   final image.

The problem is, because of the alpha handling (we clear each layer to
rgba(255,255,255,0)), it effectively blends the transparency twice,
resulting incorrect final color.

I did a bit calculation, the alpha of the layer vs the actual color is
something like
newAlpha = 16 * sqrt (originalApha)

I could change the fill color manually to achieve the result for some
stuff, but for fonts / line drawing etc, that would not work due to the
underlying anti-aliasing logic.

I am wondering what is the general terminology for this problem as well as
the agg specific way of doing it. The reason why I am asking is because
there appears to be multiple places that could do it, but I am not sure
which one would be the "correct" way doing it.

Here is my current layer creation in agg:

agg::rendering_buffer renderBuffer;
agg::pixfmt_rgba32 pixelFormat;
agg::rasterizer_scanline_aa<> ras;
agg::scanline_p8 m_scanline;
agg::renderer_base<agg::pixfmt_rgba32> rbase;

int bufferSize = width * height * 4;
byteBuffer = new unsigned char[m_bufferSize];
renderBuffer.attach(byteBuffer, width, height, width * 4);
pixelFormat.attach(renderBuffer);
rbase.attach(pixelFormat);

rbase.clear (Color (0xff, 0xff, 0xff, 0));

Could anyone help?

Thanks,

Heng Yuan

[Jim Barry]

If you want to do compositing, the (destination) layers need to be pixfmt_rgba32_pre. You can find a brief explanation in the version 2.2 release notes.

By the way, rgba(1, 1, 1, 0) is not valid in premultiplied space, since each rgb component should be <= alpha.

[Coconut]

Thanks a lot. I will try it out.