Re: [Algorithms] Directx9 ambient occlusion

[Fabian G. <ry...@gm...>]

On 11/9/2010 9:40 AM, Jeff Gooloolie wrote:
> I am making a dx9 ao shader.
>
> I have the ao greyscale image in a texture, and the rendered scene in
> the backbuffer.
>
> What blendmodes (dx9 fixed function pipeline) would I need to multiply
> the ao texture against the backbuffer.
>
> something like this : backbuffer=backbuffer*aotexture
>
> I currently have the following, but it is not blending with the backbuffer.
>
> gRender->SetRenderState(D3DRS_ALPHABLENDENABLE,TRUE);
> gRender->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_ZERO);
> gRender->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_SRCCOLOR);

This blend setup looks right for what you're trying to do.

> gRender->SetTextureStageState( 0, D3DTSS_COLOROP, D3DTOP_MODULATE );
> gRender->SetTextureStageState( 0, D3DTSS_COLORARG1, D3DTA_TEXTURE );
> gRender->SetTextureStageState( 0, D3DTSS_COLORARG2, D3DTA_DIFFUSE );
> gRender->SetTextureStageState( 0, D3DTSS_ALPHAOP, D3DTOP_SELECTARG2 );
> gRender->SetTextureStageState( 0, D3DTSS_ALPHAARG2, D3DTA_DIFFUSE );

Did you double-check whether the diffuse colors are set up correctly? 
Also, what is the effect you're seeing? If you're only using texture 
stage 0, did you remember to disable stage 1 (which also auto-disables 
all following ones)?

> Is it even possible to blend a texture with the existing backbuffer?

Yes, certainly - way back before even double-texturing texture hardware, 
alpha blending was the *only* way to do this.

This is not really an algorithmic question though - consider asking such 
questions on the directxdev mailing list.

Cheers,
-Fabian