Re: [Algorithms] Filtering

[Andreas B. <and...@gm...>]

Well that certainly explains it. I guess I'm stuck with doing the
filtering myself then. Thanks anyway guys!

On Fri, Oct 1, 2010 at 4:54 PM, Michael Bunnell <mi...@fa...> wrote:
> You guessed right. The loss of precision is in the texture units.
> Unfortunately, 8 bit components are filtered to 8 bit results (even though
> they show up as floating point values in the shader). This is true
> for nvidia gpus for sure and probably all other gpus.
>
> -mike
>
> ----- Original Message -----
> From: Stefan Sandberg
> To: Game Development Algorithms
> Sent: Friday, October 01, 2010 1:45 AM
> Subject: Re: [Algorithms] Filtering
> Assuming you're after precision, what's wrong with doing it manually? :)
> If performance is what you're after, and you're working on textures as they
> were intended(ie, game textures or video or something like that, not
> 'data'), you could separate contrast & color separately, keeping high
> contrast resolution, and downsampled color, and
> you'd save both bandwidth and instr.
> If you simply want to know 'why', I'm guessing loss of precision in the tex
> units?
> You've already ruled out shader precision from your own manual filtering,
> so doesn't leave much else, imo..
> Other than manipulating the data you're working on, which is the only thing
> you -can- change I guess, I cant really see a solution,
> but far greater minds linger here than mine, so hold on for what I assume
> will be a lengthy description of floating point math as
> it is implemented in modern gpu's :)
>
>
> On Fri, Oct 1, 2010 at 9:57 AM, Andreas Brinck <and...@gm...>
> wrote:
>>
>> Hi,
>>
>> I have a texture in which I use the R, G and B channel to store a
>> value in the [0, 1] range with very high precision. The value is
>> extracted like this in the (Cg) shader:
>>
>> float
>> extractValue(float2 pos) {
>>    float4 temp = tex2D(buffer, pos);
>>    return (temp.x * 16711680.0 + temp.y * 65280.0 + temp.z * 255.0) *
>> (1.0 / 16777215.0);
>> }
>>
>> I now want to sample this value with bilinear filtering but when I do
>> this I don't get a correct result. If I do the filtering manually like
>> this:
>>
>> float
>> sampleValue(float2 pos) {
>>        float2 ipos = floor(pos);
>>        float2 fracs = pos - ipos;
>>        float d0 = extractValue(ipos);
>>        float d1 = extractValue(ipos + float2(1, 0));
>>        float d2 = extractValue(ipos + float2(0, 1));
>>        float d3 = extractValue(ipos + float2(1, 1));
>>        return lerp(lerp(d0, d1, fracs.x), lerp(d2, d3, fracs.x), fracs.y);
>> }
>>
>> everything works as expected. The values in the buffer can be seen as
>> a linear combination of three constants:
>>
>> value = (C0 * r + C1 * g + C2 * b)
>>
>> If we use the built in texture filtering we should get the following
>> if we sample somewhere between two texels: {r0, g0, b0} and {r1, g1,
>> b1}. For simplicity we just look at filtering along one axis:
>>
>> filtered value = lerp(r0, r1, t) * C0 + lerp(g0, g1, t) * C1 +
>> lerp(b0, b1, t) * C2;
>>
>> Doing the filtering manually:
>>
>> filtered value = lerp(r0 * C0 + b0 * C1 + g0 * C2, r1 * C0 + g1 * C1 +
>> b1 * C2, t)  =
>>                   = (r0 * C0 + b0 * C1 + g0 * C2) * (1 - t) + (r1 *
>> C0 + g1 * C1 + b1 * C2) * t =
>>                   = (r0 * C0) * (1 - t) + (r1 * C0) * t + ... =
>>                   = lerp(r0, r1, t) * C0 + ...
>>
>> So in the world of non floating point numbers these two should be
>> equivalent right?
>>
>> My theory is that the error is caused by an unfortunate order of
>> floating point operations. I've tried variations like:
>>
>> (temp.x * (16711680.0 / 16777215.0) + temp.y * (65280.0/16777215.0) +
>> temp.z * (255.0/16777215.0))
>>
>> and
>>
>> (((temp.x * 256.0 + temp.y) * 256.0 + temp.z) * 255.0) * (1.0 /
>> 16777215.0)
>>
>> but all exhibit the same problem. What do you think; is it possible to
>> solve this problem?
>>
>> Regards Andreas
>>
>>
>> ------------------------------------------------------------------------------
>> Start uncovering the many advantages of virtual appliances
>> and start using them to simplify application deployment and
>> accelerate your shift to cloud computing.
>> http://p.sf.net/sfu/novell-sfdev2dev
>> _______________________________________________
>> GDAlgorithms-list mailing list
>> GDA...@li...
>> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
>> Archives:
>> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>
> ________________________________
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
>
> ________________________________
>
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>