gdalgorithms-list

[Algorithms] How do people use shader effect "compile" parameters?

[Robin G. <rob...@gm...>]

I'm doing a little research into how people use shader effects with an
emphasis on understanding and reasoning about them from source. My
question to the group goes like this:

When defining a shader effect, a single translation unit of code is
used to generate both a fragment and a vertex shader. These functions
can share global variables and structures, and the place where the
arbitration happens is in the interaction between the function
signature and the "compile" declaration inside each pass, e.g.

 float4 global_color  = { 0.3f, 0.3f, 0.3f, 1.0f }; void MyVertexShader( in
float4 pos : POSITION,
                      in float4 color) : POSITION { ... }
 technique {
    pass {
       ...
       VertexProgram = compile vs_3_0 MyVertexShader(global_color * 2.0f);
    }
 }

In this example the function "MyVertexShader" is parameterized with an
expression, and the documentation for both Cg and HLSL state that
these parameters are arbitrary "constant expressions" - any expression
that can evaluate to an R-value of the correct type. These expressions
are free to read global variables, operate with any literal value, use
typedef user types, struct accessors, array indexes, conditional
operators :? and the full range of intrinsic functions in their
evaluation.

Yet almost all the examples I see in the SDKs or online hardly touch
this feature, maybe substituting in a different sampler, doing vector
normalization or a simple biasing expression.

Q: If I were to make a tool that enforced only simple use of these
expression parameters (no intrinsics, no structures, no typedefs, no
conditionals), what cool tricks would I be giving up? Is using these
arbitrary expressions a key technique for anyone?

Thanks in advance,

- Robin Green.

Re: [Algorithms] How do people use shader effect "compile" parameters?

[Ian T. <ITh...@re...>]

I've used static pipeline control structures to manage combinations of shaders.

struct ShaderConfig
{
	bool bLighting;
	bool bScattering;
	bool bShadowing;
};

I then have modifier functions that mutate these configs.

ShaderConfig StdConfig_2_0 =
{
	true,
	false,
	true,
};

ShaderConfig StdConfig_3_0 = 

ShaderConfig NoShadows( ShaderConfig sc )
{
	sc.bShadowing = false;
	return sc;
}

technique t
{
	pass p
	{
		VertexShader = compile vs_2_0 Standard_v( NoShadows( StdConfig_2_0 ) );
		...
       }
}

This allowed us to define standard settings for shader models (we supported 1.1, 2.0 (and variants), 3.0, and later 4.0).

We generated large technique files (1000s of techniques, duplicate shaders shared) due to our need/desire to toggle settings and have them compile down optimally.

This handles most things except for differences in interpolator usage.  To manage that better, we sometimes 'compile'd a wrapper that would just store the parameters it needed from an uber function. 

I would rarely pass in static variables and would just leave them to be globals.

________________________________________
From: Robin Green [mailto:rob...@gm...] 
Sent: Wednesday, September 24, 2008 2:08 PM
To: Game Development Algorithms
Subject: [Algorithms] How do people use shader effect "compile" parameters?

I'm doing a little research into how people use shader effects with an
emphasis on understanding and reasoning about them from source. My
question to the group goes like this:

When defining a shader effect, a single translation unit of code is
used to generate both a fragment and a vertex shader. These functions
can share global variables and structures, and the place where the
arbitration happens is in the interaction between the function
signature and the "compile" declaration inside each pass, e.g.

 float4 global_color  = { 0.3f, 0.3f, 0.3f, 1.0f };
 void MyVertexShader( in float4 pos : POSITION,
                      in float4 color) : POSITION { ... }
 technique {
    pass {
       ...
       VertexProgram = compile vs_3_0 MyVertexShader(global_color * 2.0f);
    }
 }

In this example the function "MyVertexShader" is parameterized with an
expression, and the documentation for both Cg and HLSL state that
these parameters are arbitrary "constant expressions" - any expression
that can evaluate to an R-value of the correct type. These expressions
are free to read global variables, operate with any literal value, use
typedef user types, struct accessors, array indexes, conditional
operators :? and the full range of intrinsic functions in their
evaluation.

Yet almost all the examples I see in the SDKs or online hardly touch
this feature, maybe substituting in a different sampler, doing vector
normalization or a simple biasing expression.

Q: If I were to make a tool that enforced only simple use of these
expression parameters (no intrinsics, no structures, no typedefs, no
conditionals), what cool tricks would I be giving up? Is using these
arbitrary expressions a key technique for anyone?

Thanks in advance,

- Robin Green.

Re: [Algorithms] How do people use shader effect "compile" parameters?

[rumzeus <ru...@gm...>]

Ian Thomson wrote:
> I've used static pipeline control structures to manage combinations of shaders.
> 
> struct ShaderConfig
> {
> 	bool bLighting;
> 	bool bScattering;
> 	bool bShadowing;
> };

> ShaderConfig StdConfig_2_0 =
> {
> 	true,
> 	false,
> 	true,
> };

> technique t
> {
> 	pass p
> 	{
> 		VertexShader = compile vs_2_0 Standard_v( NoShadows( StdConfig_2_0 ) );
> 		...
>        }
> }

> We generated large technique files (1000s of techniques, duplicate shaders shared) due to our need/desire to toggle settings and have them compile down optimally.

I'm trying to get this to "compile down optimally", but I seem to be 
missing something. Shouldn't the two techniques below both compile down 
to the same single instruction?

----- shader.fx
struct Config { bool Enabled; };
Config std = { false };

float4 PS_DirectBoolParam(float4 inTexCoord : TEXCOORD0,
                           uniform bool configEnabled) : COLOR0 {
   if(configEnabled) return pow(inTexCoord,pow(inTexCoord,3));
   else return inTexCoord;
}

float4 PS_ViaStruct(float4 inTexCoord : TEXCOORD0,
                     uniform Config config) : COLOR0 {
   if(config.Enabled) return pow(inTexCoord,pow(inTexCoord,3));
   else return inTexCoord;
}

technique T_DirectBoolParam {
   pass Pass0 {
     PixelShader = compile ps_2_0 PS_DirectBoolParam(false);
   }
}

technique T_ViaStruct {
   pass Pass0 {
     PixelShader = compile ps_2_0 PS_ViaStruct(std);
   }
}
----- end shader.fx

 > fxc /Tfx_2_0 shader.fx
Microsoft (R) D3D10 Shader Compiler 9.24.949.2307
Copyright (C) Microsoft Corporation 2002-2007. All rights reserved.


//listing of all techniques and passes with embedded asm listings

technique T_DirectBoolParam
{
     pass Pass0
     {
         //No embedded vertex shader
         pixelshader =
             asm {
             //
             // Generated by Microsoft (R) HLSL Shader Compiler 
9.24.949.2307
                 ps_2_0
                 dcl t0
                 mov oC0, t0

             // approximately 1 instruction slot used
             };
     }
}

technique T_ViaStruct
{
     pass Pass0
     {
         //No embedded vertex shader
         pixelshader =
             asm {
             //
             // Generated by Microsoft (R) HLSL Shader Compiler 
9.24.949.2307
             //
             // Parameters:
             //
             //   struct
             //   {
             //       bool Enabled;
             //
             //   } std;
             //
             //
             // Registers:
             //
             //   Name         Reg   Size
             //   ------------ ----- ----
             //   std          c0       1
             //

                 ps_2_0
                 dcl t0
                 mul r0, t0, t0
                 mul r0, r0, t0
                 log r1.x, t0.x
                 log r1.y, t0.y
                 log r1.z, t0.z
                 log r1.w, t0.w
                 mul r0, r0, r1
                 exp r1.x, r0.x
                 exp r1.y, r0.y
                 exp r1.z, r0.z
                 exp r1.w, r0.w
                 cmp r0, -c0.x, t0, r1
                 mov oC0, r0

             // approximately 13 instruction slots used
             };
     }
}

Re: [Algorithms] How do people use shader effect "compile" parameters?

[Jon W. <jw...@gm...>]

Robin Green wrote:
> Yet almost all the examples I see in the SDKs or online hardly touch
> this feature, maybe substituting in a different sampler, doing vector
> normalization or a simple biasing expression.

You would think that changing the globals after first loading/compiling 
an effect would then re-compile the pixel and vertex shader when you 
bind it, or it would actually generate a binding to that global as part 
of the compiled code. However, I have not had great luck with making 
this happen -- as far as I can tell, that isn't actually supported (or 
wasn't when I tried it).

Sincerely,

jw

Re: [Algorithms] How do people use shader effect "compile" parameters?

[Tom F. <tom...@ee...>]

I've used this technique a lot myself. I'm sure I've seen this used in
samples - which is why I used it :-) It's a pretty elegant way of
specializing "uber-shader"-like HLSL code without actually generating
shaders with conditional branches in.

That said, I've never actually inspected the output to check that it did do
the specialization.

TomF.


> -----Original Message-----
> From: Ian Thomson [mailto:ITh...@re...]
> Sent: Wednesday, September 24, 2008 2:55 PM
> To: Game Development Algorithms
> Subject: Re: [Algorithms] How do people use shader effect "compile"
> parameters?
> 
> I've used static pipeline control structures to manage combinations of
> shaders.
> 
> struct ShaderConfig
> {
> 	bool bLighting;
> 	bool bScattering;
> 	bool bShadowing;
> };
> 
> I then have modifier functions that mutate these configs.
> 
> ShaderConfig StdConfig_2_0 =
> {
> 	true,
> 	false,
> 	true,
> };
> 
> ShaderConfig StdConfig_3_0 =
> 
> ShaderConfig NoShadows( ShaderConfig sc )
> {
> 	sc.bShadowing = false;
> 	return sc;
> }
> 
> technique t
> {
> 	pass p
> 	{
> 		VertexShader = compile vs_2_0 Standard_v( NoShadows(
> StdConfig_2_0 ) );
> 		...
>        }
> }
> 
> This allowed us to define standard settings for shader models (we
> supported 1.1, 2.0 (and variants), 3.0, and later 4.0).
> 
> We generated large technique files (1000s of techniques, duplicate
> shaders shared) due to our need/desire to toggle settings and have them
> compile down optimally.
> 
> This handles most things except for differences in interpolator usage.
> To manage that better, we sometimes 'compile'd a wrapper that would
> just store the parameters it needed from an uber function.
> 
> I would rarely pass in static variables and would just leave them to be
> globals.
> 
> ________________________________________
> From: Robin Green [mailto:rob...@gm...]
> Sent: Wednesday, September 24, 2008 2:08 PM
> To: Game Development Algorithms
> Subject: [Algorithms] How do people use shader effect "compile"
> parameters?
> 
> I'm doing a little research into how people use shader effects with an
> emphasis on understanding and reasoning about them from source. My
> question to the group goes like this:
> 
> When defining a shader effect, a single translation unit of code is
> used to generate both a fragment and a vertex shader. These functions
> can share global variables and structures, and the place where the
> arbitration happens is in the interaction between the function
> signature and the "compile" declaration inside each pass, e.g.
> 
>  float4 global_color  = { 0.3f, 0.3f, 0.3f, 1.0f };
>  void MyVertexShader( in float4 pos : POSITION,
>                       in float4 color) : POSITION { ... }
>  technique {
>     pass {
>        ...
>        VertexProgram = compile vs_3_0 MyVertexShader(global_color *
> 2.0f);
>     }
>  }
> 
> In this example the function "MyVertexShader" is parameterized with an
> expression, and the documentation for both Cg and HLSL state that
> these parameters are arbitrary "constant expressions" - any expression
> that can evaluate to an R-value of the correct type. These expressions
> are free to read global variables, operate with any literal value, use
> typedef user types, struct accessors, array indexes, conditional
> operators :? and the full range of intrinsic functions in their
> evaluation.
> 
> Yet almost all the examples I see in the SDKs or online hardly touch
> this feature, maybe substituting in a different sampler, doing vector
> normalization or a simple biasing expression.
> 
> Q: If I were to make a tool that enforced only simple use of these
> expression parameters (no intrinsics, no structures, no typedefs, no
> conditionals), what cool tricks would I be giving up? Is using these
> arbitrary expressions a key technique for anyone?
> 
> Thanks in advance,
> 
> - Robin Green.
> 
> -----------------------------------------------------------------------
> --
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the
> world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-
> list

Re: [Algorithms] How do people use shader effect "compile" parameters?

[Jeff R. <je...@gm...>]

Of course there's alway the pre-processor, no? Then you can really be sure
of whats happening. I use it for uber-shader type control in GLSL. Just
prepend some #define values to your string before you hand it over to the
API to be compiled.

Jeff Russell
8monkey Labs

On Thu, Sep 25, 2008 at 12:42 AM, Tom Forsyth
<tom...@ee...>wrote:

> I've used this technique a lot myself. I'm sure I've seen this used in
> samples - which is why I used it :-) It's a pretty elegant way of
> specializing "uber-shader"-like HLSL code without actually generating
> shaders with conditional branches in.
>
> That said, I've never actually inspected the output to check that it did do
> the specialization.
>
> TomF.
>
>
> > -----Original Message-----
> > From: Ian Thomson [mailto:ITh...@re...]
> > Sent: Wednesday, September 24, 2008 2:55 PM
> > To: Game Development Algorithms
> > Subject: Re: [Algorithms] How do people use shader effect "compile"
> > parameters?
> >
> > I've used static pipeline control structures to manage combinations of
> > shaders.
> >
> > struct ShaderConfig
> > {
> >       bool bLighting;
> >       bool bScattering;
> >       bool bShadowing;
> > };
> >
> > I then have modifier functions that mutate these configs.
> >
> > ShaderConfig StdConfig_2_0 =
> > {
> >       true,
> >       false,
> >       true,
> > };
> >
> > ShaderConfig StdConfig_3_0 =
> >
> > ShaderConfig NoShadows( ShaderConfig sc )
> > {
> >       sc.bShadowing = false;
> >       return sc;
> > }
> >
> > technique t
> > {
> >       pass p
> >       {
> >               VertexShader = compile vs_2_0 Standard_v( NoShadows(
> > StdConfig_2_0 ) );
> >               ...
> >        }
> > }
> >
> > This allowed us to define standard settings for shader models (we
> > supported 1.1, 2.0 (and variants), 3.0, and later 4.0).
> >
> > We generated large technique files (1000s of techniques, duplicate
> > shaders shared) due to our need/desire to toggle settings and have them
> > compile down optimally.
> >
> > This handles most things except for differences in interpolator usage.
> > To manage that better, we sometimes 'compile'd a wrapper that would
> > just store the parameters it needed from an uber function.
> >
> > I would rarely pass in static variables and would just leave them to be
> > globals.
> >
> > ________________________________________
> > From: Robin Green [mailto:rob...@gm...]
> > Sent: Wednesday, September 24, 2008 2:08 PM
> > To: Game Development Algorithms
> > Subject: [Algorithms] How do people use shader effect "compile"
> > parameters?
> >
> > I'm doing a little research into how people use shader effects with an
> > emphasis on understanding and reasoning about them from source. My
> > question to the group goes like this:
> >
> > When defining a shader effect, a single translation unit of code is
> > used to generate both a fragment and a vertex shader. These functions
> > can share global variables and structures, and the place where the
> > arbitration happens is in the interaction between the function
> > signature and the "compile" declaration inside each pass, e.g.
> >
> >  float4 global_color  = { 0.3f, 0.3f, 0.3f, 1.0f };
> >  void MyVertexShader( in float4 pos : POSITION,
> >                       in float4 color) : POSITION { ... }
> >  technique {
> >     pass {
> >        ...
> >        VertexProgram = compile vs_3_0 MyVertexShader(global_color *
> > 2.0f);
> >     }
> >  }
> >
> > In this example the function "MyVertexShader" is parameterized with an
> > expression, and the documentation for both Cg and HLSL state that
> > these parameters are arbitrary "constant expressions" - any expression
> > that can evaluate to an R-value of the correct type. These expressions
> > are free to read global variables, operate with any literal value, use
> > typedef user types, struct accessors, array indexes, conditional
> > operators :? and the full range of intrinsic functions in their
> > evaluation.
> >
> > Yet almost all the examples I see in the SDKs or online hardly touch
> > this feature, maybe substituting in a different sampler, doing vector
> > normalization or a simple biasing expression.
> >
> > Q: If I were to make a tool that enforced only simple use of these
> > expression parameters (no intrinsics, no structures, no typedefs, no
> > conditionals), what cool tricks would I be giving up? Is using these
> > arbitrary expressions a key technique for anyone?
> >
> > Thanks in advance,
> >
> > - Robin Green.
> >
> > -----------------------------------------------------------------------
> > --
> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> > challenge
> > Build the coolest Linux based applications with Moblin SDK & win great
> > prizes
> > Grand prize is a trip for two to an Open Source event anywhere in the
> > world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > GDAlgorithms-list mailing list
> > GDA...@li...
> > https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> > Archives:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-
> > list
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>