Re: [Algorithms] Shader system design for shader centric engine

[Jason H. <jas...@di...>]

Jon Watte wrote:
> If you know what the set of parameters for a shader is, and it won't 
> "ever" change, then you can do that.
>
> For us, that's not true for a few reasons:
>
> 1) We do "Live Everything" -- if the file on disk changes, all 
> dependencies changes within the running instance. Gives quick 
> turn-around time.
> 2) We use this system (in a rather more fleshed-out form) for almost 
> every data dependency, not just shaders.
> 3) Because this system can be driven from XML, and the application has a 
> DOM, we can add entirely new shader parameters/dependencies by just 
> changing some XML, no re-compilation needed. As we're selling a 
> licensable platform/toolkit, this is important :-)
>
>   

I don't see how #1 precludes a simpler load-time binding, nor would #3.  
Or are they only fixed up on load in your system?  Each developer will 
of course implement their method in a way that suits their needs and 
tastes.  Shaders should not require any game code recompilation under 
any circumstances.  I am curious how you can introduce new types of data 
through XML and bind to them in a shader, given that your method appears 
to use some sort of static type information to find data.  Wouldn't that 
require compilation to do a DECLARE_PROPERTY()?

I'm intrigued by #2.  Most people who attempt this kind of dependency 
system simply switch to holding their data in a script language that 
does it intrinsically.  You've built something like it out of C++.  So, 
in your system, does this binding happen once until the dependencies 
change?  And it can fetch the data without branches, without trying to 
determine whether it's changed every time you access it?

What I've done, personally, is build a generic symbolic asset linker 
that the runtime can use to perform symbolic re-linking at runtime as 
needed.  This makes it possible to do relocation of assets in memory as 
needed and simply re-fixup pointers.  It also allows me to unload/reload 
an asset if a file changes on disk.  I built a tool-side pre-linker that 
allows consolidation of assets and removal of symbolic metadata so I can 
choose to bind whole levels for final releases, or during development 
just use one asset at a time in broken-out form.  This pushes 90% of the 
complexity into the tools and a tiny bit into the runtime.  Seems to 
address most of the issues I can think of, even though I haven't had 
much chance to put my own system through its paces.  The runtime is 
still in development.

Thanks,
JH