Re: [Algorithms] Some question about "Lighting and Material ofHalo3" and "Lightmap compression in Halo 3"

[Sébastien L. <seb...@do...>]

> even if the compression get a 4:1 ratio,

=> even if the compression get a 6:1 ratio,

 

De : Sébastien Lagarde 
Envoyé : mercredi 3 février 2010 11:01
À : Game Development Algorithms
Objet : Re: [Algorithms] Some question about "Lighting and Material ofHalo3" and "Lightmap compression in Halo 3"

 

Hi all, 

 

I take some time to try to fully understand what was saying :),

I have increase my level of understanding, thanks to both of you Sam and David ,

I made the difference between incoming lighting and diffuse irradiance now, and better understand the convolution with cosine lobe.

 

Alright for this part. Thanks to you Peter-Pike, for pointing the problem, in fact I didn't notice the rotation in local frame, and now I think I have 

more question than before :)

 

I tried to understand what Halo 3 do cause this is a real XBOX360 shipped game with SH lighting in it with all console constraint on memory and shader performance.

So I try to figure what tricks they used :)

 

What I think now for halo 3 for their SH lightmap is:

Incoming lighting generated by global illumination solver are projected in 9S H coefficient by color channel

SH coefficient are rotated in tangent space

Then they extract dominant light (intensity and direction).

I suppose they store the DC + linear SH term , so 4 coefficient by channel, (as stated by Peter-Pike)

And they store the intensity (RGB). This give the 5 * 3 float value

 

At runtime, I suppose they used the linear SH term to retrieve the dominant light direction, 

the dominant light is used for the analytic cook torrance specular model.

the dominant light should be used for diffuse part too:

I think they need to extract dominant light intensity from SH, so they light with one diffuse directional light, 

and SH coefficient remaining after extraction of dominant light intensities. I think the extraction is done in shader and not in SH lightmap in order to be

able to recover the right dominant light direction (I am certainly wrong :) ).

 

But after that, I am lost with the rotation in local frame:

 

I don't get the thing for the equation (5). which only require the 3 ZH coefficient.

As my SH coefficient are store in tangent space, I am already in the case of equation (5).

so I don't need to rotate SH coefficient once again (maybe I am wrong here, do I need to rotate with the tangent space normal extract from normal map ?),

and so I only require SH coefficient for i = 0, 2, 6 ?

The convolution by cosine lobe seems to be applied at this time, but two thing are missing, 

the constant: sqrt(4 pi / (2l + 1)) and the divide by PI to turn irradiance into the exit radiance.

(But in their shader code provided, they divided lightprobe_color by Pi at end of diffuse_reflectance, so maybe they just not say it)

 

But doing the convolution at this time mean that we store incoming lighting in SH lightmap, and not diffuse irradiance.

So extracting dominant light from incoming lighting is maybe wrong ?

 

Last though, with their 15 float to store, they said they used two DXT5 by SH band, 

mean we require 10 DXT5 to store only one lightmap. even if the compression get a 4:1 ratio, this require 10Mo for a SH lightmap of 1024x1024 without mipmap

Which seems very huge for console memory even with a good streaming texture system...

 

I am just curious about all these SH tricks, 

Anyway, thanks for the help you already provide to me.

 

Lagarde Sébastien

 

De : Peter-Pike Sloan [mailto:pet...@ho...] 
Envoyé : lundi 1 février 2010 22:46
À : gda...@li...
Objet : Re: [Algorithms] Some question about "Lighting and Material of Halo3" and "Lightmap compression in Halo 3"

 

Hi,

 

I believe the source of confusion is that in this equation (5) things are expressed in the local coordinate frame, the fact that they show 3 coefficients implies they are using quadratic SH (so when rotated into the local frame and integrated against a clamped cosine function you only need the 3 ZH coefficients.)  The general case (where you don't know the coordinate frame, or you want to evaluate for any normal) requires all 9 coefficients.

I think the "5*3" comes from the compression mentioned in Hao's slides (linear SH + RGB for directional light in "optimal linear" direction.  Which is 5*3 scalars.)

 

The compression work is Wang et al, I think Yaohua's slides are more indicative of what was actually used...

 

Peter-Pike Sloan

 

________________________________

Date: Fri, 29 Jan 2010 16:28:57 +0100
From: seb...@do...
To: gda...@li...
Subject: [Algorithms] Some question about "Lighting and Material of Halo3" and "Lightmap compression in Halo 3"

Hello all, 

 

I tried to contact the author of this two (now old) paper, without success,:

"Lighting and Material of Halo 3" published at siggraph 2008 (http://ati.amd.com/developer/SIGGRAPH08/Chapter01-Chen-Lighting_and_Material_of_Halo3.pdf)

and GDC 2008 conference "Lightmap compression in Halo 3" (http://toomuchlogic.com/Lightmap_Compression_2008_02_22.pdf)

so I will ask some question to the list, if anyone is interested by the subject and has better understanding of math than me :)

 

1. "Lighting and Material of Halo 3"  About equation (5) the diffuse reflectance using SH basis

the diffuse reflectance using SH basis is : k_d R_d Sum Lambda_i A_i

A_i is the projection of the cosine lobe in SH, and as it is radially symetric.

all coefficient with m != 0 are 0.

After that the author give the first three term of A_i.

I wondering how many band are use for this calculation  ? (order 3, 4 or more ?)

 

I read from "Stupid spherical harmonics tricks" from Perter Pike sloan (http://www.ppsloan.org/publications/StupidSH36.pdf)

that order 3 SH is sufficient for approximate light source

but for HDR light sources he recommand order 5.

As order 4 is 0 (From paper http://www.eecs.berkeley.edu/~ravir/lighting.pdf I get the formula for A_i)

This mean 4 coefficient to store by color channel.

As I am pretty sure the author store  HDR data, can someone lighten me ?

 

2. About texture storage (which are deduced from above statement)

 

I try to figure out how are encoded the incident radiance in their SHLightmap.

>From GDC 2008 conference "Lightmap compression in Halo 3" I can read that the author need to

store for each texel a vector of 5 * 3 float values.

I don't figure what are the values exactly.

 

My assumption is that "3" is for each channel color RGB,

But I can't figure what's the 5 is ?

Are they 5 first band of SH order like I suppose above (but as I said, we only need 4 coefficient in this case)

or maybe order 6 ?

 

In this same paper later I found:

A. Two DXT5 texture for each SH coefficient channel (HDR, positive/negative)

And

B. Each band of the SH coefficients (RGB) are converted to Luvw space

 

I suppose that Luvw space is what is describe in this paper "Rendering from Compressed High Dynamic Range Textures on

Programmable Graphics Hardware" by Peter Pike sloan and al.(ftp://ftp.research.microsoft.com/pub/tr/TR-2006-152.pdf)

 

What I don't understand is that A and B seem different. I can't understand what is store.

Are they storing for each band the triplet RGB of SH coeeficient, mean 3 float value for two DXT5 x order

or do they store store 5 SH coefficient for channel color R in two DXT5 ?

 

So what are the total storage cost of all the 5 * 3 float value in term of DXT5 texture ?

Cause It looks like to be pretty big.

 

Thanks for anyone interested by this post

 

Best regards

 

Lagarde Sébastien