gdalgorithms-list

[Algorithms] Spherical harmonics for room acoustic modelling

[Oscar F. <os...@tr...>]

I will admit first off I don't have a great knowledge of things like
Pre-computed Radiance Transfer but I was having a chat with a colleague
recently on how audio research lags behind video research in gaming.

One idea that we were talking about is using spherical harmonics in a
similar way to PRT to do some sort of Pre-computed Acoustic Transfer.

Basically you'd add a new material type to a room that would give you the
sound reflectivity of a given surface.   You could then use the same
techniques (obviously calculated with something like OpenCL) to give you a
good model of how sound moves round a room.  Has anyone done anything like
this?

It kinda struck me that you'd be able to model things like acoustic
reflections off a hard surface, acoustic dispersion from soft surfaces as
well as getting things like occluders thrown in as a bonus.  Add to that you
could use the same techniques as are used for translucent surfaces to model
sound transfer through a surface and the slight bending of audio round
corners.

Are there any papers on the subject at all?  It struck me as something worth
finding more out about as it would give the ability to create truly rich
sound scapes with very little actual fiddling needed to get it right.  You'd
just pass the model through your PAT calculation engine then play sounds and
get them playing right.

Any thoughts?

Cheers!

Oscar

Re: [Algorithms] Spherical harmonics for room acoustic modelling

[Jon W. <jw...@gm...>]

You can calculate the impulse response of any environment analytically these
days.
The main problem is that it's really computationally expensive to apply a
very long convolution response to all sound.
Creative tried 10-15 years ago to drive up the compute power for sound, with
the Live! series (based on E-mu DSPs) and the (somewhat finicky) EAX API.
Another company, called Aureal, actually went a step further, and allowed
you to specify a crude representation of your geometry, but they died before
they could get mainstream traction.
However, what actually happened was that nobody cared, and sound is now just
a 50 cent DAC hooked to a raw DMA engine on the south bridge of the
motherboard, so any processing has to use that precious resource, the
very-expensive general-purpose core CPU.
Good luck telling your AI, rendering and physics people that you'd like 1/4
the CPU power for applying audio effects :-(

Sincerely,

jw


--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world. Nevertheless,
whether we get there willingly or not, we shall soon have lower consumption
rates, because our present rates are unsustainable.



On Sat, Jul 17, 2010 at 12:09 PM, Oscar Forth <
os...@tr...> wrote:

> I will admit first off I don't have a great knowledge of things like
> Pre-computed Radiance Transfer but I was having a chat with a colleague
> recently on how audio research lags behind video research in gaming.
>
> One idea that we were talking about is using spherical harmonics in a
> similar way to PRT to do some sort of Pre-computed Acoustic Transfer.
>
> Basically you'd add a new material type to a room that would give you the
> sound reflectivity of a given surface.   You could then use the same
> techniques (obviously calculated with something like OpenCL) to give you a
> good model of how sound moves round a room.  Has anyone done anything like
> this?
>
> It kinda struck me that you'd be able to model things like acoustic
> reflections off a hard surface, acoustic dispersion from soft surfaces as
> well as getting things like occluders thrown in as a bonus.  Add to that you
> could use the same techniques as are used for translucent surfaces to model
> sound transfer through a surface and the slight bending of audio round
> corners.
>
> Are there any papers on the subject at all?  It struck me as something
> worth finding more out about as it would give the ability to create truly
> rich sound scapes with very little actual fiddling needed to get it right.
>  You'd just pass the model through your PAT calculation engine then play
> sounds and get them playing right.
>
> Any thoughts?
>
> Cheers!
>
> Oscar
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>

Re: [Algorithms] Spherical harmonics for room acoustic modelling

[Jeff R. <je...@8m...>]

I don't have much to add on a technical level, other than to say I really
like the sound of this idea (no pun intended).

It does seem like maybe the problem has too many dimensions for just a
simple SH encoded precomputation though. My SH powers are limited, but in
lighting terms your problem sounds similar to determining the light passing
through a point in space (the listener's head) from a single local light
source.

The folks over at Crytek are doing some interesting things with "light
propagation volumes" for their new tech. Seems like a similar approach might
work for determining audio propagation as well.
http://www.crytek.com/technology/presentations/

Jeff

On Sat, Jul 17, 2010 at 2:09 PM, Oscar Forth <
os...@tr...> wrote:

> I will admit first off I don't have a great knowledge of things like
> Pre-computed Radiance Transfer but I was having a chat with a colleague
> recently on how audio research lags behind video research in gaming.
>
> One idea that we were talking about is using spherical harmonics in a
> similar way to PRT to do some sort of Pre-computed Acoustic Transfer.
>
> Basically you'd add a new material type to a room that would give you the
> sound reflectivity of a given surface.   You could then use the same
> techniques (obviously calculated with something like OpenCL) to give you a
> good model of how sound moves round a room.  Has anyone done anything like
> this?
>
> It kinda struck me that you'd be able to model things like acoustic
> reflections off a hard surface, acoustic dispersion from soft surfaces as
> well as getting things like occluders thrown in as a bonus.  Add to that you
> could use the same techniques as are used for translucent surfaces to model
> sound transfer through a surface and the slight bending of audio round
> corners.
>
> Are there any papers on the subject at all?  It struck me as something
> worth finding more out about as it would give the ability to create truly
> rich sound scapes with very little actual fiddling needed to get it right.
>  You'd just pass the model through your PAT calculation engine then play
> sounds and get them playing right.
>
> Any thoughts?
>
> Cheers!
>
> Oscar
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>



-- 
Jeff Russell
Engineer, 8monkey Labs
www.8monkeylabs.com

Re: [Algorithms] Spherical harmonics for room acoustic modelling

[Oscar F. <os...@tr...>]

>
> I don't have much to add on a technical level, other than to say I really
> like the sound of this idea (no pun intended).
>

badum tish! ;)


> It does seem like maybe the problem has too many dimensions for just a
> simple SH encoded precomputation though. My SH powers are limited, but in
> lighting terms your problem sounds similar to determining the light passing
> through a point in space (the listener's head) from a single local light
> source.
>

Effectively yes ... thats what you'd be doing.  You may be right on SH not
being up to it but thats why I thought I'd start this discussion :)


> The folks over at Crytek are doing some interesting things with "light
> propagation volumes" for their new tech. Seems like a similar approach might
> work for determining audio propagation as well.
> http://www.crytek.com/technology/presentations/


Cheers I'll take look.  Gonna have to switch my head over to "Understanding
unfathomable language mode" ... perhaps the few cans of cider wasn't SUCH a
great plan :D

Re: [Algorithms] Spherical harmonics for room acoustic modelling

[Oscar F. <os...@tr...>]

>
> You can calculate the impulse response of any environment analytically
> these days.
> The main problem is that it's really computationally expensive to apply a
> very long convolution response to all sound.
> Creative tried 10-15 years ago to drive up the compute power for sound,
> with the Live! series (based on E-mu DSPs) and the (somewhat finicky) EAX
> API.
> Another company, called Aureal, actually went a step further, and allowed
> you to specify a crude representation of your geometry, but they died before
> they could get mainstream traction.
> However, what actually happened was that nobody cared, and sound is now
> just a 50 cent DAC hooked to a raw DMA engine on the south bridge of the
> motherboard, so any processing has to use that precious resource, the
> very-expensive general-purpose core CPU.
>

This is a real shame because good sound REALLY enhances the sense of
immersion in any game. Echoes in a valley for example.

I guess the problem is that I've taken a 2.5 year hiatus from games dev and
have been working on hardcore audio processing algorithms.  As a result
whenever I play a game now I'm ACUTELY aware of just how badly wrong things
sound.  Its either hugely exaggerated or just plain wrong.


> Good luck telling your AI, rendering and physics people that you'd like 1/4
> the CPU power for applying audio effects :-(
>

hehehe ... I was more thinking about 1/4 of the GPU power as I'd have doubts
you'd get good enough response time on CPU alone ...

Re: [Algorithms] Spherical harmonics for room acoustic modelling

[Oscar F. <os...@tr...>]

I should also add that this is more from the point of view as to how it
would work.  Even if we don't have the power now we may well do soon and its
more of a spare time project I'm thinking of.

Also I still have a Diamond Monster MX300 lieing around.  I'd still use it
if you could get drivers these days :(

On 17 July 2010 21:04, Oscar Forth <os...@tr...> wrote:

> You can calculate the impulse response of any environment analytically
>> these days.
>> The main problem is that it's really computationally expensive to apply a
>> very long convolution response to all sound.
>> Creative tried 10-15 years ago to drive up the compute power for sound,
>> with the Live! series (based on E-mu DSPs) and the (somewhat finicky) EAX
>> API.
>> Another company, called Aureal, actually went a step further, and allowed
>> you to specify a crude representation of your geometry, but they died before
>> they could get mainstream traction.
>> However, what actually happened was that nobody cared, and sound is now
>> just a 50 cent DAC hooked to a raw DMA engine on the south bridge of the
>> motherboard, so any processing has to use that precious resource, the
>> very-expensive general-purpose core CPU.
>>
>
> This is a real shame because good sound REALLY enhances the sense of
> immersion in any game. Echoes in a valley for example.
>
> I guess the problem is that I've taken a 2.5 year hiatus from games dev and
> have been working on hardcore audio processing algorithms.  As a result
> whenever I play a game now I'm ACUTELY aware of just how badly wrong things
> sound.  Its either hugely exaggerated or just plain wrong.
>
>
>> Good luck telling your AI, rendering and physics people that you'd like
>> 1/4 the CPU power for applying audio effects :-(
>>
>
> hehehe ... I was more thinking about 1/4 of the GPU power as I'd have
> doubts you'd get good enough response time on CPU alone ...
>

Re: [Algorithms] Spherical harmonics for room acoustic modelling

[Deano <de...@ra...>]

Search for wave tracing in the literature. For acoustics, wave behavior is
vital unlike most lighting equations, as such you need to model waves
transfer and also for most audio surfaces unlike visual surfaces they are
not 100% reflective and so subsurface scattering is way more important.

 

There is some interesting work out there, including at least one GPGPU
real-time system.

 

Deano

 

From: Oscar Forth [mailto:os...@tr...] 
Sent: 17 July 2010 20:10
To: Game Development Algorithms
Subject: [Algorithms] Spherical harmonics for room acoustic modelling

 

I will admit first off I don't have a great knowledge of things like
Pre-computed Radiance Transfer but I was having a chat with a colleague
recently on how audio research lags behind video research in gaming.

 

One idea that we were talking about is using spherical harmonics in a
similar way to PRT to do some sort of Pre-computed Acoustic Transfer.

 

Basically you'd add a new material type to a room that would give you the
sound reflectivity of a given surface.   You could then use the same
techniques (obviously calculated with something like OpenCL) to give you a
good model of how sound moves round a room.  Has anyone done anything like
this?

 

It kinda struck me that you'd be able to model things like acoustic
reflections off a hard surface, acoustic dispersion from soft surfaces as
well as getting things like occluders thrown in as a bonus.  Add to that you
could use the same techniques as are used for translucent surfaces to model
sound transfer through a surface and the slight bending of audio round
corners.

 

Are there any papers on the subject at all?  It struck me as something worth
finding more out about as it would give the ability to create truly rich
sound scapes with very little actual fiddling needed to get it right.  You'd
just pass the model through your PAT calculation engine then play sounds and
get them playing right.

 

Any thoughts?

 

Cheers!

 

Oscar

Re: [Algorithms] Spherical harmonics for room acoustic modelling

[Oscar F. <os...@tr...>]

Sweet!  I'll take a look, ta :)

You wouldn't happen to have any links that would save me a bit of time and
google trawling?  No worries if not :)

On 17 July 2010 21:03, Deano <de...@ra...> wrote:

>  Search for wave tracing in the literature. For acoustics, wave behavior
> is vital unlike most lighting equations, as such you need to model waves
> transfer and also for most audio surfaces unlike visual surfaces they are
> not 100% reflective and so subsurface scattering is way more important.
>
>
>
> There is some interesting work out there, including at least one GPGPU
> real-time system.
>
>
>
> Deano
>
>
>
> *From:* Oscar Forth [mailto:os...@tr...]
> *Sent:* 17 July 2010 20:10
> *To:* Game Development Algorithms
> *Subject:* [Algorithms] Spherical harmonics for room acoustic modelling
>
>
>
> I will admit first off I don't have a great knowledge of things like
> Pre-computed Radiance Transfer but I was having a chat with a colleague
> recently on how audio research lags behind video research in gaming.
>
>
>
> One idea that we were talking about is using spherical harmonics in a
> similar way to PRT to do some sort of Pre-computed Acoustic Transfer.
>
>
>
> Basically you'd add a new material type to a room that would give you the
> sound reflectivity of a given surface.   You could then use the same
> techniques (obviously calculated with something like OpenCL) to give you a
> good model of how sound moves round a room.  Has anyone done anything like
> this?
>
>
>
> It kinda struck me that you'd be able to model things like acoustic
> reflections off a hard surface, acoustic dispersion from soft surfaces as
> well as getting things like occluders thrown in as a bonus.  Add to that you
> could use the same techniques as are used for translucent surfaces to model
> sound transfer through a surface and the slight bending of audio round
> corners.
>
>
>
> Are there any papers on the subject at all?  It struck me as something
> worth finding more out about as it would give the ability to create truly
> rich sound scapes with very little actual fiddling needed to get it right.
>  You'd just pass the model through your PAT calculation engine then play
> sounds and get them playing right.
>
>
>
> Any thoughts?
>
>
>
> Cheers!
>
>
>
> Oscar
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>

Re: [Algorithms] Spherical harmonics for room acoustic modelling

[Don W. <don...@ya...>]

You might want to take a look at Ambisonics as a good starting point:

    http://en.wikipedia.org/wiki/Ambisonics

The Furse-Malham set extend this to 2nd and 3rd order SH:

    http://www.muse.demon.co.uk/3daudio.html

This kind of tech was effectively used in DiRT by Codemasters:

    
http://blog.ambisonia.com/2007/08/30/interview-with-simon-goodwin-of-codemasters-on-the-ps3-game-dirt-and-ambisonics/


And of course, you can rotate them... ;)

Cheers,
- Don

    





________________________________
From: Oscar Forth <os...@tr...>
To: Game Development Algorithms <gda...@li...>
Sent: Sat, July 17, 2010 9:22:26 PM
Subject: Re: [Algorithms] Spherical harmonics for room acoustic modelling

Sweet!  I'll take a look, ta :)

You wouldn't happen to have any links that would save me a bit of time and 
google trawling?  No worries if not :)


On 17 July 2010 21:03, Deano <de...@ra...> wrote:

Search for wave tracing in the literature. For acoustics, wave behavior is vital 
unlike most lighting equations, as such you need to model waves transfer and 
also for most audio surfaces unlike visual surfaces they are not 100% reflective 
and so subsurface scattering is way more important.
> 
>There is some interesting work out there, including at least one GPGPU real-time 
>system.
> 
>Deano
> 
>From:Oscar Forth [mailto:os...@tr...] 
>Sent: 17 July 2010 20:10
>To: Game Development Algorithms
>Subject: [Algorithms] Spherical harmonics for room acoustic modelling
> 
>I will admit first off I don't have a great knowledge of things like 
>Pre-computed Radiance Transfer but I was having a chat with a colleague recently 
>on how audio research lags behind video research in gaming.
> 
>One idea that we were talking about is using spherical harmonics in a similar 
>way to PRT to do some sort of Pre-computed Acoustic Transfer.
> 
>Basically you'd add a new material type to a room that would give you the sound 
>reflectivity of a given surface.   You could then use the same techniques 
>(obviously calculated with something like OpenCL) to give you a good model of 
>how sound moves round a room.  Has anyone done anything like this?
> 
>It kinda struck me that you'd be able to model things like acoustic reflections 
>off a hard surface, acoustic dispersion from soft surfaces as well as getting 
>things like occluders thrown in as a bonus.  Add to that you could use the same 
>techniques as are used for translucent surfaces to model sound transfer through 
>a surface and the slight bending of audio round corners.
> 
>Are there any papers on the subject at all?  It struck me as something worth 
>finding more out about as it would give the ability to create truly rich sound 
>scapes with very little actual fiddling needed to get it right.  You'd just pass 
>the model through your PAT calculation engine then play sounds and get them 
>playing right.
> 
>Any thoughts?
> 
>Cheers!
> 
>Oscar
>------------------------------------------------------------------------------
>This SF.net email is sponsored by Sprint
>What will you do first with EVO, the first 4G phone?
>Visit sprint.com/first-- http://p.sf.net/sfu/sprint-com-first
>_______________________________________________
>GDAlgorithms-list mailing list
>GDA...@li...
>https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
>Archives:
>http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>



      

[Algorithms] Fwd: Spherical harmonics for room acoustic modelling

[Oscar F. <os...@tr...>]

I got this mail direct to my e-Mail so I've forwarded it to the mailing list
:)

Thanks Stephen.  VERY interesting post :)

---------- Forwarded message ----------
From: stephen mccaul <ste...@gm...>
Date: 17 July 2010 21:44
Subject: Re: [Algorithms] Spherical harmonics for room acoustic modelling
To: os...@tr...


My list response bounced. silly email addresses.

So i will send it to you directly.



http://en.wikipedia.org/wiki/Ambisonics

uses spherical harmonics to compute a 3d approximation of the pressure about
a point.

There are a variety of people working on real time room response technology.

Here are a few papers:

http://www-sop.inria.fr/reves/Nicolas.Tsingos/publis/ntsin_aes35_reverb.pdf
http://www.aes.org/e-lib/browse.cfm?elib=15157
http://www.mee.tcd.ie/~gkearney/ResearchPapers/Masterson_Kearney_and_Boland_AES35_Submission


Its a bit expensive to run a convolution per voice given today's consoles
(maybe on the ps3 or if you wrote
your own decoder so you can keep the signal in the frequency domain from the
get go.





Oscar Forth wrote:

> I will admit first off I don't have a great knowledge of things like
> Pre-computed Radiance Transfer but I was having a chat with a colleague
> recently on how audio research lags behind video research in gaming.
>
> One idea that we were talking about is using spherical harmonics in a
> similar way to PRT to do some sort of Pre-computed Acoustic Transfer.
>
> Basically you'd add a new material type to a room that would give you the
> sound reflectivity of a given surface.   You could then use the same
> techniques (obviously calculated with something like OpenCL) to give you a
> good model of how sound moves round a room.  Has anyone done anything like
> this?
>
> It kinda struck me that you'd be able to model things like acoustic
> reflections off a hard surface, acoustic dispersion from soft surfaces as
> well as getting things like occluders thrown in as a bonus.  Add to that
> you
> could use the same techniques as are used for translucent surfaces to model
> sound transfer through a surface and the slight bending of audio round
> corners.
>
> Are there any papers on the subject at all?  It struck me as something
> worth
> finding more out about as it would give the ability to create truly rich
> sound scapes with very little actual fiddling needed to get it right.
>  You'd
> just pass the model through your PAT calculation engine then play sounds
> and
> get them playing right.
>
> Any thoughts?
>
> Cheers!
>
> Oscar
>
>
>
> ------------------------------------------------------------------------
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
>