[Libxtract-devel] libxtract, SC UGens and FFT

[Dan S. <dan...@el...>]

Hi Jamie,

Sorry you haven't heard much from me re libxtract. Would still like to 
use it, and I'm currently thinking about how best to use it in SC UGens.

The main thing I want to do right now is FFT-based feature extraction. I 
would much rather use SC's existing FFT infrastructure, rather than have 
libxtract "repeat" the FFT (mainly for efficiency's sake). So, for 
example, to use libxtract's flatness measure it would look something like:

   chain = FFT(buf, sig);
   flatness = Flatness.kr(chain);

The problem with this is that libxtract wants to see the FFT data in a 
different format. So the Flatness UGen would need to convert the FFT 
data to a list of magnitudes + a list of frequencies. Again, for 
efficiency it would be bad for each ugen to repeat this conversion, so 
I'm thinking it may be best to create a UGen which does the conversion, 
which the feature extractors can hang off the end of.

If for example it's called "Mags", the process would go something like this:

   chain = FFT(buf, sig);
   Mags(chain, magbuf); // "magbuf" will be filled with data in 
libxtract-style format
   flatness = Flatness.kr(chain, magbuf); // "chain" is needed so we 
know when to fire, but "magbuf" contains the data that will be processed

It might seem an odd intermediate stage from an SC user's point of view, 
but actually the Mags UGen would be quite useful - it would have the 
side effect of loading the magnitudes into a buffer in a form that could 
easily be accessed and manipulated from the lang side.

I'd guess that it should be possible to create an analogous UGen using 
the peak-picker, to make it easy to apply the same measures but only to 
the located peaks. That's not my focus though.

I'd appreciate your thoughts on this. I'll probably be doing some coding 
this week along these lines.

Dan

-- 
Dan Stowell
Centre for Digital Music
Queen Mary, University of London
Mile End Road, London E1 4NS
http://www.elec.qmul.ac.uk/department/staff/research/dans.htm
http://www.mcld.co.uk/