Re: [Audacity-devel] phonetic diagrams (vowel sound formant analysis) - where to start?

[James C. <cr...@in...>]

David Shen wrote:

> I read the Audacity project of the month on the sourceforge web site.
> In the "Where do you see your project going" section, James mentioned
> that to anticipate new GUI components, such as surrounds sound controls,
> diff displays and phonetic diagrams.  I am interested in the phonetic
> diagrams (vowel sound formant analysis) because I don't pronounce
> certain vowel correctly. :)

Hi David.

The feature I'm planning is to help me pronounce words in German -
it should also help foreign langauge students trying to get the
differences between 'ship' and 'sheep' and similar.  Visually it will
be a grid with the option to display the phonetic symbols for various
vowel sounds, or alternatively representative words with those
sounds, at various positions.  You'll say a word like 'sheep' and
Audacity will then show you where on the grid what you just
said lands up, so you can see how close you are to the target
words.  Clicking on any point on the grid will also generate the
appropriate sound - so we'll also have a kind of word-sound
morpher.  I'm planning too to re-use code from the spectral
plots to show you what the dominant frequencies are in the
'vowel part' of the sound.

However, this is still a long way off in the future and other
changes need to happen first.  You've probably read
about the Mezzo initiative which is reworking the wave-form
code, making it fully independent of the GUI and at the same
time scriptable.  My intention is that the GUI will get a similar
working-over, but in a more one-step-at-a-time way.

> I have downloaded the audacity source code and compile it on my Windows XP
> with Visual C++ 6.0.  Since I am a newbie, do you have any recommendation
> on what I should do next?  (e.g. which C++/C code should I be familiar
with?
> Is there any book/article that I should read?)

So much depends on where your interests lie.

If you're interested in the GUI side and writing new widgets,
e.g. to display vowel sound hits and misses, you need to
immerse yourself in the wxWidgets documentation, and try out
some of their sample apps.  Alternatively you could look at
functions like OnPaint() in Meter.cpp in Audacity to see how
Audacity creates new GUI elements using the drawing primitives.
If you're already familiar with wxWidgets, the best way to get
familiar with the Audacity GUI code is to have in mind some
changes you would like to make.  One suggestion is that
you look at the spectral plot, Audacity, view->plot-spectrum,
the code for which is in file FreqWindow.cpp, and figure out
how to make the labels for the axes of the graph label the actual
grid lines rather than as currently off to the right or below.  It's
something that I've meant to fix for a while but never got round
to.  That plot, incidently, is at the moment your best guide to
visually seeing differences in vowel sounds.  You should be able
to see the three dominant frequencies in a vowel sound from it.

If you're interested in helping make the sound anlaysis feature a
reality sooner, e-mail me off-list and we can discuss the stages to
getting there.

Best regards,

James.