#69 GLMixer as Audio - Musique Concret interface.

[DJonsson]

I know this was mentioned before by Bruno, still though was wondering if GLMixer could be used as high quality wav file mixer at some point in time. There was talk about combining video mixing with audio, so that the volume of clips would be analogous to the brightness of the images.

Interesting concept. The thing is the GLMixer interface could provide a more organic way to mix various sounds in the production of audio art, that that of using sliders, combined with video clips could also develop some interesting results.

[RAUL ESPINOSA]

If someone mentions to me how GLMixer handles the "analogous to the brightness of the images" I could help implement something basic, in audio.

[DJonsson]

On 9/12/2018 6:16 PM, RAUL ESPINOSA wrote:

If someone mentions to me how GLMixer handles the "analogous to the
brightness of the images" I could help implement something basic, in
audio.

Volume would equal brightness. Or brightness would equal volume.
Wave files or perhaps video clips with audio tracks could also be used,
Simply using this intuitive interface for sound with waves could be enough.

I volunteer to test this feature.

Thanks - D.L. Jonsson

[tickets:#69] https://sourceforge.net/p/glmixer/tickets/69/ GLMixer
as Audio - Musique Concret interface.

Status: open
Milestone: 2.0
Created: Tue May 22, 2018 03:49 PM UTC by DJonsson
Last Updated: Tue May 22, 2018 03:50 PM UTC
Owner: nobody

I know this was mentioned before by Bruno, still though was wondering
if GLMixer could be used as high quality wav file mixer at some point
in time. There was talk about combining video mixing with audio, so
that the volume of clips would be analogous to the brightness of the
images.

Interesting concept. The thing is the GLMixer interface could provide
a more organic way to mix various sounds in the production of audio
art, that that of using sliders, combined with video clips could also
develop some interesting results.

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/glmixer/tickets/69/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

[Bruno Herbelin]

Thanks Raul for offering to contribute.

Implementing audio sources would not be so hard in practice. The equivalent of 'VideoFile' class should be done for 'AudioFile', and linked to a new type of source 'AudioSource' : the volume could then be linked to the opacity [0.0 1.0] which is changed in the mixing view.

The more conceptual problem I always considered for integrating audio within glmixer is the possible confusion / incompatibility between visual and audio mix: showing two videos should not imply mixing their audio as this only produce noize! And having icons of pure audio sources (e.g. mp3) in the middle of graphics sources can be confusing.

Would you propose to add a special view for audio ?
Or a special type of source in the mixing view?

[DJonsson]

On 9/13/2018 10:23 PM, Bruno Herbelin wrote:

Thanks Raul for offering to contribute.

Implementing audio sources would not be so hard in practice. The
equivalent of 'VideoFile' class should be done for 'AudioFile', and
linked to a new type of source 'AudioSource' : the volume could then
be linked to the opacity [0.0 1.0] which is changed in the mixing view.

The more conceptual problem I always considered for integrating audio
within glmixer is the possible confusion / incompatibility between
visual and audio mix: showing two videos should not imply mixing their
audio as this only produce noize!

Thanks all for this discussion!

Here are some ideas and thoughts.

Compatibility between visual and audio

In summary the interface could include
_allow videos to have their audio turned on or off.
_allow audio to be seen with a speaker icon tagged w/ number or text ie.
...
other components/controls of length, looping, end with black=silence,
start from beginning
are natural

possible confusion / incompatibility between visual and audio mix

As we know already music and visual arts share harmony (composed
dissonance/consonance),
rhythm (resonance of form) with other elements directly related. The
basic elements can
be broken down.

Showing two (or more) videos and mixing the sound, would simply require
thoughtful
pre-production for the clips, sequences and glm sessions similar to how
GL Mixer
can currently be used.

Directly analogous to the visual montage methods / considerations we
make with GL Mixer.

In Video

• elements of colour,
• contrast,
• sufficient negative (or black) space to enable other images to shine
  through
• narrative or other structure...
• and so on.

In Audio

• musical key
• scales
• contrast
• rhythm
• equalization (allowing space in the audio spectrum for sound to shine
  through).
• narrative or other structure...
• and so on.

And having icons of pure audio sources (e.g. mp3) in the middle of
graphics sources can be confusing.

Would require some flight time to develop technique, like any tool used
for art.
A speaker icon representing audio would do the job.

Would you propose to add a special view for audio ?

No, the view should remain the same, that is the beauty of GL Mixer
its simplicity and intuitive GUI/Metaphor.

Or a special type of source in the mixing view?

A widget might be handy for setting left-right balance, adding
echo, equalization...
Still keeping things as simple as possible in the first revision
might be wise, and in fact may all that is needed.

I hope that makes sense.

Again thanks everyone for this discussion.

Darrell Jonsson

---

[tickets:#69] https://sourceforge.net/p/glmixer/tickets/69/ GLMixer
as Audio - Musique Concret interface.

Status: open
Milestone: 2.0
Created: Tue May 22, 2018 03:49 PM UTC by DJonsson
Last Updated: Wed Sep 12, 2018 04:16 PM UTC
Owner: nobody

I know this was mentioned before by Bruno, still though was wondering
if GLMixer could be used as high quality wav file mixer at some point
in time. There was talk about combining video mixing with audio, so
that the volume of clips would be analogous to the brightness of the
images.

Interesting concept. The thing is the GLMixer interface could provide
a more organic way to mix various sounds in the production of audio
art, that that of using sliders, combined with video clips could also
develop some interesting results.

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/glmixer/tickets/69/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

[Bruno Herbelin]

Your suggestions start to make sense in my mind.
I am interested in this challenge of natural user interface to make sense of audio and visual mix.
Starting to have ideas on how to program this :)

Raul, would you be ok to help in providing the class AudioFile.h / .cpp ?
It would ideally be the audio equivalent of :
https://sourceforge.net/p/glmixer/Source/HEAD/tree/trunk/src/VideoFile.h

We would in addition need a global AudioManager : I have no idea how to implement a multi platform (Windows, OSX and Linux) audio mixer.

The option to use the audio layer from Qt is not recommended as it is obsolete (the whole GUI of GLmixer should be ported to Qt5). Any recommendations ?

[Jorge Rosa]

Hello all.
THANKYOU Bruno for your fantastic work and share it with us all!
(I don´t know if you are looking for an audio layer, but...) I´m using SFML (it contains a simple, yet powerfull audio support, and its "crossplatform").

• Only plays "free" audio codecs, of course, but this is not an issue, in my opinion:
  https://github.com/SFML/SFML/tree/master/src/SFML/Audio
• Hope this reference helps... :/

[DJonsson]

Your suggestion looks interesting are there any images of the interface anywhere?

Looking at documentation it appears to be more a library for developers, than
a GUI friendly application, please correct me if I'm mistaken.

In the past I've used sox and linux scripting for such tasks, I suppose though SFML
would provide more control and options.

https://www.sfml-dev.org/tutorials/2.5/audio-sounds.php