I am pleased to introduce GSmartMix. GSmartMix has
recently been accepted to Google SoC. It aims is to bring a new sound
experience in Gnome. Thus, I guess this effort should be discussed
here. Ensonic, Stefan Kost, is the mentor of this project. Raphel
Slinckx is the co-mentor.
If you are not already familiar with this project, you could have a look at http://live.gnome.org/GSmartMix
before reading this. I will also do my best to change the GNOME
sound support, but *it's not* the main goal. For the moment, GSmartMix
is a project that I want to design to be run aside and optionnaly,
without any API change.
The following are some technical suggestions on GSmartMix, I would like to discuss with you.
GNOME API and esd discussions will go in gnome.multimedia mailing list.
I don't make any difference between rule or class for the moment: it's a description of the sound: "music", "voice", "event"...
=== UI ===
The ui provide a per class sound level sliders, and some rule parameters:
- foreground sound level
- background sound level
- fading speed
- and list of class that we put in foreground when this class has to play
(queue policy: first in stay in fg (fifo), last in go in fg (lifo), none (all fg))
I would like to add a new tab in the GNOME "Volume Control" application with this content:
Regarding the Topaz mockup of volume controler and the per-app volume control (
GSmartMix benefits will partially fit this. I don't know how we could
deal with "multiple instance of an application" problem... This is why
I was briefly thinking of a Metacity integration, in order to
differenciate each sound source easily. But I am wondering if one
"foreground app" per class for common sounds (voice, video, music..)
will help to clean up this list. This way, only "foreground app" will
be shown in the list. Only some class, like "events" won't need this
"front" scheme. We don't care of a per app "event" sound level
probably.Is it really what we want?
=== Sinks: a new element or a gstbaseaudiosink integration ===
I have tried to depict my initial overview of GSmartMix here:
Different design solutions can be thought up. What will be the best or the consensus?
==== "Base patch" solution ====
The idea is to handle
dbus communication at the baseaudiosink class level. ensonic, how would
be handled the volume control/transformation at this level?
This way, every audiosink will be seen over dbus.
- all compiled audio sinks will have a libdbus dependency (if we compile gstlib with this support)
- intrusive patch in gst
- every app will have to suffer/benefit the smart level control
==== "New element" solution ====
Formerly, I was thinking
of a kind of GStreamer "identity" bin elements, inserted before the
real audio sink. They will catch some gstramer events
(status/streaminfo..) and send them over dbus. After that they will
obey GSmartMix server orders and update a volume level. The "volume"
element manipulated would be found by looking up the parent bin. The
"volume_element" property could be overriden. If none is found, a new
one will be added in GSmartSink bin in order to control volume output.
This way, I hope to be able to provide GSmartMix without having to
change anything in the application side and in GStreamer.
Look at this sketch:
- another plugin => a different pipeline
- all applications/usage will not benefit from GSmartMix (those who use directly alsasink for instance)
- more complex solution?
==== Polypaudio ====
jeff pointed me out Polypaudio. I
was not thinking hard of using it, because it sounded like an odd but
good esd replacement. Which is not the aim of my work. But, I must
admit I am quite impressed by its state. It features some of the
required interfaces for my project. Definetly, my will is to fill the
gab between gnome, gstreamer and polypaudio.
Recent discussion with lennart on #polypaudio about GSmartMix: http://pastebin.com/747392
Drawback (if the effort is to patch polypaudio):
- the volume level adjusted at the server side will not be updated
in the client side (for example, the volume slider of your favourite
music player): I believe I can fix this or is there already a solution
in polypaudio? The polypaudio author, Lennart, will post on this
subject, onthe GStreamer GstMixer flaw.
- we still depends officialy on a sound server (and not on ALSA/dmix), which could be badly interpreted
==== Remarks ====
- I believe sinks must initialise and "block" sound output until the dbus server did not reply to adjust their level.
- of course, I would also like to bring new things to the
client side, like the ability to describe sound, like the ability to
give sound "class" and "priority". Could it be done with standard
GStramer API, with a bus message ?
=== GSmartMix, a dbus server ===
I like the simplicity
of devil's pie. I wonder if I could improve s-exp interpreter to
support GSmartMix needs (fifo on class, loops on other sinks ?). So
that the server could be written with light C code. If more freedom is
required, Python is probably the language of choice.
But, it could be written differently.
I also wonder how
the dbus server could monitor its slaves: the communication need to be
in a "connected" mode with the peer (I think of an application crash,
an abnormal behaviour...).
Marc-André Lureau, GSmartMix