Windows / Realtek "Speaker Fill", Basic Upmix Stereo to 5.1, and Equalizer APO.

[OscarL]

Hello guys.

I just wanted to share some info regarding the use of Equalizer APO (EAPO) and/or Microsoft's (or Realtek's) "Speaker Fill".

Before installing EAPO, the Realtek control pannel for my 5.1 onboard soundcard (HDA Audio, codec: ALC662) used to show a checkbox for "Speaker Fill" (when setting the system in 5.1 mode).

After installing EAPO, that control disapeared.

It seems that that "effect" is not really from Realtek software, but standard from Windows (can be seen in the "Enhancements" tab in the "Properties" dialog for a given "Audio Device" from Control Panel).

The only way to get it back, is using EAPO's Configurator.exe's troubleshooting options, and install EAPO as post-mix only (I'm also installing it as "SFX/EFX", not sure if other options work).

Installing EAPO as pre-mix, disables the "Speaker Fill", and also the "Loudness Correction" APOs (also from Microsoft).

But... before I realized that... I ended using EAPO to have a basic "Upmix Stereo to 5.1" effect that I copied from somewhere.

Said upmix requires having EAPO installed as pre-mix (AFAIK), to only do the upmix in case of stereo input.

I'll copy that EAPO "code" here for reference:

---

# Basic Stereo Upmix to 5.1 (or 7.1... uncomment the SL/SR ones):
Stage: pre-mix
If: inputChannelCount == 2
Copy: MUP=0.5*L+0.5*R
Copy: C=MUP
Copy: SUB=MUP
Copy: RL=L
Copy: RR=R
#Copy: SL=RL
#Copy: SR=RR
EndIf:

---

The thing is... it works, but the effect is quite different from Microsoft's "Speaker Fill".

After much fiddling... I came up with an EAPO configuration that, at least for me, sounds exactly like what "Speaker Fill" does.

Just in case someone else is interested in using something like that, while also using EAPO for other things (I use it to set the Front Panel Headphones jack as "Rear Channels", and the Back Panel "Side Speakers" jack as "Front Speakers", this is needed due to some hardware quirks of my setup), I'm posting my config here.

Anyway.... here's my attempt to mimic Microsoft's Speaker Fill APO using EAPO:

---

# Mimic Microsoft's "Speaker Fill" by Upmixing Stereo to 5.1 (or 7.1):
# The effect is as close as I could get it to be (not knowing the algorithm used by MS).
# The change of phase in RL/RR (the - sign) seems to help "expand" the surround channels, otherwise they feel too mono (tested with headphones as "side/rear" speakers).

Stage: pre-mix

If: inputChannelCount == 2
Copy: MIX=L+R
Copy: DIF=L+-1.*R
Copy: C=0.5*MIX
Copy: RL=-0.5*DIF
Copy: RR=0.5*DIF
Channel: RL RR
Filter: ON HP Fc 80 Hz

# I don't have a 7.1 system to see what MS does here, so... guess this will do for now:
Copy: SL=RL
Copy: SR=RR

# MS doesn't seems to do anything with the SUB channel, at least that I could detect. Maybe we should try to create some "bass boost" signal here?
Copy: SUB=0.5*MIX
Channel: SUB
Filter: ON LP Fc 600 Hz
Filter: ON LS Fc 300 Hz Gain 3 dB
EndIf:

---

I've tested this with music and movies. Using headphones in each output jack, and comparing the effects of this config against Speaker Fill.

As I've said... they sound pretty much alike to me. Maybe some of you can refine it, or at least see what it does in a 7.1 setup.

The main difference between "Speaker Fill" and the "Basic Upmix" is that usually the voices remain in the front outputs only (thanks to the cheap trick of substracting R from L), while the sounds that are stereo only, get sent to the rear channels. The basic upmix simply copies the front channels to back, and creates a mono mix for Center and SUB.

Ok, that is all. Maybe some of you find it interesing to toy with. Feel free to correct me in anything!

--
OscarL

P.S.

Sorry if the English or the name of things doesn't makes much sense (my English is self-taught, and my Windows install is in Spanish, so I'm "translating" the name of some things, and have no idea if those names are the one used in English).

PS2: Equalizer APO ROCKS!

(edit 1: fixed the SR = SR typo, to SR=RR as intended)
(edit xx: reduced the gain of the fake sub signal to 3 dB from a crazy 12 dB)

[OscarL]

As a follow up, I'd like to mention that HeSuVi also has an interesting upmix from stereo AND 5.1, to 7.1.

Said upmixes can be found in HeSuVi's matrix.txt and matrix51.txt files, as one-liners.

The following is just an "expansion" of those one-liners, just for readability's sake:

# HeSuVi's upmix
Stage: pre-mix
If: inputChannelCount == 2
  Copy: L=0.5*L
  Copy: R=0.5*R
  Copy: RL=0.3*L+-0.2*R
  Copy: RR=-0.2*L+0.3*R
  Copy: C=0.2*L+0.2*R
  Copy: SUB=0.0
  Copy: SL=0.45*L+-0.25*R
  Copy: SR=-0.25*L+0.45*R
EndIf:

If: inputChannelCount == 6
  Copy: RL=0.4*5+0.1*6
  Copy: RR=0.1*5+0.4*6
  Copy: SL=0.15*L+-0.10*R+0.45*5
  Copy: SR=-0.10*L+0.15*R+0.45*6
EndIf:

Personally, I tend to comment out those L=0.5*L R=0.5*R, as they attenuate too much the front signals for my taste, but... otherwise, an interesting config to play with.

Compared to Microsoft's Speaker Fill, HeSuVi's feels less drastic. Should be interesing to try it with the full 7.1 setup that, sadly, I don't have :-)

Anyway... just another interesting option to consider for your upmixing needs.

[Peter Verbeek]

Great effort to create an upmix. It will be appreciated. A few remarks. First of all the "Speaker fill" is probably from Realtek. It's an audio enhancement or better said an Audio Processing Object (APO) like Equalizer APO is. APO's are part of the driver software of a manufacturer, in this case Realtek. Each manufacturer has different APO's so every upmix may differ per driver and per manufacturer. It's difficult to get the specs of such APO's so also the upmixing method.

Also Equalizer APO tries to leave the existing APO's activated but sometimes there is a conflict between the APO's of a device and Equalizer APO. If so, there isn't much to do then install Equalizer APO either on the post-mix or on the pre-mix part of the audio stream. By unchecking the "Use original APO" checkboxes the audio enhancements of the driver will be deactivated hence loosing things like upmix/speaker fill and other enhancements.

And you correct by using the "Stage: pre-mix" command before using the inputChannelCount variable. But this means the Equalizer APO should be installed on the pre-mix part. Else the stage command is ignored and the inputChannelCount won't always reflect the proper number of channels of the input audio source.

[OscarL]

Thanks for the explanations, Peter! Much appreciated!

I had assumed that said "Speaker Fill", and the "Loudness Equalizer" enhancements were from Microsoft and not from Realtek, because those two are the only ones that read "Provider: Microsoft", while the others read "Provider: Realtek" when you select them in the "Enhancements" tab for a particular audio device properties. ¯\_(ツ)_/¯

Anyway... I'm very happy to be able to leave E-APO installed both as pre and post-mix... and being able to play around with things like this :-)

One thing I've noticed regarding my version of the Speaker Fill (and also present with the original one)... in some "heavily" compressed audio tracks... it tends to generate some unpleasant artifacts in the rear/side channels. I guess that "cancelling the common sounds" tends to make some higher pitched, compression-related sounds more prominent (really noticeable with AAC <= 128 Kbps).

On those problematic tracks... HeSuVi's version seems much more pleasant to use, at least for my ears (as it is "less drastic" than the other effect).

Both "MS/Realtek" "Speaker Fill" and HeSuVi work best for movies, IMO, while I like the "copy front-to-back" upmixes for music (at least on the things I've tried so far).

Always good to have some choices!

Edit: The following docs also made me think they were from Microsoft:

https://github.com/MicrosoftDocs/windows-driver-docs/blob/staging/windows-driver-docs-pr/audio/audio-signal-processing-modes.md#audio-effects

[Peter Verbeek]

The following docs also made me think they were from Microsoft

They could certainly be but what I don't get is that sound enhancement vary per driver, sometimes per version, and per manufacturer. If from Microsoft they should publish it for every output audio device. This is entirely possible without any issues. Of course by using their own APO architecture. On the other hand, I'm glad they don't publish it for everything as more Equalizer APO issues may come from it.

I'm not sure if it has anything to do with the artifacts but looking at your upmix I do see some possible issues. Consider the first 2 lines:

Copy: L=0.5*L
Copy: R=0.5*R

These commands reduce both channels. This causes a lost of audio resolution (bits) and volume. These "0.5 channels" are then used in the next commands. This means this lower resolution/volume is then fed to the other channels. To avoid this you might use virtual channels. For instance the (simple) upmix in the Peace equalizer (a popular interface for Equalizer APO) does this:

Stage: pre-mix
If: inputChannelCount == 2
Copy: MUP=0.5*L+0.5*R
Copy: C=MUP
Copy: RC=MUP
Copy: SUB=MUP
Copy: RL=L
Copy: RR=R
Copy: SL=L
Copy: SR=R
EndIf:
Stage: post-mix

Where "MUP" is such a virtual channel that leaves L and R unaltered.

Another issue might be due to the phase transition (the - sign):

Copy: RL=0.3*L+-0.2*R

Although I'm not an audio expert, a phase change of 180 degrees seems odd to me.

This might be an issue but maybe intended this way:

Copy: SUB=0.0

This simply mutes the subwoofer (LFE channel).

And this is an oddity in itself:

Copy: RL=0.4*5+0.1*6
Copy: RR=0.1*5+0.4*6

As channels 5 and 6 are always RL and RR these commands are the same as:

Copy: RL=0.4*RL+0.1*RR
Copy: RR=0.1*RL+0.4*RR

When having an input of 6 channels (5.1) why mix the rear channels? The only ones that don't have audio are the side channels. A simple mixing of the front and rear channels to the sides will do.

[OscarL]

Hey Peter, thanks again for the reply!

Just to clarify... I had some artifacts (noticeable on some tracks) using my "Mimic Microsoft's/Realtek's Speaker Fill" version, and also using the "original Speaker Fill" from MS/Realtek.

The one you mention where the issues might be... it's the "expanded" version of HeSuVi's upmix from stereo or 5.1 to 7.1 (that I included for reference, as a follow up).

Notice that I started that config with # HeSuVi's upmix (on my second post), and "mine" with # Mimic Microsoft's "Speaker Fill" by Upmixing Stereo to 5.1 (or 7.1): (on my first original post).

In my version basically boils down to:

Copy: MIX=L+R
Copy: DIF=L+-1.*R
Copy: C=0.5*MIX
Copy: RL=-0.5*DIF
Copy: RR=0.5*DIF
Channel: RL RR

That phase change (while summing up L/R) is based on what I remember of a pretty old trick that was popular in the late 90s. There were plugins for Winamp (and its clones, later) that did something similar both to "extra stereo", and for "voice removal" purposes.

My intention being... to use, in the rear channels, only an attenuated copy of the "stereo-only sounds" present at front channels, in the hopes of providing a "fake surround" (leaving the voices/central sounds in the front).

Can't really talk about HeSuVi' author's intention, but feels as a more subtle version of what I came up with, trying to understand/emulate the "original" Speaker Fill. See "Foot Note" at the end.

That "front attenuation" in HeSuVi's version:

Copy: L=0.5*L
Copy: R=0.5*R

is why I wrote that I tend to comment-out those lines when using that config.

That Sub=0.0 is also from HeSuVi's version. In mine I seem to be trying to kill the sub-woffer with that crazy 12 dB gain "fake bass" filter I had!

I edited that down to 3 dB just in case someone tries the config on my first post.

Regarding that final oddity you mention:

Copy: RL=0.4*5+0.1*6
Copy: RR=0.1*5+0.4*6

seems to be part of HeSuVi's attempt to upmix 5.1 to 7.1, along with the next lines

Copy: SL=0.15*L+-0.10*R+0.45*5
Copy: SR=-0.10*L+0.15*R+0.45*6

I can't really test it, but doesn't sounds too crazy to me, in the context of trying to upmix a 5.1 source to 7.1.

I guess that my poor English didn't make clear enough with which versions I had some artifacts with, or that my posts weren't structured well enough to make them easier to follow. Sorry for that :-)

Foot Note:

Searching for those old plugins, XMMS's in particular (as they were open source)... I found the following in old info, that explains it better:

Effect: Extra stereo

Adds a greater sense of stereo depth to the MP3. At too high a level it degrades, but a small amount is nice!

Effect: Voice removal

Attempt to remove the vocals from a track. Depends on the music, it can work very well or make the track sound like it's played underwater.

That last one seems to agree with you Peter, and the culprit seems to be in the phase change :-D

But as noticed... can work very well depending on the track! (and that matches my experience).

That "underwater" effect describes what I was hearing. It is also an usual artifact in badly compressed audio, and that's why I assumed it was due to AAC tracks with not enough bitrate.

Ok.I dragged this long enough. Thanks for the patience!

[OscarL]

For what it's worth, I've found the source code for the mentioned old plugins:

https://github.com/p/xmms-gtk2/tree/master/Effect

A quick conversion to E-APO config, might look like this (minus the limiting to valid 16 bit values):

Voice removal:

Copy: L=R+-1.0*L 
Copy: R=L+-1.0*R

Extra Stereo:

Change the 2.5 value to adjust the strength of the effect.

Copy: AVG=0.5*L+0.5*R

Copy: LDiff=L+-1.0*AVG
Copy: RDiff=R+-1.0*AVG

Copy: L=AVG+2.5*LDiff
Copy: R=AVG+2.5*RDiff

It seems I ended up mixing up a bit of both in my head for my upmix :-D

Hey, don't blame me! I was drawing from memory of code I read almost 20 years ago :-D

Anyway... now that I read it... dunno why I ended up using a mono signal for the rear speakers (changed in phase, becase it being mono DID bothered me :-D).

Here's a slightly revised, and posibly final from my part, version, that better reflects my original intentions:

Mimic "Speaker Fill":

# Mimic "Speaker Fill" by Upmixing Stereo to 5.1 (7.1):
Stage: pre-mix
If: inputChannelCount == 2
  Copy: Center=0.5*L+0.5*R
  Copy: LDiff=L+-1.*R
  Copy: RDiff=R+-1.*L
  Copy: RL=0.5*LDiff
  Copy: RR=0.5*RDiff
  Channel: RL RR
  Filter: ON HP Fc 80 Hz

  If: outputChannelCount > 4
    Copy: C=Center
    # MS doesn't seems to do anything with the SUB channel. Let us fake a cheap SUB signal here:
    Copy: SUB=Center
    Channel: SUB
    Filter: ON LP Fc 600 Hz
    Filter: ON LS Fc 300 Hz Gain 3 dB
  EndIf:

  If: outputChannelCount > 6
    # I don't have a 7.1 system to see what MS does here, so... guess this will do for now:
    Copy: SL=RL
    Copy: SR=RR
  EndIf:
EndIf:

I'll stop spamming now :-D

[Peter Verbeek]

Well, that's last upmix config seems about spot on to me :) The usage of the phase is a nice addition. It does give a sense of depth or whatever one would call it. Great. Perhaps a trick I can introduce on the Effects panel of the Peace equalizer with a TM of OscarL ;)

Perhaps ending the config with post-mix:

Stage: post-mix

This is less CPU intensive as can be read in the Stage command wiki. Unless of course Equalizer APO isn't installed on the post-mix part.

Neat trick with the voice removal. I'm not an expert on audio matters so I'm curious how this could/would work.

There's only what possible issue left. There's a note by the developer of Equalizer APO stating possible issues arising from filtering the low frequencies of a subwoofer (LFE channel). I guess it's a matter of trying it.

[Francois Baart]

Hi Oscar. This is exactly what I was looking for. Thanks

"A Way To take a Chrome Stereo movie (Because chrome will only output stereo even if the source is multichannel) and expand it to fill my 5.1 speaker setup. "

For stereo music upmix I use a plain:

Stage: pre-mix
  Copy: C=0.5*L+0.5*R RL=L RR=R LFE=0.5*L+0.5*R

A Stereo MOVIE Mix to 5.1 Speakers seems like such an easy thing to do. Calculate the common sound and copy it to the center channel, and only play the left and right unique sounds on the left and right.

Your solution worked quite well but I noticed a loss of stereo separation, and some nasty artifacts.
*Stereo separation loss because the Left and right only sound is also played on the Centre channel.
Strange Artifacts because of the problem below.

You sent me down a path of learning and frustration. :)

Copy: LDiff=L+-1.*R

I Realized that with the Copy command that you showed: the Common sound between left and right is removed (Great) , but that we end up with the Left only portion AND the inverse of the Right only portion(Bad).

I had to go back to my Math text books from school to realize there is actually no precise way to do this:
Left Stereo = Common+Left Ls = C+L
Right Stereo = Common+Right Rs = C+R
From these two equations Calculate C or L or R (So 4 variables, and 2 Unknowns, cannot be solved)

Some years ago I had a Yamaha Surround AV processor with an upmix function, that did an amazing job. Anybody have any idea what method such AV processors use to upmix?
and maybe a way to do something like this in EqualizerAPO?

[Fabo W]

you need to use a matrix https://en.wikipedia.org/wiki/Matrix_decoder

note that the stereo seperation and phase issues is always a problem while upmixing. even dolby matrixes have loss. afaik the only way to extract a center without loss is https://en.wikipedia.org/wiki/Trifield

I would solve the issue at the root: why use Chrome to play back a movie? don't!

[Francois Baart]

Thanks for the pointers Fabo

There is some heavy reading and quite a lot that's way over my head. So I will certainly do a lot of Learning from those and related sites.

I Totally agree with: solve the issue at the source, and I do when I can, by using for example the Netflix app (full 5,1 support) instead of the Netflix web site through Chrome (Stereo only).
But for now there is no YouTube app for Windows, so even if the Youtuber encoded in 5.1, I am left with Chrome's stereo downmix.

[Francois Baart]

Here is my config.txt for those that are interested

I use the Realtek Audio control utility to manage the speaker distance and Attenuation.
IMPORTANT: Set "Bass Management enhancement " to Off

# Stereo Movie Mix to 5.1

# Change this to your device, or Remove completely.
Device: Speakers Realtek(R) Audio {e2f682a1-2299-4dc5-afd6-39a7f98211b9}

Stage: pre-mix
If: inputChannelCount == 2
  Copy: Mix=0.5*L+0.5*R

   # The line below is a best effort as we get the Left Audio(good), and the inverted right audio (undesirable)
   Copy: LDiff=Mix+-1.0*R

   # The line below is a best effort as we get the Right Audio(good), and the inverted Left audio (undesireable)
   Copy: RDiff=Mix+-1.0*L

   #  Line below seemed like a good idea but After some math I realized that Adding LDiff and RDiff will always result in Zero
   # Copy: Mono=Mix+-0.5*LDiff+-0.5*RDiff


 #Channel Assignment Section
   # This gives back a bit more stereo separation but introduces more artifacts.
   Copy: L=L+0.5*LDiff R=R+0.5*RDiff 

   Copy: C=Mix RL=LDiff RR=RDiff LFE=Mix

  # Uncomment the line below to test just the Left and Right unique audio
  # Copy: L=LDiff R=RDiff C=0.0 RL=0.0 RR=0.0 LFE=0.0


 # This is to sort of enhance the speech on the center channel. Comment out if you don't like this effect
  Channel: L R RL RR LFE
      Preamp: -2 dB
  Channel: C
      Filter: ON HS Fc 4000 Hz Gain -2.0 dB
      Filter: ON LS Fc 1000 Hz Gain -2.0 dB

EndIf:

# Bass Management Section based on speaker physical bass capability
# Frequencies shown here are specific to my speakers, check your speaker specs for your values.
# We need this because we disabled Bass Management in the Realtek config.
# Duplicated filters is to get a steeper roll off (more of a LP OR HP cut)
Channel: L R
  Filter: ON HP Fc 44 Hz
  Filter: ON HP Fc 44 Hz
  Filter: ON HP Fc 44 Hz
Channel: C
   Filter: ON HP Fc 70 Hz
   Filter: ON HP Fc 70 Hz
   Filter: ON HP Fc 70 Hz
Channel: RL RR
  Filter: ON HP Fc 85 Hz
  Filter: ON HP Fc 85 Hz
  Filter: ON HP Fc 85 Hz
Channel: LFE
  Filter: ON LP Fc 160 Hz   
  Filter: ON LP Fc 160 Hz
  Filter: ON LP Fc 160 Hz