Menu ▾ ▴

virtual_surround_with_headtracking_tutorial

Sergejs
Attachments

Fully-featured individual 7.1 surround with head-tracking on a PC

Probably the most accurate and advanced sound over headphones in the world.

Tutorial how to achieve system-wide binauralized 7.1 sound over headphones with optional headtracking that uses any SOFA HRTF and optional individual headphone calibration.

Tutorial Status Date Written Date Last Updated
Ready. 2022-08-05 2023-01-07

Example how fully configured and running virtual surround setup looks after this tutorial:

Video version of this tutorial:

Help - You are welcome to contact author about any extra information, corrections or other input. New tools and approaches appear literally every month, so this page can be updated as needed.

Introduction

Currently the most advanced system-wide solution for accurate sound over headphones on a computer can provide:

  • Individual HRTF in SOFA format that precisely describes unique properties of listener's ear and head shape for sounds coming from any direction. (Note that majority of affordable commercial alternatives make rough HRTF approximations which are very far from true individual HRTFs).
  • Headtracking to enhance immersion and sound localization through natural head movements relative to the stationary virtual speakers, instead of a typical case where "the world" rotates along with the headphones.
  • 7.1 channel surround sound format. This is the best format openly supported by Windows PCs (as of 2022). Note that Dolby Atmos and other channel-free sound rendering does exist on computers, but that requires direct integration with specific software. On system-wide level Windows 11 currently does not output sound to more than 7 speakers and ".1" mono subwoofer, so 7.1 is what we can use.
  • Individual headphone calibration - the same headphones can sound very different on different people and without individual equalization the results are quite unpredictable. Thankfully individual headphone calibration is quite easy to perform and easy to apply.

This tutorial demonstrates how to achieve this state-of-the-art virtual surround so that it applies to the primary Windows sound playback device and therefore is effortlessly usable for all media, internet, games and movies on a PC (in other words - works "system-wide").

  • NOTE: This method produces very similar results (sound) as HeSuVi with Advanced SOFA to HeSuVi conversion with the main difference being - it supports head-tracking. This method will also can be a bit more flexible for real-time adjustments in VST plugins. Main difference - this approach does not apply Impulcifer (AutoEq) filtering to the HRTF which smooths out some sharp peaks and dips.

  • NOTE that parts of this tutorial would help to do the same setup on Apple Mac or Linux, but I personally do not know the specifics of those platforms.

Setup overview

In this tutorial we will install and configure a setup where:

  1. A virtual soundcard program accepts 7.1 sound from any software running on a PC and routes the sound for further processing (exception: VR applications, which usually have their own sound pipeline)
  2. A VST plugin host program receives sound from the virtual soundcard and provides the environment to run all the necessary VST plugins.
  3. A binauralizer VST plugin applies the SOFA HRTF data to accurately virtualize the sound for 7.1 speakers into the left and right headphone signal.
  4. Optional EQ VST plugin that could correct issues with the headphone tonality (because all headphones are somewhat different and also sound different on different people).
  5. Optional Headtracker that can enable head rotation relative to the sound before binauralization.
  6. Optional Reverb or other effect VST plugins to further shape the virtual listening environment for a given purpose or preference.

Tutorial step-by-step

Note, this tutorial describes an example, working audio setup that meets state-of-the-art criteria. Experienced users are welcome to experiment and exchange some of the proposed software with other equivalents in order to reduce latency, try other HRTF algorithms or for any other reason.

Install the software

  1. Download and install Voicemeeter Banana (VoicemeeterProSetup.exe v2.0.6.2 or newer) from the official homepage.

    • More advanced Voicemeeter Potato version works as well, but Banana already has everything needed for HRTF use. Just do NOT use the basic Voicemeeter (non-pro) - basic Voicemeeter does not have "Insert Virtual ASIO" feature.
    • Note that Voicemeeter Banana is Donationware - "For END USER, Voicemeeter is free to use! You pay what you want when you want if you find it useful. Thanks for your participation!" "For any professional use, you may pay the recommended license price on the webshop."
    • There is a good official manual for Voicemeeter as well as many YouTube instruction videos (because this software is very popular among podcasters and streamers who need to mix sound from various sources and apply realtime sound effects).

  2. Download and install SPARTA plugins - all open-source, cross-platform SPARTA VST plugins come as one package with an installer.

  3. Download and unzip VSTHost (vsthostx64.zip). - - verified! - works great with SPARTA plugins. Not fully open source, but absolutely great software for Windows. Use vsthostx64.zip version, but other versions likely work as well.

    • VSTHost is free, but Not fully open source (even though there is an open source version of it).
    • As there is no installer, it is up to user to place the files into a good permanent location, for example C:\Program Files\vsthostx64

  4. Optional Headtracking - if headtracking will be used, it needs to be compatible with SPARTA Binauraliser.

  5. Optional additional VST plugins. Voicemeeter Banana already provides some basic tone controls and even 15-band equalizer, but any reverb or other effects may require additional VST plugins. Thankfully most VST plugins will work inside VSTHost alongside the necessary SPARTA plugins.

Configure Virtual soundcard

Voicemeeter Banana (VB) is the program of choice for this HRTF setup and also is great for use with HeSuVi/Equalizer-APO. It has several very useful extra features that are missing in other alternative virtual sound cards. There may be some latency or resource usage questions remaining, but for the most part the VB is almost flawless.

  1. When you first open VB, it will ask to set A1 audio output device. Use the "A1" dropdown to choose the sound card that is connected to the main headphones. In this example headphones are connected to Apple USB-C to 3.5mm dongle where the best available option is WDM connection.

    • The same audio hardware may be accessible by multiple APIs and there is a clear order of preference in order to minimize latency: ASIO >> WDM >> WASAPI exclusive >> KS >> WASAPI >> Direct Sound (DS) >> MME. Note that ASIO is the best option only if it is used with native ASIO driver (which usually must be installed by user). Official ASIO drivers are available for most professional audio interfaces, but are not common for consumer and gaming audio devices (ASIO2All, FlexASIO and other virtual ASIO drivers can only make latency worse).

  2. On the main VB screen we need to set some more options:

    • First, you can ignore crossed out controls because they are not needed for our use-case. Those hardware inputs, AUX virtual input and various extra outputs can be used for other purposes and will not be affected by this HRTF setup.
    • Make sure that "Voicemeeter VAIO" is routed to A1 output ("A1" button is green). All other "Voicemeeter VAIO" routing can be disabled to save resources.
    • You can "Mute" all unused outputs (not mandatory) in the "Master section". Only the "A1" output remains unmuted.
    • The rest of the buttons can be left as-is, or as seen in the illustration.

  3. Virtual Inputs have their own buffering which (unfortunately) limits the minimum possible latency of this setup:

    • Right click on the number right under "Voicemeeter VAIO" text.
    • Pick the smallest value that works (this step may need to be done as optimization step after everything is already working).
    • "2048" samples is likely to work. (note that this latency is still quite high for action games and timing sensitive audio).
    • Continue further for more buffering/latency related settings*

  4. In the "Menu" there are more useful options:

    • "Run on Windows Startup" option can be useful
    • Set up volume control shortcuts by picking the best option from "Shortcut Key (Hook)". We want to control "Level Output A1" by either standard "Volume Up/Down" media buttons or there is an option to use Ctrl+F10 / F11 / F12 .
    • "Limit Remote Gain to 0dBfs" is a good idea to not exceed 100% volume when it is adjusted by shortcuts.
    • Finally click on "System Settings / Options"...

  5. In VB "System Settings / Options":

    1. Choose sample rate and be Consistent! - 44100 or 48000 Hz.
      • 44.1kHz and 48kHz is basically the same thing. Lower sampling rate will use less computing resources and it is impractical to run all audio processing at higher than 48kHz (so 96kHz and more is not recommended, but in principle possible).
      • In this example I use 44.1kHz because most of the music CD audio is natively in 44.1kHz. At the same time DVD and movie audio is usually at 48kHz, so you can choose which audio will be resampled and which will pass natively.
      • Make sure your chosen sample rate is the same in all places to avoid unnecessary internal resampling.
    2. Buffering - use multiples of 128! and as low as it works. Buffering determines the sound latency which is somewhat important for watching videos and editing sound, but latency is crucial for headtracking and action games.
      • Note that SPARTA Binauraliser does not support buffers lower than 128 and not-multiples of 128. With other VST plugins or other audio setups this limitation may not apply.
      • You should only care about Buffer values for the API that is in use. In this example it is WDM,
      • Try 128 samples on ASIO - it is likely to work.
      • For the WDM and other APIs 512 sample buffer is not bad, but after initial testing try lowering buffer to 256 and even 128 samples. Too-low buffer can be immediately detected by noticeable crackling noises in the headphones - it means that some audio data gets lost and the buffer needs to be either restarted or increased. Note that as you will change buffer values in VB, it may be necessary to re-start the VST hosting software to properly apply changes. It is possible that good, working buffer values start produce crackling noises not because the buffer is too small, but because of the repeated change of buffer settings.
    3. "Patch Insert" must be enabled. These 8 lit-up squarish icons on the bottom mean that VB will send all incoming sound over "Insert virtual ASIO" for external processing which is how VST plugin host gets connected.
      • Try to only enable these 8 channels! Any additional channels will show up in the same "Insert virtual ASIO" device. If any channels ahead of these 8 are activated, it will be necessary to correctly specify which channels of the "Insert virtual ASIO" are the eight 7.1 sound channels.

  6. In addition to settings within VB, it is important to configure both virtual and actual audio devices in Windows Sound Control Panel:

    • All these settings can be changed from the legacy "Sound" options in the Windows Control Panel, but some of these settings can also be changed in other ways.
    • First Set Default audio device to VoiceMeeter Input - this is the virtual output that all other software should use.
    • Sample rate must be consistent with settings within VB.
      • The 44.1 or 48kHz must match on both your actual hardware soundcard and "VoiceMeeter Input".
    • Try to select at least 24bit sound (16bit can cause audible issues in some scenarios).
    • Make sure that headphone Level = Windows volume is 100%. With VB, system volume should be adjusted inside VB on the "A1 Fader Gain". Note that ASIO drivers always have volume at 100%, but with other APIs you may notice that the volume is too low in case VB is using output device which has its volume turned down in Windows.
    • "Spatial sound must be OFF" for headphones. As this setup will produce its own spatial sound, you do not want any additional processing applied by Windows or any other program.
    • "Enable audio enhancements" - ??? This option is good to turn off if you are sure that it will not be needed. This option is necessary to use Equalizer APO on this device, which we will probably not need, but it is good to know what are you disabling.
    • Finally configure "VoiceMeeter Input" for 7.1 Surround. This is a key setting to enable 7.1 surround over headphones. Note that using VB this option is free - it has no impact if you listen to 2ch Stereo, but adds a lot better surround sound if you listen to movies, games or other sources that can output sound to 7.1 speaker system.

This should conclude Voicemeeter Banana setup. At this stage the sound in the headphones should be completely normal and it should still be easy to adjust volume or use computer as always.

Some extra features to try in VB:

  1. You can use "Bass", "Mid" and "Treble" controls to make basic tonality adjustments on your headphones without any additional software.
  2. There is a more advanced built in Equalizer that can be opened as a standalone program from Start menu or by pressing Menu ==> Run other Voicemeeter tools ==> Run 15 Bands Graphic EQ.

  3. Even better equalizer - EQPro is available by right-clicking the EQ button above the "A1" volume fader. This is 5-band fully parametric equalizer for each of Master outputs. (Just don't forget to left-click on EQ button to actually enable this equalizer).

  4. Under Menu ==> M.I.D.I. Mapping there is a long list of commands that can be mapped to a MIDI controller. In practice you can control the tone controls and volume from any compatible device + use other VB features.

Configure VST plugin host

VSTHost (vsthostx64.zip) by Hermann Seib is a very powerful yet lightweight program that can conveniently run any sequence of VST plugins on audio data that comes in and out via normal Windows audio devices (real or virtual). This specific program provides all the necessary options, graphical user feedback, level adjustments to prevent clipping and other useful features in this mostly free program.

  1. After you extract "vsthostx64.zip" to a permanent location, the most interesting files are:

    • Detailed manual: "VSTHost.pdf" which can explain more tricky features. And
    • "vsthost.exe" - the actual program (for which you should create a shortcut or pin it to the taskbar).

  2. When you open vsthost.exe it is good to set up a preset which are called "Performance":

    1. Check all tick-boxes - specifically "Reload" and "Autosave PlugIn Banks". The other 3 options are enabled by default.
    2. Go to "Save As..." to open Performance editing menu:
      • Pick the first item in the list
      • Name for your preset/Performance.
      • Press OK.

  3. Set up audio device by going to Devices ==> Wave...:

    • Input device does not matter when ASIO is used (note that only one ASIO driver can be used at a time, that applies to both input and output).
    • Output should be set to "ASIO: Voicemeeter Insert Virtual ASIO".
    • Sample rate must be Consistent with VB and sound card settings.
    • Buffer options will likely be limited by the buffer setting selected in VB. Remember that you may get crackling sound even with safe buffer values (like 512) if the settings are recently changed. Try re-starting the computer to be 100% certain that buffer value is too small for your setup.

  4. Check/fix ASIO channel mapping - to process the correct ASIO audio channels in VSTHost.

    • Go to Devices ==> Asio Channel Selection... and select which ASIO channels will be processed by VSTHost (with VoiceMeeter - only "IN#4").
    • four output it is enough to only select 2 channels, because Binuaraliser should never output more than 2 headphone channels.

  5. Load VST plugins by going to File ==> New PlugIn...:

    • First one in chain should be sparta_binauraliser.dll which will apply your SOFA HRTF to the 7.1 speaker signal.
    • After Binauraliser connects sparta_multiconv.dll which will apply global headphone correction.

  6. Fix the connections by double-clicking the "Chain After" buttons in the corners the 2 last boxes and changing tick-boxes:

    • We need a signal flow from Input >to> Binauraliser >to> MultiConv >to> Output.
    • If you want to use Reverb plugins, insert them too, but in a correct sequence:
      • Ideally reverb should be applied on 7.1 channels before binauralising. This is crucial if headtracking is used.
      • For some algorithms reverb can be applied after binauralising, but then reverb will rotate together with headtracking.
      • There actually is a simple built-in reverb plugin in VSTHost: "File ==> PlugIns ==> BuiltIn ==> Effect ==> Reverb". It is based on FreeVerb and more info available inside "VSTHost.pdf" manual.

  7. There are some extra features for every plugin box - small buttons to:

    • Bypass any plugin to hear the difference.
    • Plugin Edit button (looks like a dial/clock) to open VST plugin interface.
    • Plus a few more, including useful "Info" button .

  8. Remember that VSTHost is "donationware" - if you find it useful, please contribute to the community and specifically send over a reasonable amount to Hermann Seib for all his efforts. Most of us can't imagine how to make something like VSTHost, but he just gives this awesome tool for you and your business whenever you need it. Go to Help ==> Donate to see the PayPal details.

  9. Close and re-open VSTHost! This will save your current configuration.

At this point VSTHost is fully configured. It is actually perfectly safe to open and close VSTHost at any time while the audio is playing. The sound will seamlessly continue playing through Voicemeeter with or without VSTHost processing.

Configure VST plugins

SPARTA Binauraliser

SPARTA Binauraliser is the main component of the signal chain that actually processes each of the 7.1 speaker signals (or any number - from 1 to 64 speakers) with the HRTF data to output 2 signals for the headphones. As in the screenshot above, all settings are done in the VST plugin interface:

  1. Import a 7.1 speaker preset from this tutorial by pressing on a small "Import" button next to "Inputs". Note, SPARTA Binauraliser does not have any suitable built-in presets for 7.1 surround.

    • Windows has some compatibility issues caused by Confusion of 7.1 and 5.1 channel order in Windows. This is not a big issue, but there are preset 2 files because of this:
      1. SPARTA 7.1 config for 7.1 and Windows standard.json - This is the primary preset to use on Windows - it is necessary to get correct sound with 7.1 content. This config follows WaveFormatExtensible ordering for 7.1: Left, Right, Center, LFE, BackLeft, BackRight, SideLeft, SideRight.
      2. SPARTA 7.1 config for 5.1 and Dolby standard.json - This is a file that can be useful for solving an issue that side-surrounds are mapped to rear-surrounds when playing 5.1 content in most software. This config follows Dolby ordering for 7.1: Left, Right, Center, LFE, SideLeft, SideRight, BackLeft, BackRight. (This ordering is naturally backwards compatible with 5.1)
      3. Note, you are free to make your own adjustments to each speaker position and saving your own presets using "Export" button. For example it may sound nice to raise the Elevation angle of some channels.
    • When the preset is loaded, you should see that the Binauraliser is now configured to process 8 inputs and the coordinates of each input in space.

  2. Under HRIR, load your SOFA HRIR file for the correct sample rate. (remember that HRIR is the same as HRTF only calculated in the time domain).

    • "Apply Diffuse-Field EQ" - ??? Depending on your plans for headphone equalization this should be checked or un-checked.
      • NO - in case you will use some sort of Individual headphone calibration and have no need for generic Diffuse-Field EQ.
      • YES - in case you need a reasonable tonality without going through the hassle of individual headphone calibration.

  3. "Enable Rotation" is the option that turn on/off adjustment of relative head rotation compared to the virtual speaker position. If you use compatible headtracker like this one, then the angle values will already update in real-time no matter the setting. But even without headtracker, this can be used for debugging, for example:
    * Tip - check the surround sound mapping - enable rotation, then set the Yaw to 90 degrees and then click on the speakers in the Windows 7.1 speaker setup (see screenshot example) to get the sound out of them. As the Yaw=90deg, one of the side-surrounds will be straight ahead, and the rest of the speakers will be easy to localize.

  4. When you are done, check that there are no red messages in the plugin. For example, plugin can warn about invalid buffer size or sample rate mismatch.

SPARTA MultiConv

SPARTA MultiConv is used for global headphone equalization in this setup.

  • Leave this plugin Bypassed IF you did-not perform some sort of Individual headphone calibration - it is only useful if you have a .wav file from AutoEq that you would like to use.

  • If you do have a .wav correction file, then just load the file and make sure that the number of inputs= 2

Headtracking

First make sure you know all Headtracker options!

The only tested headtracker for SPARTA Binauraliser so far is Supperware Headtracker (many other headtrackers have been used by others, but I have no experience with them). Supperware Headtracker software has an actual "SPARTA" profile which "just-works" - when headtracker is active, SPARTA Binauraliser starts receiving head position data and no further configuration is needed - just "Enable rotation" in the plugin and go.

  • Note, if there are multiple SPARTA VST plugins active on the computer - only one of them will receive headtracker data, so be careful that you do not have some program running SPARTA VST plugin in the background that stops all other SPARTA instances from getting headtracker signal.
---------------------------------------------- Part of the "HRTF Application notes" ----------------------------------------------
Back to wiki home >>> Highly recommended: Everyday use of SOFA HRTF <<< Good to know: General HRTF usage advice

Related

Wiki: Advanced_SOFA_to_HeSuVi_conversion
Wiki: Application_notes_1_General_HRTF_usage_advice
Wiki: Confusion_of_7-1 and 5-1_channel_order_in_Windows
Wiki: Earful_tutorial
Wiki: Everyday_use_of_SOFA_HRTF
Wiki: Home
Wiki: tips_to_configure_HeSuVi