Menu
▾
▴
linuxsampler-devel
|
From: joo b. <joo...@ya...> - 2020-03-31 16:16:51
|
Dear All, So far, it seems the SFZ engine is not supporting any Opcode for polyphony, note polyphony, or self masking. This has been coming up in the mailing list every now and then, but yet the Opcodes are unsupported. The main problem with lacking note_polyphony is that playing a 3-voice chord repeatedly with a sustain pedal down is enough to bump into severe XRuns, even on my studio computer and RME soundcard. There is no way to avoid the XRuns for such a playing if the note_polyphony is not implemented... I wonder if anyone has written a NKSP program to tackle note polyphony? Or if there is a plan to implement it? I know that some might suggest using "off_by" Opcode, but that is really not a working solution when my Piano sample has 30 layers of velocity and I cannot meaningfully turn some groups off... Cheers (and be safe!) Ebad |
|
From: Christian S. <sch...@li...> - 2020-04-01 09:29:25
|
On Dienstag, 31. März 2020 18:16:38 CEST joo bian wrote: > The main problem with lacking note_polyphony is that playing a 3-voice chord > repeatedly with a sustain pedal down is enough to bump into severe XRuns, > even on my studio computer and RME soundcard. There is no way to avoid the > XRuns for such a playing if the note_polyphony is not implemented... Which makes me wonder whether you are mixing here sound design aspects with what looks like a fundamental issue with your installation there. I mean it is not normal that you encounter any xruns by just playing a piano with LS, even not with a 20 year old hardware and $5 built-in sound chip. As soon as you hit the current polyphony limit setting the sampler's voice stealing kicks in to prevent any xruns. Are you overriding the default setting for max. polyphony and max. disk streams? If yes, what values have you picked? Did you compile LS with optimization turned on (i.e. with compiler flag -O3)? > I wonder if anyone has written a NKSP program to tackle note polyphony? Well you could also release notes by NKSP script of course, sure, with any customized logic you might think of. Such a script would just need like 6 lines of NKSP code, so it should not be a big deal for you to try out. However as mentioned, it looks like you rather have a problem with your LS installation there. > Or if there is a plan to implement it? At least not on my side. Usually new feature appear only when people decide to implement them by themselves and providing a patch for their changes. CU Christian |
|
From: joo b. <joo...@ya...> - 2020-04-03 18:23:13
|
Thank you Christian,
You pointed out several things. I will focus in this reply on disc stream and polyphony in the LinuxSampler setting and will touch on the other, when I am sure I am getting the settings right.
So I use Arch linux because I can install LinuxSampler (stable and svn releases) from Pamac. I recently switched to SVN development releases and I am currently using the SVN version (r3751-1) from <AUR (en) - linuxsampler-svn>. That said, I did not compile LinuxSampler with any additional argument. Everything should have been the default installation. I also note that I only use the SFZ engine.
Based on what you wrote, it seems I should not really mention my Hardware (for reference: I am using Linux Manjaro 18 with a 3500mb/s M2.0 ssd, 16GB Ram, RME soundcard, and 2.2 Ghz quadcore CPU.
I get xruns when playing intricate piano solos with sustain pedal down when I reach the polyphony limit (I believe). I never get x-runs with simpler piano solos. The SFZ pianos I am developing for SFZ have multiple sounds per key so I reach the limit much faster than basic piano sound libraries.
X-Runs
======
I have tried to figure out why these x-runs happen. Interestingly, the disk streams do not seem to have any role on my x-runs. I get the X-runs "frequently" with the default settings of "64" polyphony and "90" disk streams. The only way I could completely avoid x-runs is to reduce the polyphony to 48 voices. Anything above this number would cause me x-runs. I get the x-runs with higher disk streams (as high as 9000 and as low as 64 streams). If I understand it correctly, disk streams load more data on the Ram? I see that because when I increase the disk stream LinuxSampler begins to take much more Ram in the system. But I am not sure what is actually happening. In anyway, unfortunately, 48 voice polyphony is relatively low for a complex instrument where each key has several layers of voices.
I am not quite sure what is going on. I have a real-time kernel, JACK2 runs without a problem in real-time, and I even tried running LinuxSampler with real-time priority but basically nothing could help with x-runs but reducing polyphony.
Any idea?
Cheers,
Ebad
On Wednesday, April 1, 2020, 11:29:44 AM GMT+2, Christian Schoenebeck <sch...@li...> wrote:
On Dienstag, 31. März 2020 18:16:38 CEST joo bian wrote:
> The main problem with lacking note_polyphony is that playing a 3-voice chord
> repeatedly with a sustain pedal down is enough to bump into severe XRuns,
> even on my studio computer and RME soundcard. There is no way to avoid the
> XRuns for such a playing if the note_polyphony is not implemented...
Which makes me wonder whether you are mixing here sound design aspects with
what looks like a fundamental issue with your installation there. I mean it is
not normal that you encounter any xruns by just playing a piano with LS, even
not with a 20 year old hardware and $5 built-in sound chip. As soon as you hit
the current polyphony limit setting the sampler's voice stealing kicks in to
prevent any xruns.
Are you overriding the default setting for max. polyphony and max. disk
streams? If yes, what values have you picked?
Did you compile LS with optimization turned on (i.e. with compiler flag -O3)?
> I wonder if anyone has written a NKSP program to tackle note polyphony?
Well you could also release notes by NKSP script of course, sure, with any
customized logic you might think of. Such a script would just need like 6
lines of NKSP code, so it should not be a big deal for you to try out.
However as mentioned, it looks like you rather have a problem with your LS
installation there.
> Or if there is a plan to implement it?
At least not on my side. Usually new feature appear only when people decide to
implement them by themselves and providing a patch for their changes.
CU
Christian
_______________________________________________
Linuxsampler-devel mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxsampler-devel |
|
From: Christian S. <sch...@li...> - 2020-04-04 10:52:02
|
On Freitag, 3. April 2020 20:23:00 CEST joo bian wrote: > Thank you Christian, > You pointed out several things. I will focus in this reply on disc stream > and polyphony in the LinuxSampler setting and will touch on the other, when > I am sure I am getting the settings right. > > So I use Arch linux because I can install LinuxSampler (stable and svn > releases) from Pamac. I recently switched to SVN development releases and I > am currently using the SVN version (r3751-1) from <AUR (en) - > linuxsampler-svn>. That said, I did not compile LinuxSampler with any > additional argument. Everything should have been the default installation. You can certainly check whether you see -O3 in the output while the sampler is compiling. The sampler must be compiled with optimization turned on, which only happens if the compiler flags (which you see on the console) include "-O3". > I also note that I only use the SFZ engine. > > Based on what you wrote, it seems I should not really mention my Hardware > (for reference: I am using Linux Manjaro 18 with a 3500mb/s M2.0 ssd, 16GB > Ram, RME soundcard, and 2.2 Ghz quadcore CPU. > > I get xruns when playing intricate piano solos with sustain pedal down when > I reach the polyphony limit (I believe). I never get x-runs with simpler > piano solos. The SFZ pianos I am developing for SFZ have multiple sounds > per key so I reach the limit much faster than basic piano sound libraries. It would make sense to start with a simple setup first, i.e. start by using ALSA as audio output instead of JACK. Another important thing is the actual latency setting (i.e. buffer size setting). Start with higher latencies first (e.g. 50ms), and if you don't get dropouts, lower the latency setting step by step. If you are running with an extreme low latency setting of e.g. <1ms then it is not as simple as buying the most powerful and most expensive hardware on the market, you also have to tweak the system to actually being able to deliver within that small time frame (at least on Linux). Keep in mind Linux' default environment were servers, and many default settings are hence not suitable for low latency / a.k.a. realtime applications by default. > > X-Runs > ====== > I have tried to figure out why these x-runs happen. Interestingly, the disk > streams do not seem to have any role on my x-runs. I get the X-runs > "frequently" with the default settings of "64" polyphony and "90" disk > streams. As a rule of thumb: the most important setting here is the max. voices, which you have to balance accordingly with your audio driver's buffer size (latency setting). For the amount of max. streams setting you would usually simply always choose a value approximately 30% higher than your max. voices setting. > The only way I could completely avoid x-runs is to reduce the > polyphony to 48 voices. Anything above this number would cause me x-runs. I > get the x-runs with higher disk streams (as high as 9000 and as low as 64 > streams). If I understand it correctly, disk streams load more data on the > Ram? I see that because when I increase the disk stream LinuxSampler begins > to take much more Ram in the system. But I am not sure what is actually > happening. Disk streaming is used to read as much sample data as possible in real-time from the underlying HD/SSD on demand. Without disk streaming, entire sample data would need to be preloaded into RAM. So without disk streaming, if you'd load a sample library being 20 GB in size, you would accordingly need >20 GB RAM just for loading the sound. With disk streaming enable, only a small portion of the sample data needs to be preloaded into RAM, the rest is directly read from disk whenever required. By increasing the max. streams setting, RAM usage will increase, yes, because each disk stream uses a buffer (in RAM) which is used for reading sample data from disk, while the voice is reading from the other end of that buffer. > In anyway, unfortunately, 48 voice polyphony is relatively low > for a complex instrument where each key has several layers of voices. > > I am not quite sure what is going on. I have a real-time kernel, JACK2 runs > without a problem in real-time, and I even tried running LinuxSampler with > real-time priority but basically nothing could help with x-runs but > reducing polyphony. Well, depends on what you mean with "JACK2 runs without a problem". Obviously you get xruns when system load increases. So the question is whether your JACK use cases so far ever involved high system loads. CU Christian |
|
From: joo b. <joo...@ya...> - 2020-04-06 15:24:22
|
Thank you Christian for the detailed reply, I kept changing the JACK and LS buffer, polyphony, etc, but I am still getting the X-Runs. So at this point, I think it's time to go back to your previous comment whether LinuxSampler was compiled with optimization flag turned on. I had never came across the optimization requirement until you mentioned it. - I checked the building files on AUR for installing LinuxSampler within Pamac. They only apply './configure && make' as the README says. Therefore, if the optimization flag is not turned on by default (which I think it is not), none of them is built with optimization. Optimizing LinuxSampler installation------------------------------------------------ - I googled about optimizing LinuxSampler and I landed on a document for compiling for Debian http://www.linuxsampler.org/debian.html#build_backend. I edited the "./debian/rules" file and added the arguments relevant to my machine: "CXXFLAGS="-O3 -march=x86-64 -mtune=native -msse -march=x86-64 -mfpmath=sse -ffast-math -fomit-frame-pointer -funroll-loops" ./configure " ./configure --prefix=/usr --mandir=\$${prefix}/share/man --infodir=\$${prefix}/share/info" But from now on, I couldn't figure out what happens on my Arch Linux (Manjaro 18). I saved the file but after compiling, it seems it is compiling it with "-o2" optimization. I can't figure it out how to make the compilation take these changes into account. I also noticed that in the "./debian/rules" file, there are two occasions where the CXXFLAGS are mentioned, one under "configure-stamp" and the other under "clean:" sections. The file does not give a hint how to progress here. Could you please help me get this setting to run for compiling LS? Disk streams usage---------------------------------- As a general question, if I have 32GB of DDR4 2133 RAM which is much faster than my M2.0 SSD. I wish to dedicate all of this RAM (as long as my computer is functioning) to LinuxSampler. Is this something recommended by you? I am preparing my instrument for playing live on stage and this is why I am trying to push the limits as much as possible. Would it be alright to load the entire sample library in RAM, if I have enough RAM to do so? And would that help me to optimize the performance for very rapid melodies with sustain pedal on? My understanding was that it is preferable to make use of the RAM as much as possible, which is why I invested in hardware with capacity of 32GB of RAM... Thank you again for your time | | | | LinuxSampler for Debian | | | On Saturday, April 4, 2020, 12:52:18 PM GMT+2, Christian Schoenebeck <sch...@li...> wrote: On Freitag, 3. April 2020 20:23:00 CEST joo bian wrote: > Thank you Christian, > You pointed out several things. I will focus in this reply on disc stream > and polyphony in the LinuxSampler setting and will touch on the other, when > I am sure I am getting the settings right. > > So I use Arch linux because I can install LinuxSampler (stable and svn > releases) from Pamac. I recently switched to SVN development releases and I > am currently using the SVN version (r3751-1) from <AUR (en) - > linuxsampler-svn>. That said, I did not compile LinuxSampler with any > additional argument. Everything should have been the default installation. You can certainly check whether you see -O3 in the output while the sampler is compiling. The sampler must be compiled with optimization turned on, which only happens if the compiler flags (which you see on the console) include "-O3". > I also note that I only use the SFZ engine. > > Based on what you wrote, it seems I should not really mention my Hardware > (for reference: I am using Linux Manjaro 18 with a 3500mb/s M2.0 ssd, 16GB > Ram, RME soundcard, and 2.2 Ghz quadcore CPU. > > I get xruns when playing intricate piano solos with sustain pedal down when > I reach the polyphony limit (I believe). I never get x-runs with simpler > piano solos. The SFZ pianos I am developing for SFZ have multiple sounds > per key so I reach the limit much faster than basic piano sound libraries. It would make sense to start with a simple setup first, i.e. start by using ALSA as audio output instead of JACK. Another important thing is the actual latency setting (i.e. buffer size setting). Start with higher latencies first (e.g. 50ms), and if you don't get dropouts, lower the latency setting step by step. If you are running with an extreme low latency setting of e.g. <1ms then it is not as simple as buying the most powerful and most expensive hardware on the market, you also have to tweak the system to actually being able to deliver within that small time frame (at least on Linux). Keep in mind Linux' default environment were servers, and many default settings are hence not suitable for low latency / a.k.a. realtime applications by default. > > X-Runs > ====== > I have tried to figure out why these x-runs happen. Interestingly, the disk > streams do not seem to have any role on my x-runs. I get the X-runs > "frequently" with the default settings of "64" polyphony and "90" disk > streams. As a rule of thumb: the most important setting here is the max. voices, which you have to balance accordingly with your audio driver's buffer size (latency setting). For the amount of max. streams setting you would usually simply always choose a value approximately 30% higher than your max. voices setting. > The only way I could completely avoid x-runs is to reduce the > polyphony to 48 voices. Anything above this number would cause me x-runs. I > get the x-runs with higher disk streams (as high as 9000 and as low as 64 > streams). If I understand it correctly, disk streams load more data on the > Ram? I see that because when I increase the disk stream LinuxSampler begins > to take much more Ram in the system. But I am not sure what is actually > happening. Disk streaming is used to read as much sample data as possible in real-time from the underlying HD/SSD on demand. Without disk streaming, entire sample data would need to be preloaded into RAM. So without disk streaming, if you'd load a sample library being 20 GB in size, you would accordingly need >20 GB RAM just for loading the sound. With disk streaming enable, only a small portion of the sample data needs to be preloaded into RAM, the rest is directly read from disk whenever required. By increasing the max. streams setting, RAM usage will increase, yes, because each disk stream uses a buffer (in RAM) which is used for reading sample data from disk, while the voice is reading from the other end of that buffer. > In anyway, unfortunately, 48 voice polyphony is relatively low > for a complex instrument where each key has several layers of voices. > > I am not quite sure what is going on. I have a real-time kernel, JACK2 runs > without a problem in real-time, and I even tried running LinuxSampler with > real-time priority but basically nothing could help with x-runs but > reducing polyphony. Well, depends on what you mean with "JACK2 runs without a problem". Obviously you get xruns when system load increases. So the question is whether your JACK use cases so far ever involved high system loads. CU Christian _______________________________________________ Linuxsampler-devel mailing list Lin...@li... https://lists.sourceforge.net/lists/listinfo/linuxsampler-devel |
|
From: Christian S. <sch...@li...> - 2020-04-07 13:11:10
|
On Montag, 6. April 2020 17:24:08 CEST joo bian wrote: > Thank you Christian for the detailed reply, > I kept changing the JACK and LS buffer, polyphony, etc, but I am still > getting the X-Runs. So at this point, I think it's time to go back to your Not JACK, use ALSA! Like I mentioned before, start with a simple setup by using ALSA. If it works for you (without dropouts) with ALSA, go ahead with JACK. If you still get dropouts with ALSA, name the concrete parameters that you were using for a) sample rate, b) buffer size, c) max. voices, d) max. streams and e) precise sound device being used. What file format are your samples in (wav, ogg vorbis, flac, ...)? Have you tried a .gig file instead for comparison? What about trying to install an OS that is already preconfigured for low latency like Ubuntu Studio for instance? > But from now on, I couldn't figure out what happens on my Arch Linux > (Manjaro 18). I saved the file but after compiling, it seems it is > compiling it with "-o2" optimization. That's OK. It is probably not the root of your problem there. -O in general means optimization and the number after it means the level of optimization where 3 is usually the maximum optimization level. CU Christian |
|
From: joo b. <joo...@ya...> - 2020-04-07 14:37:03
|
Dear Christian, Jacek,
Thanks for your kind replies. As Christian pointed out, I also don't think that a little bit of optimization plays a significant role, giving my hardware (and the fact that I get the dropouts even while my CPU usage is less than 25% and memory usage is less than 10%). But I managed to compile it with -o3 optimization. Since the optimization arguments are mainly passed to the compiler, I could also give them directly to the compiler. So for installing the SVN version of the LinuxSampler on Arch Linux, I applied the following commands after navigating to the source directory:
```make -f Makefile.cvs./configure --disable-sf2-engine --disable-instruments-db
make CXXFLAGS='-march=x86-64 -mtune=generic -O3 -msse -ffast-math -funroll-loops -fomit-frame-pointer -fpermissive -mfpmath=sse'
```
Then in the compilation, these arguments also appear (for 64bit computers, both "-march=x86-64 -mtune=generic " are the default. also the "-ffast-math" was in the default settings and "-msse" is also by default is turned on for 64bit. But I kept them anyway).
The problem is that the issue was unaffected. I also started to doubt my RME AIO soundcard which I know is partially supported for Linux but based on my research it should work fine.
Then I tried the same MIDI piano files that I am using for testing the instrument and the setup on Carla and Liquidsfz and surprisingly, I did not get any of the X-Runs for identical JACK settings, instrument, and MIDI files. So I will start doing a general torture tests on LS SFZ engine (and compare it to GIGA) to see if I can figure out what is going on.
But for now, I have one important question. Is there a way to load the entire sample library in RAM? In the source code of LS I saw that the ./configure command can take several arguments to change the behavior of LinuxSampler. But the help file was not very detailed and I was not sure how such a configuration would work. Would you be kind and suggest me such a configuration that takes advantage of a lot of RAM and very fast M2.0 SSD? I assume I need a hint regarding:
1. --enable-stream-min-refill=2. --enable-refill-streams=3. --enable-stream-size=4. --enable-max-streams=
5. --enable-max-events=6. --enable-subfragment-size=
7. --enable-preload-samples=
and possibly the following? I was not sure if I should enable this, because I am running real-time kernel.
8. --enable-rt-exceptions=
With the settings above, I assumed if I increase the size of the each stream or somehow preload the entire library, then I can be sure there won't be any other issue with streams and I can afford up to 30 GB of sample library to be loaded. The sample libraries I am currently using are about 15-20GB each, so I can afford loading the entire SFZ instrument into RAM.
Both Carla and Liquidsfz use much more RAM than LinuxSampler (but they don't load the entire instrument into RAM. they load about 70% of it, it seems). LinuxSampler, can work with any hardware it seems, which is incredible! But I am trying to get the maximum out of my hardware before carry out the torture tests.
Thank you again for the awesome community support.
Cheers,
Ebad
On Tuesday, April 7, 2020, 3:11:35 PM GMT+2, Christian Schoenebeck <sch...@li...> wrote:
On Montag, 6. April 2020 17:24:08 CEST joo bian wrote:
> Thank you Christian for the detailed reply,
> I kept changing the JACK and LS buffer, polyphony, etc, but I am still
> getting the X-Runs. So at this point, I think it's time to go back to your
Not JACK, use ALSA! Like I mentioned before, start with a simple setup by
using ALSA. If it works for you (without dropouts) with ALSA, go ahead with
JACK.
If you still get dropouts with ALSA, name the concrete parameters that you
were using for a) sample rate, b) buffer size, c) max. voices, d) max. streams
and e) precise sound device being used.
What file format are your samples in (wav, ogg vorbis, flac, ...)?
Have you tried a .gig file instead for comparison?
What about trying to install an OS that is already preconfigured for low
latency like Ubuntu Studio for instance?
> But from now on, I couldn't figure out what happens on my Arch Linux
> (Manjaro 18). I saved the file but after compiling, it seems it is
> compiling it with "-o2" optimization.
That's OK. It is probably not the root of your problem there. -O in general
means optimization and the number after it means the level of optimization
where 3 is usually the maximum optimization level.
CU
Christian
_______________________________________________
Linuxsampler-devel mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxsampler-devel
|
|
From: Christian S. <sch...@li...> - 2020-04-08 10:17:53
|
On Dienstag, 7. April 2020 16:36:48 CEST joo bian wrote: > Dear Christian, Jacek, > > Thanks for your kind replies. As Christian pointed out, I also don't think > that a little bit of optimization plays a significant role, giving my > hardware (and the fact that I get the dropouts even while my CPU usage is > less than 25% and memory usage is less than 10%). File format of samples being used (wav, flac, ogg vorbis)? CU Christian |
|
From: joo b. <joo...@ya...> - 2020-04-08 12:05:42
|
I thought I mentioned that somewhere in the thread. I only use uncompressed Wav files to avoid any unnecessary encoding. Most of my samples are 24bit 44.1khz or 24bit 48khz and they are all Wav.
Regarding Jacek's comment, I am recompiling LS with different configurations and it seems the behavior of --enable-preload-samples= argument is a little bit unpredictable (well, to me). The default value is 32768, which in practice (for 180 max voice) it loads 730MB out of a 8.5GB sample library.
I recompiled it with setting it 10 times larger, --enable-preload-samples=327680 and it loaded 2.6 GB. Recompiled it with 100 times larger and it loaded 5.3GB. Recompiled it with 1000 time larger value of 32768000 and the amount of preloaded samples did not go anything above 5.3GB
Side note------------
I saw some unusual behavior here as well. Surprisingly, when I increased the "max voices" LS can actually by far surpass the size of the library. For the 8.5GB sample library, LS could require 12GB if I increase the max voices. I only did this for testing (no other reasons to apply such a setting). it seems each stream can also occur independent of pre-loaded samples or at least its allocated RAM memory is independent of pre-loaded sample? I'm not sure. I thought the streams should take the preloaded data into account.
On Wednesday, April 8, 2020, 12:18:01 PM GMT+2, Christian Schoenebeck <sch...@li...> wrote:
On Dienstag, 7. April 2020 16:36:48 CEST joo bian wrote:
> Dear Christian, Jacek,
>
> Thanks for your kind replies. As Christian pointed out, I also don't think
> that a little bit of optimization plays a significant role, giving my
> hardware (and the fact that I get the dropouts even while my CPU usage is
> less than 25% and memory usage is less than 10%).
File format of samples being used (wav, flac, ogg vorbis)?
CU
Christian
_______________________________________________
Linuxsampler-devel mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxsampler-devel
|
|
From: Christian S. <sch...@li...> - 2020-04-09 12:40:19
|
On Mittwoch, 8. April 2020 14:03:38 CEST joo bian wrote: > I thought I mentioned that somewhere in the thread. I only use uncompressed > Wav files to avoid any unnecessary encoding. Most of my samples are 24bit > 44.1khz or 24bit 48khz and they are all Wav. That's exactly why I was asking you for this, because it makes a huge difference regarding max. polyphony whether you are playing e.g. uncompressed vs. compressed sample data, since the CPU overhead involved is multiplied by the amount of active voices. And no, you did not mention it before. > Regarding Jacek's comment, I am recompiling LS with different configurations > and it seems the behavior of --enable-preload-samples= argument is a little > bit unpredictable (well, to me). The default value is 32768, which in > practice (for 180 max voice) it loads 730MB out of a 8.5GB sample library. > > I recompiled it with setting it 10 times larger, > --enable-preload-samples=327680 and it loaded 2.6 GB. Recompiled it with > 100 times larger and it loaded 5.3GB. Recompiled it with 1000 time larger > value of 32768000 and the amount of preloaded samples did not go anything > above 5.3GB > > Side note------------ > > I saw some unusual behavior here as well. Surprisingly, when I increased the > "max voices" LS can actually by far surpass the size of the library. For > the 8.5GB sample library, LS could require 12GB if I increase the max You might have a look at method CacheInitialSamples() in source file src/engines/InstrumentManagerBase.h to understand what's going on. There is some overhead, yes, at the end of each cached sample, artificial silence is added. The precise length of that silence is dependent to CONFIG_MAX_PITCH (a compile time user setting), that is the maximum pitch setting ever allowed/desired/expected by the user (which is IIRC 3 octaves by default). If we would not add silence to the end of each (cached) sample then the sampler's audio core would have to be much more complex just to deal with the end of samples without crashing; which is not developer lazyness; the price would simply be higher CPU load which we can avoid this way. > voices. I only did this for testing (no other reasons to apply such a > setting). it seems each stream can also occur independent of pre-loaded > samples or at least its allocated RAM memory is independent of pre-loaded > sample? I'm not sure. I thought the streams should take the preloaded data > into account. I already explained you the details of this before. Each stream must use a FIFO buffer. Every streaming algorithm works this way. No matter if its an audio sampler or a Netflix video player. I can't stress you enough: start with ALSA (since it is directly dumping to the sound card) instead of JACK (since it requires inter-process communication and adds siginficant complexity for user options & potential mistakes). Only start to proceed with JACK once it is working for you with ALSA. I would also still recommend you to test with the gig engine. I am personally almost not using the sfz engine at all. So a substantial amount of my fixes, optimizations etc. is spent on the gig engine. All engines though share a substantial amount of code though. CU Christian |
|
From: joo b. <joo...@ya...> - 2020-04-09 13:05:53
|
Thank you for your explanation Christian.
I will give it a try with ALSA as you recommended. But I did a big test the next three days, which might be interesting to the development team here.
So I finished my torture tests on both LinuxSampler SFZ and GIGA engines. Luckily, I had a single microphone Piano library of about 5.5 GB with several velocity layers and key releases in both SFZ and GIGA samples to run this test (normally I only use SFZ). For this test, I put the polyphony on 128 and voices on 180 (double the default). I also executed the test with a default installation, o3 optimization, and o3 optimization with large instrument preloading to load the instrument in RAM. But the results were NOT really affected, meaning the voice streaming procedure is working just fine.
To torture LS, I created a MIDI file with 3 channels. each channel had a piano solo by Chopin (it was not nice to listen to the output of the torture test). I created one channel in LS and configured it to play all MIDI channels. This would guarantee some troubling piano playing, enough for this test.
I repeated my tests with different settings, 44.1KHz, all the way to 192KHz sample rates. I tuned it in a way that the GIGA sample would not give any X-Runs and then tested the SFZ for identical settings. The instruments were created with 24bit 44.1KHz Wav files.
Hardware-------------
AMD quad-core 2.1 Ghz CPU, tuned to run at its maximum clock speed of 2.1Ghz with turbo boost available. RME AIO sound cardJACK2 for enabling multicore audio processing3500mb/s M2.0 SSD DDR4 2133 RAM
Results======
GIGA Engine------------------
I had no problem at all with the GIGA instrument. No X-RUN, no unusual CPU activity, everything was perfect. JACK2 DSP load also was quite low during the performance, which is often associated with X-RUNS. Also my CPU did not have sudden picks inits activity, but rather a smooth activity behavior.
SFZ Engine----------------
There seem to be some problems in the SFZ engine that generates unusual CPU activity. There were frequent sudden spikes (compared to smoothness of GIGA). The activity also seems to be higher than GIGA engine, regardless of the setting. The high amount of JACK DSP load was noticeable for the SFZ engine. And as you can guess, I had frequent X-Runs in the SFZ tests.
Conclusion=========
First, I must say that this is based on my test and my system. I cannot be sure about this conclusion unless other developers reproduce my tests to see if they get the same results. But based on my results, I can conclude that it is likely that the SFZ engine has a subprocess that requires high CPU runtime and it occurs not very frequent in a normal playing. It can be something related to voice stealing or something that is not triggered with a simple nice and slow piano playing.
On Wednesday, April 8, 2020, 02:05:53 PM GMT+2, joo bian via Linuxsampler-devel <lin...@li...> wrote:
I thought I mentioned that somewhere in the thread. I only use uncompressed Wav files to avoid any unnecessary encoding. Most of my samples are 24bit 44.1khz or 24bit 48khz and they are all Wav.
Regarding Jacek's comment, I am recompiling LS with different configurations and it seems the behavior of --enable-preload-samples= argument is a little bit unpredictable (well, to me). The default value is 32768, which in practice (for 180 max voice) it loads 730MB out of a 8.5GB sample library.
I recompiled it with setting it 10 times larger, --enable-preload-samples=327680 and it loaded 2.6 GB. Recompiled it with 100 times larger and it loaded 5.3GB. Recompiled it with 1000 time larger value of 32768000 and the amount of preloaded samples did not go anything above 5.3GB
Side note------------
I saw some unusual behavior here as well. Surprisingly, when I increased the "max voices" LS can actually by far surpass the size of the library. For the 8.5GB sample library, LS could require 12GB if I increase the max voices. I only did this for testing (no other reasons to apply such a setting). it seems each stream can also occur independent of pre-loaded samples or at least its allocated RAM memory is independent of pre-loaded sample? I'm not sure. I thought the streams should take the preloaded data into account.
On Wednesday, April 8, 2020, 12:18:01 PM GMT+2, Christian Schoenebeck <sch...@li...> wrote:
On Dienstag, 7. April 2020 16:36:48 CEST joo bian wrote:
> Dear Christian, Jacek,
>
> Thanks for your kind replies. As Christian pointed out, I also don't think
> that a little bit of optimization plays a significant role, giving my
> hardware (and the fact that I get the dropouts even while my CPU usage is
> less than 25% and memory usage is less than 10%).
File format of samples being used (wav, flac, ogg vorbis)?
CU
Christian
_______________________________________________
Linuxsampler-devel mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxsampler-devel
_______________________________________________
Linuxsampler-devel mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxsampler-devel
|
|
From: Christian S. <sch...@li...> - 2020-04-09 13:53:31
|
On Donnerstag, 9. April 2020 15:05:40 CEST joo bian wrote: > GIGA Engine------------------ > I had no problem at all with the GIGA instrument. No X-RUN, no unusual CPU > activity, everything was perfect. JACK2 DSP load also was quite low during > the performance, which is often associated with X-RUNS. Also my CPU did not > have sudden picks inits activity, but rather a smooth activity behavior. > SFZ Engine---------------- > There seem to be some problems in the SFZ engine that generates unusual CPU > activity. There were frequent sudden spikes (compared to smoothness of > GIGA). The activity also seems to be higher than GIGA engine, regardless of > the setting. The high amount of JACK DSP load was noticeable for the SFZ > engine. And as you can guess, I had frequent X-Runs in the SFZ tests. > Conclusion========= > First, I must say that this is based on my test and my system. I cannot be > sure about this conclusion unless other developers reproduce my tests to > see if they get the same results. But based on my results, I can conclude > that it is likely that the SFZ engine has a subprocess that requires high > CPU runtime and it occurs not very frequent in a normal playing. It can be > something related to voice stealing or something that is not triggered with > a simple nice and slow piano playing. So you identified some of the SFZ engine's individual code portion to be the cause of the high CPU load there (i.e. unlike with the gig engine), good! Next step would be profiling the sampler while reproducing that high CPU load there. The profiling data would give us all insight where exactly the SFZ code is spending too much CPU time on in your case. CU Christian |
|
From: joo b. <joo...@ya...> - 2020-04-09 14:12:57
|
I can gladly generate the profiling for you guys, as I really wish for this problem to be solved as soon as possible. Do you have some suggestions for how to go about the profiling?
In the linuxsampler configuration there are some arguments for detailed logging and debugging. Should I activate them? I am not very handy with Linux, but I am learning it. By that, I mean I have never done profiling for a software, but I'll be happy if LS would be the first :)
Is there some links/code suggestions I can use to make sure I can deliver the results straight on?
Cheers,
Ebad
On Thursday, April 9, 2020, 03:53:53 PM GMT+2, Christian Schoenebeck <sch...@li...> wrote:
On Donnerstag, 9. April 2020 15:05:40 CEST joo bian wrote:
> GIGA Engine------------------
> I had no problem at all with the GIGA instrument. No X-RUN, no unusual CPU
> activity, everything was perfect. JACK2 DSP load also was quite low during
> the performance, which is often associated with X-RUNS. Also my CPU did not
> have sudden picks inits activity, but rather a smooth activity behavior.
> SFZ Engine----------------
> There seem to be some problems in the SFZ engine that generates unusual CPU
> activity. There were frequent sudden spikes (compared to smoothness of
> GIGA). The activity also seems to be higher than GIGA engine, regardless of
> the setting. The high amount of JACK DSP load was noticeable for the SFZ
> engine. And as you can guess, I had frequent X-Runs in the SFZ tests.
> Conclusion=========
> First, I must say that this is based on my test and my system. I cannot be
> sure about this conclusion unless other developers reproduce my tests to
> see if they get the same results. But based on my results, I can conclude
> that it is likely that the SFZ engine has a subprocess that requires high
> CPU runtime and it occurs not very frequent in a normal playing. It can be
> something related to voice stealing or something that is not triggered with
> a simple nice and slow piano playing.
So you identified some of the SFZ engine's individual code portion to be the
cause of the high CPU load there (i.e. unlike with the gig engine), good!
Next step would be profiling the sampler while reproducing that high CPU load
there. The profiling data would give us all insight where exactly the SFZ code
is spending too much CPU time on in your case.
CU
Christian
_______________________________________________
Linuxsampler-devel mailing list
Lin...@li...
https://lists.sourceforge.net/lists/listinfo/linuxsampler-devel
|
|
From: Christian S. <sch...@li...> - 2020-04-10 09:34:27
|
On Donnerstag, 9. April 2020 16:12:42 CEST joo bian wrote: > I can gladly generate the profiling for you guys, as I really wish for this > problem to be solved as soon as possible. Do you have some suggestions for > how to go about the profiling? In the linuxsampler configuration there are > some arguments for detailed logging and debugging. Should I activate them? Profiling is done by using an external development tool. That's not a built-in feature of the sampler. There are numerous profilers out there now. IMO e.g. OProfile was always a good choice on Linux. There are various good howtos on the Internet how to perform profiling with the profiler of your choice. Just keep in mind that before you start profiling, the sampler must be compiled with debugging symbols turned on, which means additionally to the -O3 flag you also have to add the -g flag. Also: the sampler's binary must not be stripped after compilation, which e.g. the Debian packaging scripts would do by default, because stripping removes the debugging symbols from the binary again. For that reason I recommend you to not even bother to install the sampler in this case, just type CXXFLAGS="-O3 -g" ./configure && make from the source file directory to compile it, then simply run the sampler with src/linuxsampler CU Christian |
|
From: joo b. <joo...@ya...> - 2020-04-15 14:47:17
|
Here is my first attempt on profiling the SFZ engine as you recommended, using Oprofile and the settings you mentioned. This was done with the default setting of LS, i.e. 64 voice polyphony and 90 max voices. After the profiling, I get the following report ```CPU: AMD64 family15h, speed 2100 MHz (estimated) Counted CPU_CLK_UNHALTED events (CPU Clocks not Halted) with a unit mask of 0x00 (No unit mask) count 100000 CPU_CLK_UNHALT...| samples| %| ------------------ 2857302 99.9834 bash CPU_CLK_UNHALT...| samples| %| ------------------ 2112992 73.9506 liblinuxsampler.so.5.0.0 602207 21.0761 kallsyms 49469 1.7313 libjack.so.0.1.0 49023 1.7157 libc-2.31.so 14103 0.4936 libstdc++.so.6.0.28 11851 0.4148 libsndfile.so.1.0.28 5947 0.2081 [vdso] (tgid:74493 range:0x7ffc0e633000-0x7ffc0e633fff) 5729 0.2005 ext4 3855 0.1349 libpthread-2.31.so 1676 0.0587 libm-2.31.so 266 0.0093 ld-2.31.so 77 0.0027 kvm 42 0.0015 bash 30 0.0010 lt-linuxsampler 24 8.4e-04 libgcc_s.so.1 3 1.0e-04 jbd2 2 7.0e-05 libdb-5.3.so 2 7.0e-05 libmvec-2.31.so 1 3.5e-05 filter.so 1 3.5e-05 libncursesw.so.6.2 1 3.5e-05 libreadline.so.8.0 1 3.5e-05 librt-2.31.so 317 0.0111 sed CPU_CLK_UNHALT...| samples| %| ------------------ 212 66.8770 libc-2.31.so 88 27.7603 kallsyms 14 4.4164 ld-2.31.so 3 0.9464 sed 156 0.0055 ls CPU_CLK_UNHALT...| samples| %| ------------------ 77 49.3590 libc-2.31.so 66 42.3077 kallsyms 11 7.0513 ld-2.31.so 2 1.2821 ls ``` I also got messages from LinuxSampler on the terminal occasionally: ```DiskThread: there was no free slot Disk stream order failed! EngineBase: ERROR, voice stealing didn't work out! DiskThread: there was no free slot Disk stream order failed! EngineBase: ERROR, voice stealing didn't work out! DiskThread: there was no free slot Disk stream order failed! EngineBase: ERROR, voice stealing didn't work out! DiskThread: there was no free slot Disk stream order failed! EngineBase: ERROR, voice stealing didn't work out! ``` Best, Ebad On Friday, April 10, 2020, 11:34:40 AM GMT+2, Christian Schoenebeck <sch...@li...> wrote: On Donnerstag, 9. April 2020 16:12:42 CEST joo bian wrote: > I can gladly generate the profiling for you guys, as I really wish for this > problem to be solved as soon as possible. Do you have some suggestions for > how to go about the profiling? In the linuxsampler configuration there are > some arguments for detailed logging and debugging. Should I activate them? Profiling is done by using an external development tool. That's not a built-in feature of the sampler. There are numerous profilers out there now. IMO e.g. OProfile was always a good choice on Linux. There are various good howtos on the Internet how to perform profiling with the profiler of your choice. Just keep in mind that before you start profiling, the sampler must be compiled with debugging symbols turned on, which means additionally to the -O3 flag you also have to add the -g flag. Also: the sampler's binary must not be stripped after compilation, which e.g. the Debian packaging scripts would do by default, because stripping removes the debugging symbols from the binary again. For that reason I recommend you to not even bother to install the sampler in this case, just type CXXFLAGS="-O3 -g" ./configure && make from the source file directory to compile it, then simply run the sampler with src/linuxsampler CU Christian _______________________________________________ Linuxsampler-devel mailing list Lin...@li... https://lists.sourceforge.net/lists/listinfo/linuxsampler-devel |
|
From: Christian S. <sch...@li...> - 2020-04-15 16:15:20
|
On Mittwoch, 15. April 2020 16:47:06 CEST joo bian wrote: > Here is my first attempt on profiling the SFZ engine as you recommended, > using Oprofile and the settings you mentioned. This was done with the > default setting of LS, i.e. 64 voice polyphony and 90 max voices. > > After the profiling, I get the following report > ```CPU: AMD64 family15h, speed 2100 MHz (estimated) > Counted CPU_CLK_UNHALTED events (CPU Clocks not Halted) with a unit mask of > 0x00 (No unit mask) count 100000 CPU_CLK_UNHALT...| > samples| %| > ------------------ > 2857302 99.9834 bash > CPU_CLK_UNHALT...| > samples| %| > ------------------ > 2112992 73.9506 liblinuxsampler.so.5.0.0 > 602207 21.0761 kallsyms > 49469 1.7313 libjack.so.0.1.0 > 49023 1.7157 libc-2.31.so > 14103 0.4936 libstdc++.so.6.0.28 > 11851 0.4148 libsndfile.so.1.0.28 > 5947 0.2081 [vdso] (tgid:74493 > range:0x7ffc0e633000-0x7ffc0e633fff) 5729 0.2005 ext4 > 3855 0.1349 libpthread-2.31.so > 1676 0.0587 libm-2.31.so > 266 0.0093 ld-2.31.so > 77 0.0027 kvm > 42 0.0015 bash > 30 0.0010 lt-linuxsampler > 24 8.4e-04 libgcc_s.so.1 > 3 1.0e-04 jbd2 > 2 7.0e-05 libdb-5.3.so > 2 7.0e-05 libmvec-2.31.so > 1 3.5e-05 filter.so > 1 3.5e-05 libncursesw.so.6.2 > 1 3.5e-05 libreadline.so.8.0 > 1 3.5e-05 librt-2.31.so That's the default view type of most profilers which just shows the timing for the individual DLLs. What we rather need is the timing of the individual functions inside of liblinuxsampler. That's the relevant information to see where exactly in the SFZ engine the sampler consumes too much CPU time. > ``` > I also got messages from LinuxSampler on the terminal occasionally: > ```DiskThread: there was no free slot > Disk stream order failed! > EngineBase: ERROR, voice stealing didn't work out! > DiskThread: there was no free slot > Disk stream order failed! > EngineBase: ERROR, voice stealing didn't work out! > DiskThread: there was no free slot > Disk stream order failed! > EngineBase: ERROR, voice stealing didn't work out! > DiskThread: there was no free slot > Disk stream order failed! > EngineBase: ERROR, voice stealing didn't work out! > ``` Yeah, ignore them for now. You probably get them when switching sounds while still holding some notes. CU Christian |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X