Re: [Linuxsampler-devel] testing of linuxsampler and cvs activity

[<be...@ga...>]

Scrive Robert Jonsson <rob...@da...>:

> Hi guys,
> 
> I tried linuxsampler yesterday with the aid of benno, looks great! 
> 
> cpu load was low and it seemed pretty stable.
> 
> I encountered a few irks that according to benno are known issues, i'll list
> 
> them just for the record:
> 
> - clicks when doing note_off

this is because the sample is cutoff hard without any release envelope because
the enveloping stuff is still missing. So it is not an error.

> - trying the original gigapiano it sounded very strange when playing hard. To
> 
> me it sounded as if it the samples where played from the wrong key (one 
> octave up), benno told me there where issues with the midimapping which may 
> explain this.

yes exactly, Christian will add the complete articulation implementation
 soon so these strange effects will go away.
(actually libgig supports full articulation, it's linuxsampler that does not
yet honor all the values).

> - it segfaults when ctrl-c:ing out of the app.

probably due to the incorrect stopping of threads when the CTRL-C handler
is called, not a serious issue and probably very easy to fix.
As said it would be more serious if linuxsampler segfaulted during playing. :-)

> - on my system i get mlockall failures all the time, regardless of the size
> of 
> the gig file and such... how much mem does LS allocate? (I've got 512mb so it
> 
> should work rather well I think. I can run debug compiled-muse together with
> 
> jack with no problems, which both do mlockall and occupy lots of memory...)

see diskthread.h:
#define STREAM_BUFFER_SIZE 131072
#define MAX_INPUT_STREAMS 64

this means we can have max 64 disk streams (each voice if streamed from
disk exactly needs one stream).
We are talking about 16bit samples (2bytes) thus
the streams occupy
MAX_INPUT_STREAMS*STREAM_BUFFER_SIZE*2

this is not an extremely high amount, in the above case 16MB

most of the memory is consumed by the pre-cached initial parts of the samples.

see audiothread:
#define NUM_RAM_PRELOAD_SAMPLES

normally we set this value to 65536, which means for mono 16bit samples
we preload 65536*2=128KB of memory for each sample.
If the .GIG contains 1000 samples we use 128KB*1000= about 128MB of memory.
For stereo samples you must multiply the number by 2.
(in the above case 256KB of memory per stereo sample).
Preloading 65536 samples means 1.48secs of samples at 44.1kHz.
This is needed to overcome the delay that occurs for the disk thread
filling up the ringbuffers where the remaining part of the sample will
be streamed from.
The most critical case occurs when you start lots of notes at the same
time. Eg assume we have 50 samples (all different) and start them all
at the same time. This means that within 1.48secs (the RAM part of the sample),
the disk thread has to fill at least some samples in ALL 50 ringbuffers
associated to the voices. This means that in the above case the disk thread
might fail to meet the deadlines and you would risk that the audio thread
wants to fetch the audio data from the diskstream ringbuffers but the data
has not been loaded yet, leading to an error condition (we opted for note cutoff
and report the error).
The solution to this is tuning the disk streaming (there is room for improvement
by cutting read() sizes in critical situations but I will add this stuff at a
later stage), installing faster disks or increase the size of the part of sample
we preload in RAM. 
All disk based samples like giga,kontakt,halion have this problem when
many voices are triggered simultaneously and often the only solution is to
increase the RAM preload size which has the disadvantage that you can load
fewer sample libraries at the same time.
People that make extensive use of disk based sampler often have machines
with 1-2GB RAM to load as much sample libraries as possible.
There are sample libraries that require a minimum of 0.5-1GB to be loaded.
(the size of these libs is often 2-4GB).
For example NI Kontakt while having more features and offering better sample
 manipulation tools than Giga, is not that performant when it comes to streaming
like Giga (giga uses kernel based streaming, special low latency GSIF drivers
etc). This means in Kontakt you have often to increase preload sizes to up to
600KB which greatly reduces the size and number of sample libraries you can
load at the same time.
You know ... 32bit x86 boxes hit the wall at 1-2GB RAM because of the limited
 addressing space but many users using a disk based sampler would like to
install more ram, perhaps 8,16GB and more.
Even if there are already relatively cheap 64bit x86 boxes around, most notably
 the AMD Opteron / Athlon 64, Windows cannot use its full capabilities because
 the 64bit support is not it yet. 
Read this and you will see that Windows XP on AMD 64bit CPUs will take long long
time to become a viable option: (a very bumby road I would say)
http://www.pcworld.com/news/article/0,aid,112749,pg,6,00.asp

On the other hand Linux already runs on AMD 64bit CPUs and linuxsampler is
64bit compatible too.
This means we can beat the windows sampler at their own game: linuxsampler
already runs on performant 64bit boxes and what's more important we can handle
much bigger amounts of RAM which is crucial for those wanting to load many
samples at the same time.
I guess once linuxsampler's engine is production ready and can play
preexisting libraries correctly it will turn up on the windows people 
radarscreens, I assume even those using the windows samplers professionally
because those are the people that need these monster installations with
dozen, even hundreds of GBs worth of samples.
And I don't even touched the fact that linuxsampler could be clustred to
provide sampler farms that can render thousand of voices and pipe them
back in real time over ethernet without requiring expensive audio cards,
midi interfaces etc for each cluster slave.

Only time will tell we we will remain a niche product or we will get 
high visibility. For now let's make work the basic engine perfectly.


Returning to you mlockall problem:
try to lower the NUM_RAM_PRELOAD_SAMPLES eg to 16384 (do a make clean
because there are some dependencies missing in the Makefile).

This could help but keep in mind if you preload fewer samples from disk
you risk to run into the problem described above where the disk thread
is unable to fill the ringbuffer in time, leading to note cut-offs.

> 
> All in all, this looks set to be a real killer app! :)

We're all working hard to make it happen.
As said every contribution count, not only code contributions.
Ideas, testing, debugging, advices from musicians, sampler/samplelibrary
 experts all count.

> 
> The things I miss are (probably mostly on the todo already)
> 
> - gui (not strictly necessary)

I have half finished a simple load&play GUI (load samples, assign them
to midi channels and start playing, plus the GUI permits you to set
basic settings like number of voices, RAM preload sizes etc, shows you
activity and real time voice count etc).
As I stated before linuxsampler is remotely controllable via TCP/IP which
means the GUI can run on a different box and even on a different platform.
What I am working on now is the remote control protocol which is a simple
API that can be used by anone so anyone can build their favorite kind of 
remote control application.
It can be a GUI, a text based application, a script, hardware buttons etc.
The default GUI I will provide is written using the Qt library because of
its portability. This will allow to run the GUI frontend on Windows and 
Mac too.
This is because I'm assuming sooner or later we will see those linuxsampler
clusters remote controlled by Windows PCs or Macs :-).
Plus do not forget separating the frontend from the engine forces you
to follow certain rules during coding which leads to a better quality
of the code and helps to isolate errors and performance problems.


> - jack output

Of course. We will go as far that linuxsampler will not only export
the main output (where all vocies will be mixed too) but single
midi channel mix buffers which you can send to jack unprocessed
and process the channels with your own FXes or record the tracks
into eg ardour for further editing.
But for now we only support direct ALSA out because we must first
iron out potential latency problems.
(if you put jack into the game then you cannot easily figure out
if it's linuxsampler's or jack's fault).
Divide et impera is the keyword here :-)

> - soundfont support, for now I'm perfectly happy with gig support, I wonder 
> how hard it would be to add soundfont support though..?

Well soundfont offers quite some modulation possibilities 
(Peter Hanappe and Josh Green know this better than anyone).
I guess for now if you want SF2 it is better to use fluidsynth.
SF2 is IMHO a bit limited and linuxsampler aims at the professional sampling
 domain so our priority is to make .GIG working. (at a later stage we will
try to add 24bit sample in Halion / Kontakt formats too).

Or alternatively when our GIG engine will be complete you can try to
convert your SF2 files using a conversion program like cdxtract
http://www.cdxtract.com (but relatively expensive $150).
Or better, integrate the fluid engine to linuxsampler :-)

> 
> - hmmm I think I ran it as an ordinary user (with pretty good results), it 
> occurs to me that SCHED_FIFO should not be available then. LS uses SCHED_FIFO
> 
> right? Should it complain if it can't set it? (this may be a misconception of
> 
> mine though)

In theory it should complain:
    if (sched_setscheduler(0, SCHED_FIFO, &schp) != 0) {
        perror("sched_setscheduler");
        return -1;
    }

anyway sched_setscheduler() is deprecated because for threads one should
use pthread_attr_init() to set priorities.
I heard rumors that sched_setscheduler() does not work correctly on
recent threading libraries but I cannot confirm it.


> 
> ----
> And finally, a wish. I think this mailinglist is too quiet, it's hard to know
> 
> if there is any activity at all.
> 
> May I propose that you guys start announcing cvs checkins on this list?

We will certainly announce new (relevant) CVS checkins on the list 
(manually).
Regarding myself I'm working on the GUI/remote control protocol and it will
take some time to make it work well, document etc so I guess I will not
check in new stuff before 1-2 weeks. Perhaps Christian will add some stuff
meanwhile and I'm sure he will announce it on this mailing list.

BTW: for those that wantet to check out the CVS source and did not
find the exact location here is how to get the sources:

cvs -z3 -d:pserver:ano...@cv...:/home/schropp/linuxsampler co
linuxsampler

cheers,
Benno


-------------------------------------------------
This mail sent through http://www.gardena.net