#2204 Poor performance on windows using pipes

[Anonymous]

I noticed that gnuplot on windows is really slow if used with pipes.
This becomes bothersome if large gnuplot script files are send via pipes at a high rate like 20fps.
The problem is that in winmain.c line 1111 for every (!) character a new thread is started.
In the end gnuplot.exe spends almost more time in kernel than in user mode.

My suggestion would is to use a proper approach to read async from a windows pipe.
Sadly overlapped IO doesn't work with anonymous pipes.
So the fix could be a long running worker thread which pulls characters from the input queue and adds them to some buffer, which is read by the main thread.

I tried to implement just that but I don't have a real development environment for gnuplot.

I'm using Windows 10 x64 1809 and Gnuplot 5.2 patchlevel 7

[Bastian Märkisch]

You are absolutely right. Starting a new thread for every single character in a pipe is inefficient. Plus, it turns out not to be necessary at all to start a new thread. We can just use MsgWaitForMultipleObjects() directly.

A fix is now in the master branch and should be applied to 5.4 after some testing.

[Markus]

(I'm the ticket creator)
It is indeed faster now. I tested it with a custom build on the current master on my machine.
However, if the thread of the parent process is blocked Gnuplot is also not responsive.
This seems to be an artefact of my local build.

[Bastian Märkisch]

Thanks for testing. Now also applied to the 5-4-stable branch, but did not make it into the rc1.

[Bastian Märkisch]

Unfortunately, my original "solution" to this was broken. The new one is now more in line with your proposal, but I just use a one character buffer and two event objects. It still circumvents the major bottleneck of recreating the thread for every single character. I would appreciate if you could test if that still results in acceptable performance.

[Markus]

Thanks! Yes I'll test it. But I no longer own the machine where I set up the gnuplot buildsystem and the test code, thus it might take I while until I have time to recreate it.

[Bastian Märkisch]

Thanks. You can now find a binary test build in https://sourceforge.net/projects/gnuplot/files/gnuplot/testing

[Markus]

I found my old benchmark code. Sadly I don't have any old data for comparison.

I tested some gnuplot versions and was not able to notice time differences across the versions, which is strange!

I'm no longer using gnuplot on windows thus I can't confirm it with my code where the problem originally arose.

I uploaded the numbers and the code here:
https://gist.github.com/MBulli/622109e22324227a1af4af7de80834c5

[Bastian Märkisch]

Thanks. That testcase helps a lot!
Unfortunately, in my first testing using code based on your testcase, none of the three variants of the gnuplot code yields acceptable results: either the window is "blocked" (ie. events are not handled) or the data is scrammbled. So this need further work...