Noticed spikes in the data
Brought to you by:
gragus
Menu
▾
▴
I am the compiled jar file to plot a lot of real-time data coming from a robot.
I find that there are pretty frequent spikes that occur in the data especially when trying to plot at high refresh rates (like 0.5 Hz and above).
The spikes are not present in the log file .. not sure why they come up on the displayed graph. Toggling better "Show Tail" and "Show All data" sometimes makes the spikes dissapear.
Any thoughts ? This is real killer for me .. cos it results in the graph scaling to a completly different range because of the spikes and therefore its hard for me to view what I really want.
Any thoughts will be appreciated
Cheers
Kash
I ran into a similar problem myself. The problem turned out not to be LiveGraph, but the way data was being sent to it from the application whose results were being plotted.
A common way to send data to an app like LiveGraph is to redirect a command's output to a file, for example:
output-file.dat is the text file which LiveGraph reads.
But when you do this, the operating system will buffer the output from example-cmd before writing it to the output file. Each individual write from example-cmd won't be written to the output file immediately; the writes will be saved in a buffer and when the buffer gets large enough, it will be written to the file all at once. The operating system does this to improve system performance.
For many applications, such as scripts, the default buffering to use is "block buffering". That means a fixed-size block, say 1024 characters, is used for the buffer, and it is written whenever the buffer fills up. Now suppose your buffer happens to fill up in the middle of a line of output. Only part of the line will be written. LiveGraph will see that the file has changed, will read the new data, including the partly-written line, and then will plot some point whose value is based on a partly-written number.
That's why toggling between show-all and show-tail sometimes eliminates the spikes, because it forces LiveGraph to reread the file, and the file's partly-written lines may have been completed by that point.
Anyway, this behavior is obviously bad. If LiveGraph were a bit smarter about things, it would ignore partly-written lines and only process complete lines with an end-of-line marker. But it doesn't do this, so you have to solve matters yourself.
The specific solution will depend on how your application is written, but basically, you have to tell the system to do line-buffered output instead of block-buffered output. With line-buffered output, the buffer is written to the output file every time the application writes a complete line with an end-of-line marker. Thus, the last line in the file will always be a complete line, and LiveGraph will never read truncated data. Exactly how you will achieve this will depend on how your application is written, but there are usually calls to set line-buffering explicitly or to force output to be flushed at a specific point. It may help to arrange your method of performing output so that each output line is always written all at once with a single write operation.
Hope this reply helps anyone who happens to read it…