#2530 set terminal canvas doesn't seem to handle fractional seconds

[Henrique Martins]

Gnuplot 5.4 patchlevel 3 last modified 2021-12-24 on Fedora 36.
I have a file with timestamps containing milliseconds.
Plotting that file with a timefmt of '%Y/%m/%d %H:%M:%S' display fine on the qt terminal.
Plotting to the canvas terminal all x values seem to be truncated to seconds, at least when zooming in, generating a lot of vertical lines for data values within the same second.

[Ethan Merritt]

Could you please clarify whether you are referring to the reported mouse coordinates or to the lines making up the plot itself?

The time value of the current mouse coordinates are reported to the user via a standard javascript routine toUTCString(), which as you say does not know about milliseconds. It would be possible to replace this with custom-written routine that did know about milliseconds, but someone would have to write that routine. I think no change to gnuplot per se would be needed; only the dynamically loaded mousing routines in gnuplot_mouse.js would be affected.

But you mention zoom, and I don't see how zoom figures into the above. Whatever lines are drawn to make the plot are specified in screen/pixel coordinates (not sure what the proper terminology is), not in seconds or milliseconds. Zooming in or out just compresses the scale used to draw those lines. So I don't understand what could produce spurious vertical lines.

Can you attach a reproducible test case that demonstrates this?
I.e. attach the html file produced by

set terminal canvas standalone mousing 
set output 'bug2530.html'
... plot commands ...

[Henrique Martins]

It's the lines making up the plot, and while trying to get a sample for this reply I found out it depends on how much data there is!

If I plot the whole data (style is linespoints), I can't see much, that's why I need to zoom in.

In the qt terminal I can zoom in and the plot looks correct, on the html5 canvas terminal the plot rounds the x values resulting in vertical lines at the rounded values.

However, if I crop the data file to about the area I zoomed in into the full data , both terminals behave properly, even if I further zoom in!

Script is simply:

set xdata time
set timefmt '%Y/%m/%d %H:%M:%S'
set format x '%H:%M'
set style data lines
set datafile separator ','

Then for qt, I use:
plot "data" using 1:2 with linespoints

For canvas, I use:
set terminal canvas size 1024,600 standalone mousing
set output 'bug2530.html'

(My script sets also sets the jsdir canvas option, omitted here)

I'll attach two data files, one is data-full, with 58519 lines, the other data-cropped, with only 270, containing the lines from data-full with timestamps between 07:15:20.061 to 07:15:53.992.

Attached images qt.png and canvas.png show what the plots look like with the data-full, then zoomed in to about the range in data-cropped (actually a little larger), with the canvas.png showing the problem.

Again, for data-cropped the data is shown correctly on both cases.

-- Henrique

[Ethan Merritt]

I think what you are seeing is the inevitable difference in resolution between a 64-bit floating point representation of the x coordinate, which is what Qt sees, and an ascii text representation using 5 digit integers, which is what is written to the output file for interpretation by the HTML5 canvas code in a browser.

Taking the exact values from your example code

• The request canvas terminal size is 1024x600
• The gnuplot canvas terminal oversamples by a factor of 10X; i.e. it multiplies the coordinate by 10 and write the integer part to the output stream
• Therefore the x coordinates seen by the browser run from 0 to 10240
• The range of x time values in your full data set is 07:00:17.494 to 09:04:32.648
• That works out to 7440 seconds (7440200 milliseconds)
• Therefore the resolution available in the integer representation is 7440200/10240 = 726 milliseconds.

Another way of looking at this is that the 10-fold oversampling allows you to zoom by up to a factor of 10 with single-pixel precision. For zoom factors larger than 10 you can start to see that the conversion from floating point to 5-digit integer has effectively placed the original data into discrete bins of width 700 milliseconds.

This same limitation hits other terminals that use a text representation with a fixed number of digits or decimal places. By request, oversampling for the svg and cairo terminals was recently increased to provide essentially 2 decimal places of precision (100-fold oversampling) rather than a single decimal place.
Bug 2369
Bug 2499

The equivalent could be done for the canvas terminal, but there are some caveats

1. I am not sure to what extent it would be possible to mix output from the new and old gnuplot versions on the same web page or html document. Maybe it would work; maybe not; it depends on how clever we could be in making the change.
2. 100-fold oversampling still wouldn't yield millisecond precision in your example. So how far is it reasonable to push this? 1000-fold? 10000-fold? Each increase makes the text output file that much larger, which eventually starts to be an issue for large amounts of data.
3. Perhaps if it is possible to be sufficiently clever (point 1) then we could let the user specify and oversampling factor. I considered and rejected that when modifying the cairo terminals, but each terminal is worth separate thought.

[Henrique Martins]

Thanks. I started playing with different size croppings of the data to try to understand when it would break, did look at the js code, saw the 5 digit coordinates and was wondering what the 10x factor was for, but it would take me a long time to figure this out without the source code.

The data I sent is real temperature readings from a temperature controller trying to maintain a set point for device under test. Tests typically take anything from thirty minutes to a few hours, but could take days.

There's a web interface that can pull the temperature log and sometimes there's a need to look at some transients. For those long tests it won't work and the data will need to be moved to another tool (usually JMP around here, just because they have it).

Guess one more digit wouldn't be bad, but in the end, to keep this as an HTML5 canvas page I probably need to do some windowing into the data.

Thanks for the quick reply.