#1107 problem on "points palette" with binary data include NaN

[Hiroki Motoyoshi]

When I tried to plot in "points palette" style with binary data includes NaN in 3rd column,
gnuplot generates some curious results.
With X11, the window closes instantaneously after opened.
With pdfcairo, the points corresponds to NaN value are colored with
the same color with the 3rd column at previous row.

Sample code is here.

--

set xrange [0:6]
set yrange [0:6]
# set terminal pdfcairo
# set output "test.pdf"
plot "test.dat" binary record=5 format="%f%f%f" using 1:2:3 with points palette pt 7
pause -1

--

The data file "test.dat" can be generated by Ruby script,

--
data = [
1, 2, 1,
2, 3, 2,
3, 1, 0.0/0.0,
4, 5, 3,
5, 4, 0.0/0.0,
6, 2, 2,
]

open("test.dat", "w") { |out|
out.write data.pack("f18")
}

---

The sample code and data file are also in file attatachment.

My environment is
G N U P L O T
Version 4.7 patchlevel 0 last modified 2012-03-18
Build System: Darwin x86_64

[Hiroki Motoyoshi]

test codes and data

[Ethan Merritt]

--- gnuplot/src/datafile.c 2012-01-21 17:32:47.000000000 -0800
+++ gnuplot-cvs/src/datafile.c 2012-03-31 14:05:15.000000000 -0700
@@ -4693,6 +4693,12 @@ df_readbinary(double v[], int max)
line_okay = 0;
break; /* return or ignore depending on line_okay */
}
+ if (isnan(v[output])) {
+ int_warn(NO_CARET,"NaN input value");
+ // return DF_UNDEFINED;
+ line_okay = 0;
+ break;
+ }
}

/* Linear translation. */

[Hiroki Motoyoshi]

I tried your fix in my environment.
Your fix seems to resolve the problem.

Thank you for your quick response !

[Hiroki Motoyoshi]

Sorry, your quick fix generates other problems in other case (such as splot).

It seems this simple fix should not be committed.

[Ethan Merritt]

Yes, the simple patch is by itself not a complete solution.
Could you tell me the splot command that failed? I will make sure to add it to the set of test cases for a complete fix.

[Hiroki Motoyoshi]

example to show the problem brought by the simple patch

[Hiroki Motoyoshi]

I uploaded the example to show the case the splot command fails.

In the simple patch case,
I think the x-y grid structure for splot is broken when NaN values are read.

[Hiroki Motoyoshi]

How about this one.
It looks to avoid both of two problem pointed out here.

Index: datafile.c

RCS file: /cvsroot/gnuplot/gnuplot/src/datafile.c,v
retrieving revision 1.216
diff -u -r1.216 datafile.c
--- datafile.c 8 Apr 2012 17:45:40 -0000 1.216
+++ datafile.c 9 Apr 2012 15:33:17 -0000
@@ -4683,8 +4683,11 @@
/* so let the general case code handle it. */
} else if ((column <= df_no_cols)
&& df_column[column - 1].good == DF_GOOD)
- v[output] = df_column[column - 1].datum;
-
+ if ( isnan(df_column[column - 1].datum) )
+ return DF_MISSING;
+ else
+ v[output] = df_column[column - 1].datum;
+
/* EAM - Oct 2002 Distinguish between DF_MISSING
* and DF_BAD. Previous versions would never
* notify caller of either case. Now missing data

[Ethan Merritt]

Yesterday I added new code to CVS for version 4.7 addressing this issue. Please try the new version and let me know if you discover that there are still problems remaining.

[Hiroki Motoyoshi]

example to show the difference is splot between original and new CVS version

[Hiroki Motoyoshi]

Thank you for your reply.

I tried new CVS version.

First of all, It resolves the problem of "point palette with NaN" like first patch.

Second, for splot with NaN value at 3rd column, there is a small difference between original and new CVS version.

The difference appears in autoscaling of X-Y grid of splot, when NaN value are on the boundary. The attached files "test3.gp" draws two plots without setting x and y ranges [i.e. autoscaling]. First plot is same but second plot differs between original version and new CVS version of gnuplot.

For me, the difference is no problem.
But I don't know it is serious difference for somebody.

[Ethan Merritt]

Yes. The new behavior is a change from the way NaN has always worked in the past. NaN points are now treated more like "missing" points. More importantly, there is no difference between reading NaN as "using 1:2" and reading it as "using ($1):($2)".

I have attached another simple script that demonstrates this.

[Ethan Merritt]

demonstrate change in treatment of NaN

[Hiroki Motoyoshi]

example to show that "image failsafe" fails with new CVS version

[Hiroki Motoyoshi]

New CVS version causes another problem about "image failsafe"

It's my guess that NaN has a special meaning in "image failsafe" routine.

I have attached example for this issue.

[Ethan Merritt]

Thank you for your extensive tests!
There was a subtle flaw in the original patch to df_readbinary(), where it tested the wrong internal flag to decide if the NaN appears in the context of a structured array ("matrix"). This error is now corrected in the CVS version of datafile.c

[Hiroki Motoyoshi]

Thank you for resolving complicated problem.

I often use binary data includes NaN (represents missing values)
passing to gnuplot from ruby. This fix helps my work!