Weirdness in column specifications, column() vs dollar

[Juhász P. <pet...@gm...>]

Dear gnuplot list members,

today I tried to modify a plot so that it applies a default value if
one of the columns was empty, and for the longest time I couldn't
understand why it didn't work. I managed to make it work in the end,
but I still don't understand it. There are strange differences in
behavior, depending on whether `column(x)` or `$x` is used, and it even
depends on the datafile separator. I've distilled the issues I've found
into the following minimal example:

#------------------------------------------------
show version

$data_with_whitespace << EOD
1 1
2
3 3
EOD

$data_with_semicolons << EOD
1;1
2;
3;3
EOD

set table

plot $data_with_whitespace u 1:($2>0?$2:0)
plot $data_with_whitespace u 1:(column(2)>0?$2:0)
plot $data_with_whitespace u 1:(column(2)>0?column(2):0)
plot $data_with_whitespace u 1:(valid(2)?$2:0)
plot $data_with_whitespace u 1:(valid(2)?column(2):0)

set datafile separator ';'

plot $data_with_semicolons u 1:($2>0?$2:0)
plot $data_with_semicolons u 1:(column(2)>0?$2:0)
plot $data_with_semicolons u 1:(column(2)>0?column(2):0)
plot $data_with_semicolons u 1:(valid(2)?$2:0)
plot $data_with_semicolons u 1:(valid(2)?column(2):0)
#-----------------------------------------------------

Output:

#-----------------------------------------------------

	G N U P L O T
	Version 5.5 patchlevel 0    last modified 2020-06-24 

	Copyright (C) 1986-1993, 1998, 2004, 2007-2020
	Thomas Williams, Colin Kelley and many others

	gnuplot home:     http://www.gnuplot.info
	mailing list:     gnu...@li...
	faq, bugs, etc:   type "help FAQ"
	immediate help:   type "help"  (plot window: hit 'h')


# Curve 0 of 1, 2 points
# Curve title: "$data_with_whitespace u 1:($2>0?$2:0)"
# x y type
 1  1  i
 3  3  i


# Curve 0 of 1, 2 points
# Curve title: "$data_with_whitespace u 1:(column(2)>0?$2:0)"
# x y type
 1  1  i
 3  3  i


# Curve 0 of 1, 3 points
# Curve title: "$data_with_whitespace u 1:(column(2)>0?column(2):0)"
# x y type
 1  1  i
 2  NaN  u
 3  3  i


# Curve 0 of 1, 2 points
# Curve title: "$data_with_whitespace u 1:(valid(2)?$2:0)"
# x y type
 1  1  i
 3  3  i


# Curve 0 of 1, 3 points
# Curve title: "$data_with_whitespace u 1:(valid(2)?column(2):0)"
# x y type
 1  1  i
 2  0  i
 3  3  i


# Curve 0 of 1, 2 points
# Curve title: "$data_with_semicolons u 1:($2>0?$2:0)"
# x y type
 1  1  i
 3  3  i


# Curve 0 of 1, 2 points
# Curve title: "$data_with_semicolons u 1:(column(2)>0?$2:0)"
# x y type
 1  1  i
 3  3  i


# Curve 0 of 1, 3 points
# Curve title: "$data_with_semicolons u 1:(column(2)>0?column(2):0)"
# x y type
 1  1  i
 2  0  i
 3  3  i


# Curve 0 of 1, 2 points
# Curve title: "$data_with_semicolons u 1:(valid(2)?$2:0)"
# x y type
 1  1  i
 3  3  i


# Curve 0 of 1, 3 points
# Curve title: "$data_with_semicolons u 1:(valid(2)?column(2):0)"
# x y type
 1  1  i
 2  0  i
 3  3  i
#-------------------------------------------------------------------

Observations:

- it's as if the mere presence of a $X in the specification causes the
datum to be marked as invalid, and dropped entirely, if column X
doesn't contain data, no matter what the rest of the specification is.
- it depends on the datafile separator, see the NaN vs. 0 in the above
output.
- this was produced on a custom 5.5 build, but I've observed the same
on a distro-provided 5.4.x build too.

Is there any rational explanation for any of this? The documentation
doesn't seem to mention anything other than "The special symbols
 $1, $2, ... are shorthand for column(1), column(2) ...".

Bonus:
`reset` doesn't reset `set table`. The documentation of `reset` doesn't
mention that as an exception.

best regards,
Peter Juhasz