gnuplot development

>> no, x = $1, y = $2 (autogenerated row and column index of
>> the matrix)
>
> Apparently that is how the program currently interprets
> $1 and $2.  I dind't know that.
> It is very confusing because it is not consistent with all other
> plot/splot "datafile" commands. In all other cases $1 means
> column 1 of the file, regardless of whether it is being used as
> an x value.

It's not confusing, because you specify the "matrix" keyword. Same for 
(the new) "binary" stuff.

> But I would like to provide a more logical syntax for this
> operation, so that this confusion isn't necessary.

>  splot 'matrix.dat' matrix using (1+$1/100):(1+$2*10):3
> as
>  f(x) = 1 + x/100
>  g(y) = 1 + y*10
>  splot 'matrix.dat' matrix xticlabels(f(x)):yticlabels(g(y))
>
> Would that be more natural?
>
> Alternatively, we could allow dummy variables in the using specs
> directly:
>  splot 'matrix.dat' matrix using (f(x)):(g(y)):(z)
> Unfortunately that has no obvious meaning for non-matrix plots.

I would stay with my proposal
     splot 'matrix.dat' matrix using (1+$1/100):(1+$2*10):3
For me, it's easy to understand, and consistent with "binary".

---
PM

I think I've tracked the bug down; it seems to be in datafile.c: 
df_readbinary(double v[], int max).

Use this script:

set grid
set ticslevel 0.3
set auto fix
set xlabel "xxxx"
set ylabel "yyyy"
splot 'a.dat' matrix u 1:2:3
set table
replot

with this a.dat:

11  12  13  14  15
21  22	23  24  25


==> it produces:

#IsoCurve 0, 5 points
#x y z type
  0  1  21 i
  1  1  22 i
  2  1  23 i
  3  1  0 i
  4  1  0 i

#IsoCurve 1, 5 points
#x y z type
  0  0  11 i
  1  0  12 i
  2  0  13 i
  3  0  0 i
  4  0  0 i

Thus, it reads only 3 columns, not the full line.


It looks that df_readbinary() is used also for reading the ascii matrix 
file. However, I couldn't found where is the problem. Who else can find the 
problem?

---
PM

On Saturday 22 October 2005 03:02 pm, Petr Mikulik wrote:
> I think I've tracked the bug down; it seems to be in datafile.c:
> df_readbinary(double v[], int max).

Remember that this bug was already present in version 4.0,
and in version 3.7 for that matter.  The earlier versions did
not use df_readbinary for input of ascii matrix data.

> Who else can find the problem?

I have had no luck so far.
My prime suspect is get_3ddata() in plot3d.c, but I have not
found it there either.

-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

On Saturday 22 October 2005 02:55 pm, Petr Mikulik wrote:

>>>  x = $1, y = $2 (autogenerated row and column index of the matrix)

>> It is very confusing because it is not consistent with all other
>> plot/splot "datafile" commands. In all other cases $1 means
>> column 1 of the file, regardless of whether it is being used as
>> an x value.

> It's not confusing, because you specify the "matrix" keyword. Same for
> (the new) "binary" stuff.

You don't find it confusing that in 2D plots you need to say
   plot 'data' matrix using ($1):($3)

Why ($3)?


-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

On Saturday 22 October 2005 03:02 pm, Petr Mikulik wrote:
> I think I've tracked the bug down

Found it. Bleah. It really has been there since forever.
It's in df_tokenise(), and it is specific to ascii matrix data.
Here is the fix:

--- gnuplot/src/datafile.c      2005-10-21 22:50:13.000000000 -0700
+++ gnuplot-cvs/src/datafile.c  2005-10-22 21:10:08.276728128 -0700
@@ -1145,6 +1146,7 @@
             if (df_matrix) { duplication=TRUE; break; }
             df_matrix = TRUE;
 #endif
+           fast_columns = 0;
             continue;
     }

Corey Satten's monster optimization test skips all columns past the
first three unless some test has previously cleared the fast_columns
flag.  Examples of things that clear the flag are parentheses in the
using spec and the presence of the "binary" flag. The optimization
test is also skipped if there are no using specs at all.
Ascii matrix data is not covered by any of these exceptions, so
df_tokenise() only parses the first 3 columns.

Amazing that no one has complained in all the time since 3.7 came
out.

-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

> You don't find it confusing that in 2D plots you need to say
>   plot 'data' matrix using ($1):($3)
>
> Why ($3)?

I never know I can use
 	plot ... matrix
I'm still learning gnuplot :-))

It would be nice to organize that
 	plot 'a.dat' matrix
is equivalent to
 	plot 'a.dat' matrix using 1:3

***

Proposal for 'help matrix', please modify:

Datafile can be in an ascii or binary matrix format. The `matrix` flag 
indicates that the file is ascii, the `binary` or `matrix binary` stands for 
a binary format. For details, see `matrix ascii` or `matrix binary`.

Basic usage in `splot`:
      splot 'a.dat' matrix
      splot 'a.gpbin' {matrix} binary
Advanced usage in `splot`:
      splot 'a.dat' matrix using 1:2:3
      splot 'a.gpbin' {matrix} binary using 1:2:3
allows to manipulate the axes coordinates and the z-data.

Usage in `plot`:
      plot `a.dat` matrix using 1:3
      plot 'a.gpbin' {matrix} binary using 1:3
will plot rows of the matrix, while using 2:3 will plot matrix columns, and 
1:2 point coordinates. Applying the `every` option you can specify explicit
rows and columns.

Example -- rescale axes for an ascii matrix:
      splot `a.dat` matrix using (1+$1):(1+$2*10):3

Example -- plot the 3rd row of an ascii matrix:
      plot 'a.dat' matrix using 1:3 every 1:999:1:2
(rows are enumerated from 0, thus 2 instead of 3).

***

Proposal for 'help matrix ascii', please modify:

The `matrix` flag in
        {s}plot 'a.dat' matrix
indicates that the ASCII data are stored in matrix format.

The z-values are read in a row at a time, i. e.,
        z11 z12 z13 z14 ...
        z21 z22 z23 z24 ...
        z31 z32 z33 z34 ...
and so forth.

X- and y-indices of the matrix correspond to column and row index of the 
matrix, starting from 0. You can rescale or transform the axes as usual for 
a data file with three columns (x=$1, y=$2, z=$3), for example
        splot 'a.dat' matrix using (1+$1/100):(1+$2*10):3

A blank line or comment line ends the matrix, and starts a new
surface mesh. You can select among the meshes inside a file by the
`index` option to the `splot` command, as usual.


---
PM

On Saturday 22 October 2005 20:42 Ethan A Merritt wrote:
> How about if we could plot Petr's example:
>   splot 'matrix.dat' matrix using (1+$1/100):(1+$2*10):3
> as
>   f(x) = 1 + x/100
>   g(y) = 1 + y*10
>   splot 'matrix.dat' matrix xticlabels(f(x)):yticlabels(g(y))
>
> Would that be more natural?

With this, I would expect one label for each row and each column.  I
think it would be more natural to introduce some

set format x function f(x)

where f(x) is a string valued expression with a dummy variable "x".

But in the case of matrix, I prefer Petr's example.


Juergen

On Sunday 23 October 2005 12:23 am, Petr Mikulik wrote:
>
> It would be nice to organize that
>  	plot 'a.dat' matrix
> is equivalent to
>  	plot 'a.dat' matrix using 1:3

OK. That part is now in cvs.
I'm still amazed that this works.
Who knew that you could plot data rows?

I'm still confused by the documentation, and will try to 
clarify it. For example, you suggest

> The z-values are read in a row at a time, i. e.,
>         z11 z12 z13 z14 ...
>         z21 z22 z23 z24 ...
>         z31 z32 z33 z34 ...
> X- and y-indices of the matrix correspond to column and row index of the

But this is not really correct, as the use of "plot <foo> matrix"
demonstrates.  "Column 3" may or may not be a Z value, depending 
on what kind of graph you are describing. My current best effort
at writing a description is:

  Matrix values are read in a row at a time, i. e.,
          M11 M12 M13 M14 ...   (i=1)
          M21 M22 M23 M24 ...   (i=2)
          M31 M32 M33 M34 ...   (i=3)
          ... ... ... ... Mij

  The interpretation of columns in the "using" specification for
  matrix data is different from the usual case. "$1" refers to the
  matrix row index i, "$2" refers to the matrix column index j,
  and "$3" refers to the matrix entry Mij.

  Example:
    set view map
    splot <foo> matrix using ($1 * 5.):2:(sqrt($3))

  This example will plot the value sqrt(Mij) at the coordinates
  x=5i, y=j for each entry i,j represented in the matrix.

Does this description seem OK to you?  


> ***
>
> Proposal for 'help matrix', please modify:
>
> Datafile can be in an ascii or binary matrix format. The `matrix` flag
> indicates that the file is ascii, the `binary` or `matrix binary` stands
> for a binary format. For details, see `matrix ascii` or `matrix binary`.
>
> Basic usage in `splot`:
>       splot 'a.dat' matrix
>       splot 'a.gpbin' {matrix} binary
> Advanced usage in `splot`:
>       splot 'a.dat' matrix using 1:2:3
>       splot 'a.gpbin' {matrix} binary using 1:2:3
> allows to manipulate the axes coordinates and the z-data.
>
> Usage in `plot`:
>       plot `a.dat` matrix using 1:3
>       plot 'a.gpbin' {matrix} binary using 1:3
> will plot rows of the matrix, while using 2:3 will plot matrix columns,
> and 1:2 point coordinates. Applying the `every` option you can specify
> explicit rows and columns.
>
> Example -- rescale axes for an ascii matrix:
>       splot `a.dat` matrix using (1+$1):(1+$2*10):3
>
> Example -- plot the 3rd row of an ascii matrix:
>       plot 'a.dat' matrix using 1:3 every 1:999:1:2
> (rows are enumerated from 0, thus 2 instead of 3).
>
> ***
>
> Proposal for 'help matrix ascii', please modify:
>
> The `matrix` flag in
>         {s}plot 'a.dat' matrix
> indicates that the ASCII data are stored in matrix format.
>
> The z-values are read in a row at a time, i. e.,
>         z11 z12 z13 z14 ...
>         z21 z22 z23 z24 ...
>         z31 z32 z33 z34 ...
> and so forth.
>
> X- and y-indices of the matrix correspond to column and row index of the
> matrix, starting from 0. You can rescale or transform the axes as usual
> for a data file with three columns (x=$1, y=$2, z=$3), for example
>         splot 'a.dat' matrix using (1+$1/100):(1+$2*10):3
>
> A blank line or comment line ends the matrix, and starts a new
> surface mesh. You can select among the meshes inside a file by the
> `index` option to the `splot` command, as usual.
>
>
> ---
> PM

-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

> I'm still confused by the documentation, and will try to
> clarify it. For example, you suggest
>
>> X- and y-indices of the matrix correspond to column and row index of the
>
> But this is not really correct, as the use of "plot <foo> matrix"
> demonstrates.  "Column 3" may or may not be a Z value, depending
> on what kind of graph you are describing.

I would keep the x, y and z for both plot and splot cases, and add that
the default using for plot is
    plot 'a.dat' matrix using 1:3
and not using 1:2 which makes no good sense for matrices.

>   Matrix values are read in a row at a time, i. e.,
>           M11 M12 M13 M14 ...   (i=1)
>           M21 M22 M23 M24 ...   (i=2)
>           M31 M32 M33 M34 ...   (i=3)
>           ... ... ... ... Mij
>
>   The interpretation of columns in the "using" specification for
>   matrix data is different from the usual case. "$1" refers to the
>   matrix row index i, "$2" refers to the matrix column index j,
>   and "$3" refers to the matrix entry Mij.

Indexing starts from 0. Further, x is the column index j, y is row index i.

> Does this description seem OK to you?

Please combine it with my longer description with example for both 'help
matrix' and 'help matrix ascii'.

---
PM

On Sunday 23 October 2005 10:38 pm, mikulik@... wrote:
> > "Column 3" may or may not be a Z value, depending
> > on what kind of graph you are describing.
>
> I would keep the x, y and z for both plot and splot cases, and add that
> the default using for plot is
>     plot 'a.dat' matrix using 1:3
> and not using 1:2 which makes no good sense for matrices.

Sorry, I must be really dense today.
I am just not understanding your way of naming things.

In a 2D plot we are plotting y = f(x).
x is taken from the first using spec.
y is taken from the second using spec.
There is no z.

So when you say   "plot 'a.dat' matrix using 1:3"
you are asking for x from column 1 and y from column 3.
How can it be correct to document column 3 as being z
in this case?


> >   Matrix values are read in a row at a time, i. e.,
> >           M11 M12 M13 M14 ...   (i=1)
> >           M21 M22 M23 M24 ...   (i=2)
> >           M31 M32 M33 M34 ...   (i=3)
> >           ... ... ... ... Mij
> >
> >   The interpretation of columns in the "using" specification for
> >   matrix data is different from the usual case. "$1" refers to the
> >   matrix row index i, "$2" refers to the matrix column index j,
> >   and "$3" refers to the matrix entry Mij.
>
> Indexing starts from 0. 

Oops. Yes.

> Further, x is the column index j, y is row index i.

Yeah, I'm not doing so well in describing this either.
And on top of that we may be suffering from a difference in
naming conventions.  I am now uncertain whether the term
"row index" means the same thing to you that it does to me.
And even if it does, there are people in the world that use
a different convention so maybe it is better not to use the
terms at all.

I agree that the x value of the plot is taken from the index j.

But neither i nor j is "y" in the sense of a 2D graph coordinate.
The y value is take from Mij.

If there are multiple lines in the data file, then i will be
counting out successive plots in the 2D graph. So it is neither
"x" nor "y", but rather it is "plot number".

Maybe we need to write totally separate sections for the 2D case
and the 3D case.

Anyhow, I don't seem to be having much luck at writing this
section. I'll hand it back to you.
Or maybe somebody else wants to take a stab at it?

-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742

Ethan A Merritt wrote:
> On Saturday 22 October 2005 02:55 pm, Petr Mikulik wrote:
> 
> 
>>>> x = $1, y = $2 (autogenerated row and column index of the matrix)

That's indeed at least slightly strange.  It boils down to two 
questions, which are strongly inter-connected:

1) Why not use existing pseudo-column numbers 0 and -1, instead of 1 and 2?

2) Should

1 2 3
4 5 6
7 8 9

read as 'matrix' be equivalent to a file

1
2
3

4
5
6

7
8
9

read by the ordinary routines, or to

0 0 1
1 0 2
2 0 3

0 1 4
1 1 5
2 1 6

0 2 7
1 2 8
2 2 9

The current implementation seems to be implying the latter format, which 
  in turn makes column numbers 1, 2 a suitable choice.  It effectively 
gives x values $1 = $0, and y vales $2 = column(-1).

> You don't find it confusing that in 2D plots you need to say
>    plot 'data' matrix using ($1):($3)

Well, given the current documentation (as in: lack of) what 'matrix' 
actually means in terms of column numbers, any expectations would be 
wrong by default.

Thus my suggestion is: we make up our mind about the answers to the 
above questions, and add to 'help matrix' accordingly.

>>>>> x = $1, y = $2 (autogenerated row and column index of the matrix)
>
> That's indeed at least slightly strange.  It boils down to two questions, 
> which are strongly inter-connected:
>
> 1) Why not use existing pseudo-column numbers 0 and -1, instead of 1 and 2?

I think $1, $2 and $3 is logic for this matrix case.

> 2) Should
>
> 1 2 3
> 4 5 6
> 7 8 9
>
> read by the ordinary routines, or to
>
> 0 0 1
> 1 0 2
> 2 0 3

yes, to this one, as it now works -- just the docs is missing

>> You don't find it confusing that in 2D plots you need to say
>>    plot 'data' matrix using ($1):($3)

No, because you "plot" (in 2D) a 3D object (matrix with axes). That's the 
same as for "plot ... with image".

> Thus my suggestion is: we make up our mind about the answers to the above 
> questions, and add to 'help matrix' accordingly.

That's what I meant when sending proposal for the new text of 'help matrix' 
and 'help matrix ascii'. Please update it if  you like.

---
PM

>> I would keep the x, y and z for both plot and splot cases, and add that
>> the default using for plot is
>>     plot 'a.dat' matrix using 1:3
>> and not using 1:2 which makes no good sense for matrices.
>
> Sorry, I must be really dense today.
> I am just not understanding your way of naming things.
>
> In a 2D plot we are plotting y = f(x).
> x is taken from the first using spec.
> y is taken from the second using spec.
> There is no z.
>
> So when you say   "plot 'a.dat' matrix using 1:3"
> you are asking for x from column 1 and y from column 3.
> How can it be correct to document column 3 as being z
> in this case?

I consider matrix as a 3D object, and I draw its section in 2D.

See 'help binary general' and 'help binary general examples' -- it's the 
same thing, just binary not ascii, and also plot as (x,y,z) via "plot" in 
2D.

> Maybe we need to write totally separate sections for the 2D case
> and the 3D case.

I tried that in my proposal text for the two help sections, with examples.

> section. I'll hand it back to you.
> Or maybe somebody else wants to take a stab at it?

At the moment I stay with the proposed text.

---
PM