gnuplot-info

[Gnuplot-info] transpose data with gnuplot

[theozh <th...@gm...>]

sometimes, data curves are available in rows and since gnuplot requires data in columns you would have to transpose your data, correct?

Of course, you can transpose your data with external tools (Python, awk, ...), however, in some cases it might be easier to do this in gnuplot.
I am not aware of a function in gnuplot which plots rows or a function which transposes data.
So, I created the script below.
I think it is neither very elegant nor efficient, but is seems to work.
However, if you have to load the data from a file and transpose it this file needs to be loaded "Rows x Cols" times for every datapoint. Is it like that or will the file be kept in memory? This will be problematic if you have to load the data from a slow network drive.
If anybody has a better solution please let me know.

The script below will 1) create some dummy data 2) print the data 3) transpose it 4) print the transposed data

### transpose data
reset session

# create some dummy data
Rows = 5
Cols = 7
set print $Data 
LINE = sprintf("%gx%g\t",Rows,Cols)
# create header columns
do for [j=1:Cols] {
    LINE = LINE.sprintf("x[%g]",j)
    if (j<Cols) {LINE = LINE."\t"}
}
print LINE
do for [i=1:Rows] {
    LINE = ""
    do for [j=1:Cols] {
        LINE = LINE.sprintf("%g", (i-1)*Cols + j)
        if (j<Cols) {LINE = LINE."\t"}
    }
    LINE = sprintf("t[%g]\t",i).LINE
    print LINE
}
set print
print $Data 
# end dummy data

# transpose data
set autoscale    # autoscale before stats
stats $Data u 0 nooutput
set table $Dummy
set print $Transposed
do for [i=1:STATS_columns] {
    LINE = ""
    do for [j=0:STATS_records-1] {
       plot $Data u (a=stringcolumn(i),i) every ::j::j with table
       LINE = LINE.sprintf("%s", a)
       if (j < STATS_records-1) {LINE = LINE."\t"}
    }
    print LINE
}
set print
unset table
print $Transposed
# end transpose data

Re: [Gnuplot-info] transpose data with gnuplot

[Ethan A M. <eam...@gm...>]

On Wednesday, 07 November 2018 08:13:40 theozh wrote:
> sometimes, data curves are available in rows and since gnuplot requires data in columns you would have to transpose your data, correct?
> 
> Of course, you can transpose your data with external tools (Python, awk, ...), however, in some cases it might be easier to do this in gnuplot.
> I am not aware of a function in gnuplot which plots rows or a function which transposes data.

Obscure commands
================

plot nth row of data in a file
        plot 'rowdata' matrix using 1:0 every :::n::n
     
store row n to an array, then plot it
        array row_n[10]
        stats 'rowdata' matrix using 1:(row_n[$1+1]=$0,$0) every :::n::n
        plot row_n

These commands arose unintentionally out of the way "every" interacts with
"matrix". So far as I know they are undocumented, but the behavior is not
likely to change.

    cheers,

		Ethan


> So, I created the script below.
> I think it is neither very elegant nor efficient, but is seems to work.
> However, if you have to load the data from a file and transpose it this file needs to be loaded "Rows x Cols" times for every datapoint. Is it like that or will the file be kept in memory? This will be problematic if you have to load the data from a slow network drive.
> If anybody has a better solution please let me know.
> 
> The script below will 1) create some dummy data 2) print the data 3) transpose it 4) print the transposed data
> 
> ### transpose data
> reset session
> 
> # create some dummy data
> Rows = 5
> Cols = 7
> set print $Data 
> LINE = sprintf("%gx%g\t",Rows,Cols)
> # create header columns
> do for [j=1:Cols] {
>     LINE = LINE.sprintf("x[%g]",j)
>     if (j<Cols) {LINE = LINE."\t"}
> }
> print LINE
> do for [i=1:Rows] {
>     LINE = ""
>     do for [j=1:Cols] {
>         LINE = LINE.sprintf("%g", (i-1)*Cols + j)
>         if (j<Cols) {LINE = LINE."\t"}
>     }
>     LINE = sprintf("t[%g]\t",i).LINE
>     print LINE
> }
> set print
> print $Data 
> # end dummy data
> 
> # transpose data
> set autoscale    # autoscale before stats
> stats $Data u 0 nooutput
> set table $Dummy
> set print $Transposed
> do for [i=1:STATS_columns] {
>     LINE = ""
>     do for [j=0:STATS_records-1] {
>        plot $Data u (a=stringcolumn(i),i) every ::j::j with table
>        LINE = LINE.sprintf("%s", a)
>        if (j < STATS_records-1) {LINE = LINE."\t"}
>     }
>     print LINE
> }
> set print
> unset table
> print $Transposed
> # end transpose data

Re: [Gnuplot-info] transpose data with gnuplot

[theozh <th...@gm...>]

Thank you, Ethan
very interesting construct!
For certain types of data it works.

However, what I couldn't yet get to work is how to plot row data of the following type:

$Data <<EOD
# spectral data as function of time in rows
Wav 400 500 600 700 800 
t01 1.1 1.2 1.3 1.4 1.5
t02 2.1 2.2 2.3 2.4 2.5
t03 3.1 3.2 3.3 3.4 3.5
EOD

If the data was transposed, it would be very easy to plot.

$Data <<EOD
# spectra in columns
Wav t01 t02 t03 
400 1.1 2.1 3.1
500 1.2 2.2 3.2
600 1.3 2.3 3.3
700 1.4 2.4 3.4
800 1.5 2.5 3.5
EOD

plot for [i=2:4] $Data u 1:i w lp


So, a handy transpose function would be very helpful.
Theo.

Re: [Gnuplot-info] transpose data with gnuplot

[Ethan M. <eam...@gm...>]

On Thu, Nov 8, 2018 at 12:17 PM theozh <th...@gm...> wrote:

> Thank you, Ethan
> very interesting construct!
> For certain types of data it works.
>
> However, what I couldn't yet get to work is how to plot row data of the
> following type:
>
> $Data <<EOD
> # spectral data as function of time in rows
> Wav 400 500 600 700 800
> t01 1.1 1.2 1.3 1.4 1.5
> t02 2.1 2.2 2.3 2.4 2.5
> t03 3.1 3.2 3.3 3.4 3.5
> EOD
>

I have scripts that I use to plot *.csv files similar to this that are
generated
by a piece of equipment in the lab.
I agree that it would be nice to have an automatic "transpose" button or
command since opening up each *.csv file to do it in Excel is an annoying
detour.   My gnuplot scripts, although ugly, can handle the raw files.

Your case is a little worse because the first column is non-numeric.
Nevertheless here is a modified version of one of my scripts that plots
your data by rows.  Unfortunately the non-numeric 1st column causes it to
print warning messages.
Ugly? You bet.  But it works, except for extracting labels from the row
headers.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 $Data <<EOD
 # spectral data as function of time in rows
 Wav 400 500 600 700 800
 t01 1.1 1.2 1.3 1.4 1.5
 t02 2.1 2.2 2.3 2.4 2.5
 t03 3.1 3.2 3.3 3.4 3.5
 EOD

 array xval[10]
 stats $Data matrix using 1:(xval[$1+1]=$0,$0) every :::0::0  nooutput
 cols = STATS_records
 rows = |$Data| - 2
 print cols, " X values: ", xval
 set xrange [xval[2] : xval[cols]]
 show xrange
 set style data linespoints
 plot for [row=1:rows] $Data matrix using (xval[$1+1]):($3) every
:::row::row title sprintf("Row %d",row)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%