#290 wildcard & multiple using

closed-accepted
nobody
None
5
2006-10-22
2006-09-25
Janos Torok
No

Old and bad code, works only for linux but our group
uses it a lot. I hope someone rewrites it and includes
it in the distribution.

The purpose of this script is to facilitate the
command line plotting of many files with numerous
columns. (A general problem in science.)

Works for commands: plot, splot and replot

If filename contains * or ? the filename is passed to
the shell echo command to complete it. All returned
files are then plotted with the same arguments. Note
that shall also extends other wildcards like `[a-z]'
and naturally it can be used but in order to pass it
to the shell the filename must contain a star or a
question mark.

If the using field contains the `@' character and
numbers before and after (e.g. using 1:2@6) than the
plotting is modified and the plotting is executed with
all number between the given limits. For example:
> plot "a.dat" using 1:2@5
is equivalent to
> plot "a.dat" u 1:2,"a.dat" u 1:3,"a.dat" u
1:4,"a.dat" u 1:5
other example
> plot "a.dat" u 1:($2@3*2)
is equivalent to
> plot "a.dat" u 1:($2*2),"a.dat" u 1:($3*2)

One can olso specify steps by an other `@' parameter.
> plot "a.dat" u 1:2@5@2
is equivalent to
> plot "a.dat" u 1:2,"a.dat" u 1:4

Discussion

1 2 > >> (Page 1 of 2)
  • Ethan Merritt
    Ethan Merritt
    2006-09-25

    Logged In: YES
    user_id=235620

    Thank you for your patch.

    Much of this task can now be done natively in gnuplot. Only
    the key step of iteration within a plot command is missing.
    For example, to plot data from multiple files within a
    single plot command one would hypothetically do:

    # Get list of files
    list = system("ls -1 *.dat")
    # find out how many there are
    nplots = words(list)
    # define a function that returns the name of file i
    file(i) = words(list,i)
    # plot files 1 and 3 (only)
    plot file(1), file(3)
    #
    # This is the missing component!
    # No such command syntax currently exists,
    # but it could be added
    #
    plot {i=1,nplots} file(i) using 1:2 with linespoints
    #
    # The same formalism could be used to iterate over
    # fields within the same file:
    #
    plot {i=2,5} "data.dat" using 1:($i) with histograms

     
  • Petr Mikulik
    Petr Mikulik
    2006-09-27

    Logged In: YES
    user_id=31505

    I wonder why the following script produces "
    internal error : non-STRING argument"?

    # Get list of files
    list = system("ls -1 *.dat")
    # find out how many there are
    nfiles = words(list)
    # define a function that returns the name of file i
    file(i) = words(list,i)
    # plot files 1 and 3 (only)
    #plot file(1), file(3)

    # select files: from, step, to
    a=system(sprintf("seq -f 'file(%%g)' -s , %g %g %g", 1, 1,
    nfiles))

    set macros
    b="plot " . a
    #b="plot " . file(1)
    print b
    @b

     
  • Ethan Merritt
    Ethan Merritt
    2006-09-27

    Logged In: YES
    user_id=235620

    Petr:

    Because you have a typo in the definition of file(i).
    It should be
    file(i) = word(list,i)
    not
    file(i) = words(list,i)

     
  • Petr Mikulik
    Petr Mikulik
    2006-09-27

    Logged In: YES
    user_id=31505

    Well, now there is a problem: how to generate string
    file(1) title file(1), file(2) title file(2)
    ? The -f in seq allows only one %g. Use awk instead?

     
  • Ethan Merritt
    Ethan Merritt
    2006-09-28

    Logged In: YES
    user_id=235620

    Here is a relatively small and simple-minded patch that
    fills in the missing piece. It allows you to add an
    iteration clause to the "plot" command. You can iterate over
    anything, not just file names.

    set style data lines
    file(n) = sprintf("file_%d.dat",n)

    plot for [i=3,6] "file.dat" using 1:(column(i))
    plot for [k=1,99] file(k) using 1:2 title file(k)

    set style data histogram
    plot for [k=2,20] "datafile" using (column(k)) title col

    Notes:

    1) I'm not wild about the syntax, but this choice works
    without having to change the token parser

    2) Notice that the control variable can be named anything,
    and that it can appear multiple times in the plot command

    3) The is no support in this patch for multiple iterations
    within the same command. I.e., you cannot do
    plot for [i=1,2] file(i), for [j=6,7] file(j)

     
  • Ethan Merritt
    Ethan Merritt
    2006-09-29

    Logged In: YES
    user_id=235620

    In preparation for a code freeze and the run-up to a release of
    version 4.2, existing bugs and patchsets are being prioritized.

    This patchset is not on my (sfeam) list for inclusion in 4.2 and
    is therefore being marked as priority 2.

    Note that this does not mean it is a bad patch, or that it won't
    be incorporated into cvs after 4.2 is released. We can
    re-evaluate priorities after 4.2 is out.

    If you want to argue for immediate reconsideration - go right
    ahead; but do so quickly!

    Ethan Merritt

     
  • Ethan Merritt
    Ethan Merritt
    2006-09-29

    • priority: 5 --> 2
     
  • Ethan Merritt
    Ethan Merritt
    2006-09-30

    Logged In: YES
    user_id=235620

    This version is a bit cleaner and contains documentation. It
    allows one iteration specifaction per clause of the plot
    command.

     
  • Petr Mikulik
    Petr Mikulik
    2006-09-30

    • priority: 2 --> 5
     
  • Petr Mikulik
    Petr Mikulik
    2006-09-30

    Logged In: YES
    user_id=31505

    I like this patch very very much, something I really needed
    for ages!

    I propose this syntax for enumeration (matlab/octave
    compatible):

    plot for [i=1:10] ...
    plot for [i=1:2:10] ...

    and, if possible,
    plot for [i=3,5,8,13,21] ...

    I wish to have this patch for 4.2. Thanks!

     
  • Ethan Merritt
    Ethan Merritt
    2006-10-01

    Logged In: YES
    user_id=235620

    The 30-Sep-2006 version re-arranges the code so that it can
    be shared by 'plot' and 'splot'. Supporting splot required
    adding a field to the iso_curves structure.

    I thought about trying to support
    plot for [i=3,5,8,foo,baz,etc]
    but I'm not sure it's worth the effort to treat it as a
    special case. You can already do
    list = "3 5 8 ..."
    item(j) = word(list,j)
    plot for [j=1,n] file(item(j))
    That doesn't quite work for the foo, baz, etc entries however.

    I take it that for [i=1:2:10] means 1,3,5,7,9 ?
    Is that a typo for [i=1:10:2]? The languages that I can
    think of off the top of my head use the _third_ number as
    the increment, not the second.

     
  • Petr Mikulik
    Petr Mikulik
    2006-10-01

    Logged In: YES
    user_id=31505

    > I take it that for [i=1:2:10] means 1,3,5,7,9 ?

    Yes.

    > Is that a typo for [i=1:10:2]?

    No, 1:2:10 is correct in both Octave and Matlab:
    octave> 1:2:10
    ans =
    1 3 5 7 9

    > plot for [i=3,5,8,foo,baz,etc]
    > list = "3 5 8 ..."
    > item(j) = word(list,j)
    > plot for [j=1:words(list)] file(item(j))

    This is a good idea, please write it in docs. Its advantage is:
    list='5 6'; replot

     
  • Ethan Merritt
    Ethan Merritt
    2006-10-04

    Logged In: YES
    user_id=235620

    More cleanup; plug a memory leak; change syntax to
    plot for [i = start : end]
    This makes it look much like a normal plot range specifier.

    What is supposed to happen if one says
    plot for [i = 1 : -2]

    Syntax error?
    Execute once and stop?
    Don't execute at all?
    Iterate in reverse order: 1, 0, -1, -2 ?

     
  • Petr Mikulik
    Petr Mikulik
    2006-10-04

    Logged In: YES
    user_id=31505

    > plot for [i = 1 : -2]

    Don't plot at all. In order to use the reversed range, one
    has to write
    plot for [i=1:-1:-2]
    The error syntax could be: "empty range, no data or function
    to plot"

     
  • Mike Sutton
    Mike Sutton
    2006-10-11

    Logged In: YES
    user_id=722632

    I really like this patch. I have always wanted an easy way
    to plot multiple indexes from a single file with different
    line types (colors) without have to list each one
    individually.

    I think the proposed "plot for" syntax is good.

    For whats is worth, Fortran90 uses a start:end:incr style
    when accessing arrays.

     
  • Petr Mikulik
    Petr Mikulik
    2006-10-12

    Logged In: YES
    user_id=31505

    > Fortran90 uses a start:end:incr style

    I prefer [start:end] and [start:incr:end] to be compatible
    with Octave and Matlab.

     
  • Ethan Merritt
    Ethan Merritt
    2006-10-14

    Logged In: YES
    user_id=235620

    I think I would rather not add an increment at all. I'd
    rather provide the simple iterator that's already coded
    plot for [i=start:end]

    and a truly general iterator
    plot while [condition(i) next(i)] ...

    To be used as follows:

    next(i) = i+2
    condition(i) = (i<10) && (i != 7)

    plot i=3, while [condition(i) i=next(i)] ...

     
  • Petr Mikulik
    Petr Mikulik
    2006-10-14

    Logged In: YES
    user_id=31505

    Please add the increment. It is really a very very useful
    feature.

     
  • Logged In: YES
    user_id=1090807

    Gnuplot has longly been lacking the ability to plot a larger
    amount of columns using a simple plot command. Ethan, your
    iterator is really a good idea.

    In the docs, I think it should read 'item(i)' instead of
    'item[i]'.

    Juergen

     
  • Ethan Merritt
    Ethan Merritt
    2006-10-17

    Logged In: YES
    user_id=235620

    OK. This version adds an optional increment
    plot {for [i=<start>:<end>{:<incr>}} ...

    If (start > end) it will issue a warning but not actually
    skip the plot. I'd rather skip it, but to do so without
    side-effects is non-trivial. Possibly we can stuff some
    "don't draw me" flag in the plot header, but we'd still have
    to figure out a way to avoid reading any associated data and
    resetting the axis min/max ranges, etc.

     
  • Petr Mikulik
    Petr Mikulik
    2006-10-17

    Logged In: YES
    user_id=31505

    Few comments:

    * plot for [i=10:-10:-1] x+i title sprintf('x**%i',i)
    ... I think this should go in reversed order, but it does
    not. Could it go?

    * please write in 'help plot iterat' that the
    (a) start, end, inc are converted to integers (would it be
    worth to support real numbers?),
    (b) plot for the 1st iteration is always drawn (even for a
    "wrong" interval)

     
  • Ethan Merritt
    Ethan Merritt
    2006-10-19

    Logged In: YES
    user_id=235620

    The problem with gracefully handling missing/empty plots is
    now fixed [I think!] in cvs. See Bug 1579867.

    Once I sorted that out, it became much easier to handle an
    "empty" iteration by marking it NODATA; i.e. by pretending
    it was a plot of an empty file. I suspect that there may
    still be some side-effects on the auto-scaling code from the
    scanning of files being skipped over, but the simple cases
    now seem to work OK.

    E.g.
    plot sin(x), for [i=1:0] file(i), cos(x)
    will now only plot sin(x) and cos(x), will not complain
    about missing files, and will auto-scale as normal.

    The 3D case is less tested than the 2D case. Please report
    any problems.

     
  • plot-with-iteration code patch, docs, examples

     
  • Ethan Merritt
    Ethan Merritt
    2006-10-20

    Logged In: YES
    user_id=235620

    This version adds a separate demo "iterate.dem", and also
    uses the new syntax to good advantage in "histograms.dem".

    In the absence of late-breaking bug reports, I will put this
    patchset into cvs sometime this weekend.

    Special thanks to Janos Torok for inspiring this addition. I
    didn't use your actual code, but the example provided by
    your patch was key to motivating an equivalent more general
    implementation.

    As we gain experience with the simple form implemented here:
    plot for [i=<start> : <end> { : <increment>}]
    we can consider extending it to other forms of iteration. I
    still lean toward adding
    plot for [name in "A B C Q Y Z"] name.".dat" title name
    but as we discussed already, the equivalent is already
    possible using the somewhat longer version
    list = "A B C Q Y Z"
    item(i) = word(list,i)
    and then iterating over i.

     
1 2 > >> (Page 1 of 2)