|
From: Jon G. <jo...@th...> - 2014-01-30 16:25:33
|
Hi all!
When plotting phase data in gnuplot, you often encounter "jumps" in a
graph when a value wraps around 2π. A common way of dealing with this is
to apply an unwrap[1] function. What this function does is add or remove
2π from the next value to be plotted (v) as long as the difference
between v and the previous value (w) is outside the range [-π,π].
In pseudocode:
unwrap(v, w):
do
diff = v - w
v += 2π if diff < -π
v -= 2π if diff > π
while abs(diff) > π
It is also possible to implement a weighted unwrap which does the same
as the above, but instead of looking at the previous value w, it looks
at a weighted average of previous values instead. This works very well
for datasets with very low variance.
I'd love to implement this feature in gnuplot myself, but it's not
entirely clear to me how I would go about it. It seems like
standard.{c,h} would be where to start, but from what I can tell, these
functions only receive the current data point, and does not have any
knowledge of previous points. Adding a static variable to track the
previous value seems like an ugly solution, so I was hoping someone here
could point me in the right direction?
Cheers,
Jon
[1] http://www.mathworks.co.uk/help/matlab/ref/unwrap.html
|
|
From: <pl...@pi...> - 2014-01-30 17:32:38
|
On 01/30/14 15:00, Jon Gjengset wrote:
> Hi all!
>
> When plotting phase data in gnuplot, you often encounter "jumps" in a
> graph when a value wraps around 2π. A common way of dealing with this is
> to apply an unwrap[1] function. What this function does is add or remove
> 2π from the next value to be plotted (v) as long as the difference
> between v and the previous value (w) is outside the range [-π,π].
>
> In pseudocode:
>
> unwrap(v, w):
> do
> diff = v - w
> v += 2π if diff < -π
> v -= 2π if diff > π
> while abs(diff) > π
>
> It is also possible to implement a weighted unwrap which does the same
> as the above, but instead of looking at the previous value w, it looks
> at a weighted average of previous values instead. This works very well
> for datasets with very low variance.
>
> I'd love to implement this feature in gnuplot myself, but it's not
> entirely clear to me how I would go about it. It seems like
> standard.{c,h} would be where to start, but from what I can tell, these
> functions only receive the current data point, and does not have any
> knowledge of previous points. Adding a static variable to track the
> previous value seems like an ugly solution, so I was hoping someone here
> could point me in the right direction?
>
> Cheers,
> Jon
>
> [1] http://www.mathworks.co.uk/help/matlab/ref/unwrap.html
>
You should be able to implement it as a gnuplot function and call it
form plot:
as an example this will plot the integral.
integ(x)=(last=last+x)
last=0; plot datafile using 1(integ($2))
Post if you write it , this is something I'd thought of but never got
round to.
Peter.
|
|
From: Ethan A M. <sf...@us...> - 2014-01-30 17:55:05
|
On Thursday, 30 January, 2014 14:00:18 Jon Gjengset wrote: > Hi all! > > When plotting phase data in gnuplot, you often encounter "jumps" in a > graph when a value wraps around 2π. A common way of dealing with this is > to apply an unwrap[1] function. What this function does is add or remove > 2π from the next value to be plotted (v) as long as the difference > between v and the previous value (w) is outside the range [-π,π]. > > In pseudocode: > > unwrap(v, w): > do > diff = v - w > v += 2π if diff < -π > v -= 2π if diff > π > while abs(diff) > π > > It is also possible to implement a weighted unwrap which does the same > as the above, but instead of looking at the previous value w, it looks > at a weighted average of previous values instead. This works very well > for datasets with very low variance. > > I'd love to implement this feature in gnuplot myself, but it's not > entirely clear to me how I would go about it. Have a look at the running average demo: http://gnuplot.sourceforge.net/demo_4.6/running_avg.html Substituting an unwrap function for the avg5() function should do what you want. It would not require any changes to the executable code. One problem I see is that the resulting plot would not stable when zoomed. Each change in xrange would potentially shift the whole curve on y by some multiple of 2π. > It seems like > standard.{c,h} would be where to start, but from what I can tell, these > functions only receive the current data point, and does not have any > knowledge of previous points. Adding a static variable to track the > previous value seems like an ugly solution, so I was hoping someone here > could point me in the right direction? I don't think that adding a new function with a single argument is what you want. It seems to me this is more like a trivial smoothing operation, and the easiest fit into gnuplot's existing organization would be plot $DATA using 1:2 smooth unwrap with lines If you want to work on implementing this in the code, start by looking in plot2d.c for switch statements that handle SMOOTH_NONE or its alternatives. Ethan > Cheers, > Jon > > [1] http://www.mathworks.co.uk/help/matlab/ref/unwrap.html |
|
From: Jonathan T. <jt...@as...> - 2014-01-30 18:12:02
|
On Thu, Jan 30, 2014 at 02:00:18PM +0000, Jon Gjengset wrote:
> When plotting phase data in gnuplot, you often encounter "jumps" in a
> graph when a value wraps around 2??. A common way of dealing with this is
> to apply an unwrap[1] function. What this function does is add or remove
> 2pi from the next value to be plotted (v) as long as the difference
> between v and the previous value (w) is outside the range [-pi,pi].
[[...]]
Here is a perl script which does such an "unwrapping"; feel free to
use it. (I realise that this isn't quite what you asked for, which
was a gnuplot builtin function. But my perl script might be a useful
substitute for some use cases.)
--- begin perl script ---
#!/usr/bin/perl -w
# $Header: /home/jonathan/CVSROOT/src/misc/unwrap.phase,v 1.3 2006/12/19 14:33:47 jonathan Exp $
################################################################################
#
# unwrap.phase -- unwrap the phase of a time series of angles
# This program is copyright (C) 2006 by Jonathan Thornburg <jt...@ae...>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING. If not, write to
# the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
# Boston, MA 02111-1307, USA.
#
################################################################################
my $pi = 4.0*atan2(1.0,1.0);
my $default_period = 2.0*$pi;
my $default_input_column = 1;
################################################################################
my $help_msg = <<"EOF"; # expand variables
This program "unwraps" the phase of a time series of "angles". That is,
given a time series, this function adds an integer multiple of a specified
period to each value so as to minimize the total of the absolute differences
of the sums,
output_i := input_i + N_i*period
N_i chosen to minimize |output_i - output_{i-1}|
Usage:
unwrap.phase [ --period <number> ]
[ --input-column <integer> ]
[ --debug <integer> ]
<input >output
or
unwrap.phase --help # print this output
or
unwrap.phase --help-example # print this output
Any all-whitespace or comment lines in the input ('#' in column 1) are
passed through to the output unchanged. Otherwise, each input line is
split into whitespace-separated fields, and an angle is extracted. The
corresponding output line is a copy of the input line, followed by N_i
and output_i (tab-separated).
The --period optional argument (default 2*pi) specifies the period.
The --input-column optional argument (default ${default_input_column}) specifies the 1-origin
column containing the input angle.
EOF
################################################################################
my $help_example_msg = <<'EOF';
% cat unwrap.phase.test-data
# column 1 = input
# column 2 = desired output
0.0 0.0
0.3 0.3
0.6 0.6
0.9 0.9
0.2 1.2
0.5 1.5
0.8 1.8
0.1 2.1
0.4 2.4
0.7 2.7
0.0 3.0
0.9 2.9
0.5 2.5
0.1 2.1
0.7 1.7
0.3 1.3
0.9 0.9
0.5 0.5
0.1 0.1
0.7 -0.3
0.3 -0.7
0.9 -1.1
0.1 -0.9
% unwrap.phase --period 1 <unwrap.phase.test-data
# column 1 = input
# column 2 = desired output
0.0 0.0 0 0
0.3 0.3 0 0.3
0.6 0.6 0 0.6
0.9 0.9 0 0.9
0.2 1.2 1 1.2
0.5 1.5 1 1.5
0.8 1.8 1 1.8
0.1 2.1 2 2.1
0.4 2.4 2 2.4
0.7 2.7 2 2.7
0.0 3.0 3 3
0.9 2.9 2 2.9
0.5 2.5 2 2.5
0.1 2.1 2 2.1
0.7 1.7 1 1.7
0.3 1.3 1 1.3
0.9 0.9 0 0.9
0.5 0.5 0 0.5
0.1 0.1 0 0.1
0.7 -0.3 -1 -0.3
0.3 -0.7 -1 -0.7
0.9 -1.1 -2 -1.1
0.1 -0.9 -1 -0.9
EOF
################################################################################
use strict;
use Getopt::Long;
my $true = 1;
my $false = 0;
my $fuzz = 1.0e-10;
my $period = $default_period;
my $input_column = $default_input_column;
my $debug = 0;
my $help_flag = $false;
my $help_example_flag = $false;
$Getopt::Long::autoabbrev = $false; # forbid abbreviations
$Getopt::Long::getopt_compat = $false; # forbid starting options with "+"
GetOptions(
'period=f' => \$period,
'input-column=i' => \$input_column,
'debug=i' => \$debug,
'help' => \$help_flag,
'help-example' => \$help_example_flag,
)
|| die $help_msg;
if ($help_flag)
{ print $help_msg; exit 0; }
if ($help_example_flag)
{ print $help_example_msg; exit 0; }
if ($period == 0.0)
{ die "period must be nonzero!\n"; }
if ($debug > 0)
{
print "using period=${period}\n";
}
if ($debug == 99)
{
for (my $x = -1.6 ; fuzzy_LE($x,1.6) ; $x += 0.1)
{
print $x, "\t", round_to_nearest_int($x), "\n";
}
exit 0;
}
# perl uses 0-origin fields internally
$input_column -= 1;
my $previous_output = undef();
my $input_line_number = 0;
while (my $line = <STDIN>)
{
++$input_line_number;
if (($line =~ /^\s$/) || ($line =~ /^#/))
{ print $line; next; } # *** LOOP CONTROL ***
chomp $line;
my @fields = split(/\s+/, $line);
my $input = $fields[$input_column];
if (! defined($input))
{ die "couldn't find input field in input line ${input_line_number}!\n"; }
$input += 0.0; # make sure it's numeric
my $N;
if (defined($previous_output))
{
# choose $N to minimize absolute difference
# |$output - $previous_output|
# where $output is computed below
my $Nf = ($previous_output - $input) / $period;
$N = round_to_nearest_int($Nf);
if ($debug > 5)
{ print "input=${input} ==> Nf=${Nf} N=${N}\n"; }
}
else {
$N = 0;
}
if ($debug > 0)
{ print "input=${input} ==> N=${N}\n"; }
my $output = $input + $N*$period;
print $line, "\t", $N, "\t", $output, "\n";
$previous_output = $output;
}
################################################################################
#
# This function rounds a floating-point number to the nearest integer.
#
# Arguments:
# $x = The number to round.
#
sub round_to_nearest_int
{
my ($x) = @_;
# remaining code only handles values >= 0
if ($x < 0) { return -round_to_nearest_int(-$x); }
return int(0.5+$x);
}
################################################################################
#
# These functions do fuzzy comparisons of real numbers, returning
# Boolean values $true or $false.
#
# Global variables:
# $fuzz
#
sub fuzzy_EQ
{
my ($x,$y) = @_;
return (abs($x-$y) <= $fuzz) ? $true : $false;
}
sub fuzzy_LE
{
my ($x,$y) = @_;
return (($x < $y) || fuzzy_EQ($x,$y)) ? $true : $false;
}
--- end perl script ---
ciao,
--
-- "Jonathan Thornburg [remove -animal to reply]" <jt...@as...>
Dept of Astronomy & IUCSS, Indiana University, Bloomington, Indiana, USA
"There was of course no way of knowing whether you were being watched
at any given moment. How often, or on what system, the Thought Police
plugged in on any individual wire was guesswork. It was even conceivable
that they watched everybody all the time." -- George Orwell, "1984"
|
|
From: Jon G. <jo...@th...> - 2014-01-30 17:45:37
|
> Here is a perl script which does such an "unwrapping"; feel free to > use it. Thanks. This is what I'm currently doing as well, but I thought it would be nice to have a built-in way of achieving this. After all, it is a pretty common operation on phase data. Jon PS: Sorry for sending this to you twice Jonathan, but it seems I replied only to you the first time. |
|
From: Jon G. <jo...@th...> - 2014-01-30 18:25:32
|
> Have a look at the running average demo: > > http://gnuplot.sourceforge.net/demo_4.6/running_avg.html > > Substituting an unwrap function for the avg5() function should do what > you want. It would not require any changes to the executable code. Ah, yes, that should work nicely for now. > One problem I see is that the resulting plot would not stable when > zoomed. Each change in xrange would potentially shift the whole curve > on y by some multiple of 2π. This would certainly be true for continuous functions, but for data you could potentially always unwrap the entire dataset and ignore the xrange. I'm actually not entirely sure what you would expect as a user if you zoomed in a generated plot with unwrapping enabled... > I don't think that adding a new function with a single argument is > what you want. It seems to me this is more like a trivial smoothing > operation, and the easiest fit into gnuplot's existing organization > would be I agree, it does seem to fit more naturally as a smoothing operation. In fact, if you have phase data, that is exactly what it does. > If you want to work on implementing this in the code, start by looking > in plot2d.c for switch statements that handle SMOOTH_NONE or its > alternatives. I'll have a look. Is such a change something you would consider merging? Jon |
|
From: Ethan A M. <sf...@us...> - 2014-01-30 18:40:45
|
On Thursday, 30 January, 2014 17:30:32 Jon Gjengset wrote: > > Have a look at the running average demo: > > > > http://gnuplot.sourceforge.net/demo_4.6/running_avg.html > > > > Substituting an unwrap function for the avg5() function should do what > > you want. It would not require any changes to the executable code. > > Ah, yes, that should work nicely for now. > > > One problem I see is that the resulting plot would not stable when > > zoomed. Each change in xrange would potentially shift the whole curve > > on y by some multiple of 2π. > > This would certainly be true for continuous functions, but for data you > could potentially always unwrap the entire dataset and ignore the > xrange. > > I'm actually not entirely sure what you would expect as a user if you > zoomed in a generated plot with unwrapping enabled... > > > I don't think that adding a new function with a single argument is > > what you want. It seems to me this is more like a trivial smoothing > > operation, and the easiest fit into gnuplot's existing organization > > would be > > I agree, it does seem to fit more naturally as a smoothing operation. In > fact, if you have phase data, that is exactly what it does. > > > If you want to work on implementing this in the code, start by looking > > in plot2d.c for switch statements that handle SMOOTH_NONE or its > > alternatives. > > I'll have a look. > Is such a change something you would consider merging? Yes. The actual smoothing code will probably only require a few lines of C. The trickier part may be deciding whether or not it needs to be reexecuted on replot/refresh/zoom/unzoom. Ethan |
|
From: Jon G. <jo...@th...> - 2014-01-30 19:17:46
|
> Yes. > The actual smoothing code will probably only require a few lines of C. I've attached two patches, one that merges cleanly against cvs, and one that works with 4.6 (minor change, but might come in useful). I have also attached a sample data file with real phase measurements, and a plot file that plots this data with and without unwrapping. > The trickier part may be deciding whether or not it needs to be > reexecuted on replot/refresh/zoom/unzoom. The patch only implements unwrapping for datasets, and not for continuous functions (I'm not entirely sure why it has no effect for functions; it is not an intentional limitation from my side). It seems to work well with zoom/unzoom/pan/replot as far as I can see. I'd appreciate it if someone else could try this with their datasets too and see that it's not doing anything too silly. I could also implement a weighted unwrapper if anyone feels there's a need for it, but I think basic unwrap will get us quite far. Cheers, Jon |
|
From: Ethan A M. <sf...@us...> - 2014-01-30 20:52:47
|
On Thursday, 30 January, 2014 19:17:36 Jon Gjengset wrote:
> > Yes.
> > The actual smoothing code will probably only require a few lines of C.
>
> I've attached two patches, one that merges cleanly against cvs, and one
> that works with 4.6 (minor change, but might come in useful).
>
> I have also attached a sample data file with real phase measurements,
> and a plot file that plots this data with and without unwrapping.
>
> > The trickier part may be deciding whether or not it needs to be
> > reexecuted on replot/refresh/zoom/unzoom.
>
> The patch only implements unwrapping for datasets, and not for
> continuous functions (I'm not entirely sure why it has no effect for
> functions; it is not an intentional limitation from my side).
To use smoothing with a function you need to modify your plot command
so that the function is treated as sampled data:
plot 'phase.log' using 1:(atan2($3,$2)) w lines ls 1 t 'Wrapped' \
,'phase.log' using 1:(atan2($3,$2)) w lines ls 3 t 'Unwrapped' smooth unwrap \
,'+' using 1:(x-(floor(x/6.28)*6.28)-3.14) ls 4 t 'Wrapped func' \
,'+' using 1:(x-(floor(x/6.28)*6.28)-3.14) ls 5 t 'Unwrapped func' smooth unwrap \
> It seems to work well with zoom/unzoom/pan/replot as far as I can see.
>
> I'd appreciate it if someone else could try this with their datasets too
> and see that it's not doing anything too silly.
>
> I could also implement a weighted unwrapper if anyone feels there's a
> need for it, but I think basic unwrap will get us quite far.
>
> Cheers,
> Jon
|
|
From: Jon G. <jo...@th...> - 2014-01-30 22:24:08
|
> > The patch only implements unwrapping for datasets, and not for
> > continuous functions (I'm not entirely sure why it has no effect for
> > functions; it is not an intentional limitation from my side).
>
> To use smoothing with a function you need to modify your plot command
> so that the function is treated as sampled data:
>
> plot 'phase.log' using 1:(atan2($3,$2)) w lines ls 1 t 'Wrapped' \
> ,'phase.log' using 1:(atan2($3,$2)) w lines ls 3 t 'Unwrapped' smooth unwrap \
> ,'+' using 1:(x-(floor(x/6.28)*6.28)-3.14) ls 4 t 'Wrapped func' \
> ,'+' using 1:(x-(floor(x/6.28)*6.28)-3.14) ls 5 t 'Unwrapped func' smooth unwrap \
s/x/$1/
Well, the good news is that it wraps the values correctly.
The bad news is that, as you expected, the plot jumps around when you
zoom and pan.
To be honest, I don't quite know what would be expected behavior for
unwrap for continuous function though. When you change the xrange, the
unwrapping *should* change as the first point changes; that's kind of
the point of unwrap.
I can think of a couple of ways of dealing with this, but none of them
are really ideal:
- Always unwrap from x=0, but unwrap in both directions:
Tricky to implement correctly.
Problematic when the xrange is far from zero.
Could be a massive problem for functions that have branch cuts
around the origin.
- Disallow unwrapping for sampled functions:
I'm not even sure if this would be possible?
Users might want the ability to unwrap functions...
- Make no special arrangements for sampled functions:
Very odd behavior for the user if they don't know what to expect.
Simple to implement, and does "what the user asked" even though
that may not be what they expected.
- Add a "start unwrap at" parameter to the unwrap smoothing operation:
Even trickier to implement.
Complicates syntax.
Adds another source for plotting confusion for users.
Personally, I think what makes most sense is to not deal with plotting
sampled functions as a special case. Unwrap will work "as advertised",
and users who know what unwrap is supposed to do are likely to
understand what is going on when they zoom/pan.
You could also argue that unwrap doesn't even really make sense for
functions, because in many cases you could just rewrite the function so
that it doesn't wrap. I think it is relatively rare in practice for
people to write functions that *do* wrap; it is usually a phenomenon
seen in sampled data.
One separate note about the patch is that the line:
lasty = M_PI;
should possibly be changed to
lasty = 0;
The former makes sense for a signal that usually varies from 0 to 2*pi,
whereas the latter makes more sense for signals that vary from -pi to
pi. In practice it will make little difference beyond the y-intercept of
the unwrapped line.
|
|
From: Ethan A M. <sf...@us...> - 2014-01-30 23:12:53
|
On Thursday, 30 January, 2014 22:23:55 Jon Gjengset wrote: > > > > To use smoothing with a function you need to modify your plot command > > so that the function is treated as sampled data: > > > > plot 'phase.log' using 1:(atan2($3,$2)) w lines ls 1 t 'Wrapped' \ > > ,'phase.log' using 1:(atan2($3,$2)) w lines ls 3 t 'Unwrapped' smooth unwrap \ > > ,'+' using 1:(x-(floor(x/6.28)*6.28)-3.14) ls 4 t 'Wrapped func' \ > > ,'+' using 1:(x-(floor(x/6.28)*6.28)-3.14) ls 5 t 'Unwrapped func' smooth unwrap \ > > s/x/$1/ Hmm. "x" works for me. Unlesss "set parametric" is active, in which case "t" works. But $1 is also correct. > Personally, I think what makes most sense is to not deal with plotting > sampled functions as a special case. Unwrap will work "as advertised", > and users who know what unwrap is supposed to do are likely to > understand what is going on when they zoom/pan. Agreed. I was originally worried that the unwrapping might apply only to the data inside a selected zoom box, in which case it would suffer the same shift of origin as you see for sampled functions. But your code reevaluates the entire set of input data rather than just the currently INRANGE data, so that problem is avoided. > One separate note about the patch is that the line: > > lasty = M_PI; > > should possibly be changed to > > lasty = 0; > > The former makes sense for a signal that usually varies from 0 to 2*pi, > whereas the latter makes more sense for signals that vary from -pi to > pi. In practice it will make little difference beyond the y-intercept of > the unwrapped line. Origin at 0 seems more obvious in the absence of data. I have no idea which convention is more common for real world data sets. Could you write up brief section for the documentation/help file? Ethan |
|
From: Jon G. <jo...@th...> - 2014-01-30 23:52:22
Attachments:
unwrap.patch
|
> > s/x/$1/ > > Hmm. > "x" works for me. Unlesss "set parametric" is active, in which case "t" works. > But $1 is also correct. Ah, yes, it works for me with CVS as well. It wasn't working when I tried my patch on 4.6. > > Personally, I think what makes most sense is to not deal with > > plotting sampled functions as a special case. > > Agreed. I was originally worried that the unwrapping might apply only > to the data inside a selected zoom box, in which case it would suffer > the same shift of origin as you see for sampled functions. But your > code reevaluates the entire set of input data rather than just the > currently INRANGE data, so that problem is avoided. Yes, I thought it best to do it that way precisely to avoid the data jumping around too much. I suppose it might make the operation more costly for large datasets, but considering unwrapping is a fairly straightforward operation, I'm not too concerned. > > One separate note about the patch is that the line: > > lasty = M_PI; > > should possibly be changed to > > lasty = 0; > > Origin at 0 seems more obvious in the absence of data. > I have no idea which convention is more common for real world data sets. I've changed it to 0 in the patch now. Both conventions ([0,2π) and (-π,π]) are common in the real world. Wikipedia also points this out when they discuss unwrapping with regard to instantaneous phase[1]; they say "When phi(t), is constrained to its principal value, either the interval (-π, π] or [0, 2π), it is called the wrapped phase." I think sticking to 0 is marginally better because it will help move the y-intercept closer to the x-axis, but that's about it. Choosing 0 also means we don't have to justify choosing π. > Could you write up brief section for the documentation/help file? Done. Full patch against CVS attached. |
|
From: Ethan A M. <sf...@us...> - 2014-01-31 20:09:56
|
On Thursday, 30 January, 2014 23:52:12 Jon Gjengset wrote: > > Could you write up brief section for the documentation/help file? > > Done. > Full patch against CVS attached. Applied. So gnuplot 4.6.5 will have the new option "smooth unwrap", and it is already in the development version if you build from CVS. Ethan |
|
From: Jon G. <jo...@th...> - 2014-01-31 20:39:53
|
> So gnuplot 4.6.5 will have the new option "smooth unwrap", > and it is already in the development version if you build from CVS. Brilliant! Thanks for the quick turnaround and a great piece of software. Cheers, Jon |
|
From: <pl...@pi...> - 2014-02-01 08:12:29
|
On 01/31/14 21:39, Jon Gjengset wrote: >> So gnuplot 4.6.5 will have the new option "smooth unwrap", >> and it is already in the development version if you build from CVS. > > Brilliant! > Thanks for the quick turnaround and a great piece of software. > > Cheers, > Jon > Indeed. Thanks for your contribution Jon. Thanks, of course to Ethan, who is always very responsive to getting good ideas included into CVS. regards, Peter. |