Menu
▾
▴
gnuplot-beta
|
From: Dima K. <gn...@di...> - 2017-09-01 05:44:16
|
Hi. I just stumbled on another bug with rgbimages. If somebody knows what's causing it, that'd be appreciated. I'm plotting an image showing a map composed of stitched openstreetmap tiles (attached). The mapping between pixels in this image and lat/lon is linear in longitude and nonlinear in latitude. I'm plotting the image pixels onto x2y2, and using linked axes to get the latitude on y and longitude on x. The script is this: set x2tics auto set y2tics auto lon0 = -123.5614 lon_offset = 180.0 + lon0 zoom = 4. px_to_lon(x) = x/(256. * 2.**4)*360. - 180. + lon_offset lon_to_px(x) = (x-lon_offset+180.)/360. * 2.**zoom * 256. lat_offset_px = 1215.969 s1_cos(x) = (sin(x)+1)/cos(x) inv_s1_cos(x) = asin( (x**2 - 1)/(x**2 + 1) ) px_to_lat(x) = inv_s1_cos(exp((1 - (x + lat_offset_px)/(2.**zoom * 256) *2)*pi))*180./pi lat_to_px(x) = (1 - log( (sin(x*pi/180) + 1.)/cos(x*pi/180) )/pi)/2. * 2.**zoom * 256 - lat_offset_px set link x2 via lon_to_px(x) inverse px_to_lon(x) set link y2 via lat_to_px(y) inverse px_to_lat(y) plot [x=-120:-80][y=20:60] "montage_40_-99_1300miles_4.png" binary filetype=png flipy with rgbimage notitle axis x2y2, '-' with points axis x2y2 60 419 e Note that the circle representing Los Angeles lies at pixel coordinates 60,419 and I'm plotting a point there. The corresponding latlon is roughly -118,34. The pixels are on x2y2 and the point is on x2y2, so this point should be rendered in that circle. I'm not seeing that at all, however. The point is at y2=419 as it should be, but the Los Angeles circle is roughly at y2=448. The x coordinates match properly. Furthermore, zooming in makes it clear that gnuplot is confused: interactively zooming on the plotted point shows me the zoomed LA circle, even though visually the LA circle was elsewhere. Any pointers? |
|
From: Dima K. <gn...@di...> - 2017-09-01 05:44:48
Attachments:
montage_40_-99_1300miles_4.png
|
Forgot to attach the image |
|
From: sfeam <sf...@us...> - 2017-09-01 06:28:12
|
On Thursday, 31 August 2017 22:44:01 Dima Kogan wrote:
> Hi. I just stumbled on another bug with rgbimages. If somebody knows
> what's causing it, that'd be appreciated.
This is a very complicated one.
I'll look at it in more detail over the weekend.
For now I'll just note:
- The script basically does not work with 5.2 or current cvs, so I assume
you are using 5.0?
- To make the script work with 5.2 I can move the range outside the plot command:
set xrange [-120:-80]
set yrange [20:60]
plot "montage_40_-99_1300miles_4.png" binary filetype=png flipy ...
- At that point I see (I think) the same thing you do in that the single explicit
point lies North of rather than exactly on top of the Los Angeles marker.
- I'm not sure whether or not I see the same thing as you on zooming.
At first glance it looks to me that the zoomed-in placement is correct
(point is on top of LA) in the zoom view even though it was not in the
original full view. The closer in I zoom the closer to perfect the superposition
becomes. Is that what you see or do you see something else?
- So I guess my question is, would it be a correct description to say that
the plot is correct but only after a zoom operation? If so, that sounds
easier to track down than "it's always wrong".
- One possible problem occurs to me. The coordinates mapped to an
image pixel describe the center of the pixel, not a corner.
You may have constructed your mapping with that in mind, or maybe not.
If not then perhaps adjusting the mapping function will fix everything.
Ethan
> I'm plotting an image showing a map composed of stitched openstreetmap
> tiles (attached). The mapping between pixels in this image and lat/lon
> is linear in longitude and nonlinear in latitude. I'm plotting the image
> pixels onto x2y2, and using linked axes to get the latitude on y and
> longitude on x. The script is this:
>
> set x2tics auto
> set y2tics auto
>
> lon0 = -123.5614
> lon_offset = 180.0 + lon0
>
> zoom = 4.
> px_to_lon(x) = x/(256. * 2.**4)*360. - 180. + lon_offset
> lon_to_px(x) = (x-lon_offset+180.)/360. * 2.**zoom * 256.
>
> lat_offset_px = 1215.969
>
> s1_cos(x) = (sin(x)+1)/cos(x)
> inv_s1_cos(x) = asin( (x**2 - 1)/(x**2 + 1) )
>
> px_to_lat(x) = inv_s1_cos(exp((1 - (x + lat_offset_px)/(2.**zoom * 256) *2)*pi))*180./pi
> lat_to_px(x) = (1 - log( (sin(x*pi/180) + 1.)/cos(x*pi/180) )/pi)/2. * 2.**zoom * 256 - lat_offset_px
>
>
> set link x2 via lon_to_px(x) inverse px_to_lon(x)
> set link y2 via lat_to_px(y) inverse px_to_lat(y)
>
> plot [x=-120:-80][y=20:60] "montage_40_-99_1300miles_4.png" binary filetype=png flipy with rgbimage notitle axis x2y2, '-' with points axis x2y2
> 60 419
> e
>
>
> Note that the circle representing Los Angeles lies at pixel coordinates
> 60,419 and I'm plotting a point there. The corresponding latlon is
> roughly -118,34. The pixels are on x2y2 and the point is on x2y2, so
> this point should be rendered in that circle. I'm not seeing that at
> all, however. The point is at y2=419 as it should be, but the Los
> Angeles circle is roughly at y2=448. The x coordinates match properly.
>
> Furthermore, zooming in makes it clear that gnuplot is confused:
> interactively zooming on the plotted point shows me the zoomed LA
> circle, even though visually the LA circle was elsewhere.
>
> Any pointers?
|
|
From: Dima K. <gn...@di...> - 2017-09-01 06:51:46
|
sfeam <sf...@us...> writes: > On Thursday, 31 August 2017 22:44:01 Dima Kogan wrote: >> Hi. I just stumbled on another bug with rgbimages. If somebody knows >> what's causing it, that'd be appreciated. > > This is a very complicated one. > I'll look at it in more detail over the weekend. > For now I'll just note: > > - The script basically does not work with 5.2 or current cvs, so I assume > you are using 5.0? I was using "gnuplot 5.1 patchlevel 0", but it wasn't a release. It was built from the latest sources as of 2016/11/18. I just tried on the latest sources, and the problem is mostly the same. > - To make the script work with 5.2 I can move the range outside the plot command: > > set xrange [-120:-80] > set yrange [20:60] > plot "montage_40_-99_1300miles_4.png" binary filetype=png flipy ... Yep. I had to do this to make it work with the latest sources. Does that make sense? This smells like a bug too. > - I'm not sure whether or not I see the same thing as you on zooming. > At first glance it looks to me that the zoomed-in placement is correct > (point is on top of LA) in the zoom view even though it was not in the > original full view. The closer in I zoom the closer to perfect the superposition > becomes. Is that what you see or do you see something else? That's what I see too: I zoom on an area that does NOT have the circle in it (but that SHOULD have it), and I see the zoomed circle post-zoom. > - So I guess my question is, would it be a correct description to say that > the plot is correct but only after a zoom operation? If so, that sounds > easier to track down than "it's always wrong". I can't tell. There could be a systematic error, but this is a low-res map and doesn't have enough detail. > - One possible problem occurs to me. The coordinates mapped to an > image pixel describe the center of the pixel, not a corner. > You may have constructed your mapping with that in mind, or maybe not. > If not then perhaps adjusting the mapping function will fix everything. It's true that I wasn't super careful about this distinction, but I don't see 0.5 pixels of error, I see 35 pixels of error. I was hoping that this one wouldn't be tough to track down since one would expect the rgbimage to be plotted against y2 normally, and against y2 via a mapping. But here the NORMAL y2 coordinates are becoming broken. But admittedly I haven't spent much time in looking at the sources on this one. |
|
From: sfeam <sf...@us...> - 2017-09-01 14:48:12
|
On Thursday, 31 August 2017 23:51:37 Dima Kogan wrote:
> sfeam <sf...@us...> writes:
>
> > On Thursday, 31 August 2017 22:44:01 Dima Kogan wrote:
> >> Hi. I just stumbled on another bug with rgbimages. If somebody knows
> >> what's causing it, that'd be appreciated.
> >
> > This is a very complicated one.
> > I'll look at it in more detail over the weekend.
Never mind. I realized the problem overnight.
The "with image" family of plot styles (image rgbimage rgbalpha) use
a terminal-specific bitmap display routine if the terminal provides one.
This can be much faster than sending the pixels one-by-one.
But that only works correctly for linear coordinates, because the
only information sent is basically "fill this rectangle with this bitmap".
You have nonlinear axes so you need to fall back to calculating the
pixel coordinates individually. The keyword for this is "pixels".
So this morning's test is:
set xrange [-120:-80] noextend
set yrange [20:60] noextend
plot "montage_40_-99_1300miles_4.png" \
binary filetype=png flipy axes x2y2 with rgbimage pixels notitle, \
$POINT axes x2y2 with points pt 1 ps 2
That seems to work correctly (although slower).
I moved the "axes x2y2" to match what the documentation shows
but I didn't test whether that makes any difference by itself.
Probably not since the original command was accepted with no warning.
I put the point in a data block because I wanted to be able to
try various refresh/replot commands but again I didn't test whether
it makes any difference by itself. Probably not.
> > For now I'll just note:
> >
> > - The script basically does not work with 5.2 or current cvs, so I assume
> > you are using 5.0?
>
> I was using "gnuplot 5.1 patchlevel 0", but it wasn't a release. It was
> built from the latest sources as of 2016/11/18. I just tried on the
> latest sources, and the problem is mostly the same.
>
>
> > - To make the script work with 5.2 I can move the range outside the plot command:
> >
> > set xrange [-120:-80]
> > set yrange [20:60]
> > plot "montage_40_-99_1300miles_4.png" binary filetype=png flipy ...
>
> Yep. I had to do this to make it work with the latest sources. Does that
> make sense? This smells like a bug too.
Yeah. I was going to package up a final 5.2 release this
weekend, but now I'd like to pin that down first, if only to add it as a
"known problem" in the release notes.
Ethan
|
|
From: Dima K. <gn...@di...> - 2017-09-01 17:01:35
|
sfeam <sf...@us...> writes: > On Thursday, 31 August 2017 23:51:37 Dima Kogan wrote: >> sfeam <sf...@us...> writes: >> >> > On Thursday, 31 August 2017 22:44:01 Dima Kogan wrote: >> >> Hi. I just stumbled on another bug with rgbimages. If somebody knows >> >> what's causing it, that'd be appreciated. >> > >> > This is a very complicated one. >> > I'll look at it in more detail over the weekend. > > Never mind. I realized the problem overnight. > The "with image" family of plot styles (image rgbimage rgbalpha) use > a terminal-specific bitmap display routine if the terminal provides one. > This can be much faster than sending the pixels one-by-one. > But that only works correctly for linear coordinates, because the > only information sent is basically "fill this rectangle with this bitmap". > > You have nonlinear axes so you need to fall back to calculating the > pixel coordinates individually. The keyword for this is "pixels". > So this morning's test is: > > set xrange [-120:-80] noextend > set yrange [20:60] noextend > > plot "montage_40_-99_1300miles_4.png" \ > binary filetype=png flipy axes x2y2 with rgbimage pixels notitle, \ > $POINT axes x2y2 with points pt 1 ps 2 > > That seems to work correctly (although slower). Aha. That makes it work for me too. Thanks! Two questions: 1. Can we detect this failure and throw a warning, maybe? Or turn on "pixels" mode automatically? 2. Here I have pixels on x2y2 and latlon on xy. It also appears that the thing that gets drawn on the plot is linear in xy, so the image requires a nonlinear transformation (and thus we need the "pixels" option). How is it decided what is drawn on the plot? What if I wanted the drawn output to be linear in the pixels, but the rendered axes to be nonlinear? Is this a specifiable choice, or does gnuplot always pick the xy axes to render? Thanks |
|
From: Ethan A M. <sf...@us...> - 2017-09-01 19:53:53
|
On Friday, 01 September, 2017 10:01:26 Dima Kogan wrote:
> sfeam <sf...@us...> writes:
>
> > On Thursday, 31 August 2017 23:51:37 Dima Kogan wrote:
> >> sfeam <sf...@us...> writes:
> >>
> >> > On Thursday, 31 August 2017 22:44:01 Dima Kogan wrote:
> >> >> Hi. I just stumbled on another bug with rgbimages. If somebody knows
> >> >> what's causing it, that'd be appreciated.
> >> >
> >> > This is a very complicated one.
> >> > I'll look at it in more detail over the weekend.
> >
> > Never mind. I realized the problem overnight.
> > The "with image" family of plot styles (image rgbimage rgbalpha) use
> > a terminal-specific bitmap display routine if the terminal provides one.
> > This can be much faster than sending the pixels one-by-one.
> > But that only works correctly for linear coordinates, because the
> > only information sent is basically "fill this rectangle with this bitmap".
> >
> > You have nonlinear axes so you need to fall back to calculating the
> > pixel coordinates individually. The keyword for this is "pixels".
> > So this morning's test is:
> >
> > set xrange [-120:-80] noextend
> > set yrange [20:60] noextend
> >
> > plot "montage_40_-99_1300miles_4.png" \
> > binary filetype=png flipy axes x2y2 with rgbimage pixels notitle, \
> > $POINT axes x2y2 with points pt 1 ps 2
> >
> > That seems to work correctly (although slower).
>
> Aha. That makes it work for me too. Thanks! Two questions:
>
> 1. Can we detect this failure and throw a warning, maybe? Or turn on
> "pixels" mode automatically?
We can detect if the axis is mapped by "set link" or "set nonlinear",
but not whether the mapping is truly linear or nonlinear. If the
mapping is linear then the optimized bitmap transfer should still work.
So I don't think we want to fall back to pixels mode just because a
mapping exists.
> 2. Here I have pixels on x2y2 and latlon on xy. It also appears that the
> thing that gets drawn on the plot is linear in xy, so the image requires
> a nonlinear transformation (and thus we need the "pixels" option). How
> is it decided what is drawn on the plot? What if I wanted the drawn
> output to be linear in the pixels, but the rendered axes to be
> nonlinear? Is this a specifiable choice, or does gnuplot always pick the
> xy axes to render?
I don't understand the question.
When you specify coordinates for a drawable object or graph component
you have to tell the program what axial system the coordinates belong to.
See "help coordinates".
The x y x2 y2 axes are normally linear. You can change this by providing
forward and reverse mapping functions, either mapping onto a hidden linear
axis ("set nonlinear") or a paired visible axis ("set link").
In the current version of gnuplot you cannot use "set link" to make
both of the paired axes nonlinear; you would have to manually set up
mappings for both using "set nonlinear" and then set the corresponding
range endpoints for both.
Does that help?
Ethan
|
|
From: Dima K. <gn...@di...> - 2017-09-01 20:01:53
|
Dima Kogan <gn...@di...> writes: > 2. Here I have pixels on x2y2 and latlon on xy. It also appears that > the thing that gets drawn on the plot is linear in xy, so the image > requires a nonlinear transformation (and thus we need the "pixels" > option). How is it decided what is drawn on the plot? What if I wanted > the drawn output to be linear in the pixels, but the rendered axes to > be nonlinear? Is this a specifiable choice, or does gnuplot always > pick the xy axes to render? To (partly) answer my own question: the x1y1 axis is rendered to the screen. So if I take the gnuplot script in question and 1. switch the forward/backward transforms in the 'set link' commands 2. change the plot command to 'axis x1y1' from 'axis x2y2' 3. change the range specs from touching the x1,y1 axes to the x2,y2 axes then the rendered plots look fine even without the "pixels". This is because in this case, the pixels live on the x1y1 axis, which is then linear with the output terminal axes, and the y2 axis is then the nonlinear one, but this doesn't directly affect the rendering. This popped up another behavior that looks like a bug: 'set y2tics auto' doesn't work here, and even explicitly asking for y2tics positions doesn't give me the requested tics, but I want to experiment a bit more before saying much more about this one. |
|
From: Ethan A M. <sf...@us...> - 2017-09-01 21:32:13
|
On Friday, 01 September, 2017 13:01:40 Dima Kogan wrote: > Dima Kogan <gn...@di...> writes: > > > 2. Here I have pixels on x2y2 and latlon on xy. It also appears that > > the thing that gets drawn on the plot is linear in xy, so the image > > requires a nonlinear transformation (and thus we need the "pixels" > > option). How is it decided what is drawn on the plot? What if I wanted > > the drawn output to be linear in the pixels, but the rendered axes to > > be nonlinear? Is this a specifiable choice, or does gnuplot always > > pick the xy axes to render? > > To (partly) answer my own question: the x1y1 axis is rendered to the > screen. So if I take the gnuplot script in question and > > 1. switch the forward/backward transforms in the 'set link' commands > 2. change the plot command to 'axis x1y1' from 'axis x2y2' > 3. change the range specs from touching the x1,y1 axes to the x2,y2 axes > > then the rendered plots look fine even without the "pixels". This is > because in this case, the pixels live on the x1y1 axis, which is then > linear with the output terminal axes, and the y2 axis is then the > nonlinear one, but this doesn't directly affect the rendering. It may "look fine" but only one of the two treatments can be correct. One places the pixels on an evenly spaced grid, the other does not. The two rendered maps are different, although both now have a marker on top of LA. Both maps are distorted from what I am used to seeing so I cannot say which is intended. Ethan |
|
From: Dima K. <gn...@di...> - 2017-09-02 06:49:49
|
Ethan A Merritt <sf...@us...> writes: > On Friday, 01 September, 2017 13:01:40 Dima Kogan wrote: >> Dima Kogan <gn...@di...> writes: >> >> > 2. Here I have pixels on x2y2 and latlon on xy. It also appears that >> > the thing that gets drawn on the plot is linear in xy, so the image >> > requires a nonlinear transformation (and thus we need the "pixels" >> > option). How is it decided what is drawn on the plot? What if I wanted >> > the drawn output to be linear in the pixels, but the rendered axes to >> > be nonlinear? Is this a specifiable choice, or does gnuplot always >> > pick the xy axes to render? >> >> To (partly) answer my own question: the x1y1 axis is rendered to the >> screen. So if I take the gnuplot script in question and >> >> 1. switch the forward/backward transforms in the 'set link' commands >> 2. change the plot command to 'axis x1y1' from 'axis x2y2' >> 3. change the range specs from touching the x1,y1 axes to the x2,y2 axes >> >> then the rendered plots look fine even without the "pixels". This is >> because in this case, the pixels live on the x1y1 axis, which is then >> linear with the output terminal axes, and the y2 axis is then the >> nonlinear one, but this doesn't directly affect the rendering. > > It may "look fine" but only one of the two treatments can be correct. > One places the pixels on an evenly spaced grid, the other does not. > The two rendered maps are different, although both now have a marker > on top of LA. Both maps are distorted from what I am used to seeing > so I cannot say which is intended. I guess there are 3 sets of coordinates that are being related to one another: 1. Pixel coordinates in the rgbimage being plotted 2. Gnuplot x1/y1/x2/y2 coordinates 3. Terminal coordinates of the output I THINK gnuplot always maps x1 and y1 linearly into the terminal coordinates, NOT x2 and y2; please tell me if this is wrong. At the beginning of this thread I was plotting the rgbimage into x2y2, this was nonlinearly mapped to x1y1, which was in turn linearly mapped into the terminal coords. I.e. the bitmap was being nonlinearly warped. With the variation above, the rgbimage goes into x1y1, so the terminal output warps the image linearly-only, and the x1y2 axes instead show the nonlinearity. The distortion you're seeing is likely due to the linear scaling of the x1y1 axes. If you 'set size ratio -1' then you should see exactly the images you see at openstreetmap.org, which look "normal" to me since I've been looking at them for many years. Thanks for the help. |
|
From: sfeam <sf...@us...> - 2017-09-02 15:24:15
|
On Friday, 01 September 2017 23:49:39 Dima Kogan wrote:
>
> Ethan A Merritt <sf...@us...> writes:
>
> > On Friday, 01 September, 2017 13:01:40 Dima Kogan wrote:
> >> Dima Kogan <gn...@di...> writes:
> >>
> >> > 2. Here I have pixels on x2y2 and latlon on xy. It also appears that
> >> > the thing that gets drawn on the plot is linear in xy, so the image
> >> > requires a nonlinear transformation (and thus we need the "pixels"
> >> > option). How is it decided what is drawn on the plot? What if I wanted
> >> > the drawn output to be linear in the pixels, but the rendered axes to
> >> > be nonlinear? Is this a specifiable choice, or does gnuplot always
> >> > pick the xy axes to render?
> >>
> >> To (partly) answer my own question: the x1y1 axis is rendered to the
> >> screen. So if I take the gnuplot script in question and
> >>
> >> 1. switch the forward/backward transforms in the 'set link' commands
> >> 2. change the plot command to 'axis x1y1' from 'axis x2y2'
> >> 3. change the range specs from touching the x1,y1 axes to the x2,y2 axes
> >>
> >> then the rendered plots look fine even without the "pixels". This is
> >> because in this case, the pixels live on the x1y1 axis, which is then
> >> linear with the output terminal axes, and the y2 axis is then the
> >> nonlinear one, but this doesn't directly affect the rendering.
> >
> > It may "look fine" but only one of the two treatments can be correct.
> > One places the pixels on an evenly spaced grid, the other does not.
> > The two rendered maps are different, although both now have a marker
> > on top of LA. Both maps are distorted from what I am used to seeing
> > so I cannot say which is intended.
>
> I guess there are 3 sets of coordinates that are being related to one
> another:
>
> 1. Pixel coordinates in the rgbimage being plotted
> 2. Gnuplot x1/y1/x2/y2 coordinates
> 3. Terminal coordinates of the output
>
> I THINK gnuplot always maps x1 and y1 linearly into the terminal
> coordinates, NOT x2 and y2; please tell me if this is wrong.
The x1 and y1 axes may also be nonlinear, but it requires a command
like
set nonlinear x via f(x) inverse g(x)
The "set link" command will not by itself change the linearity of x1 or y1.
The x2 and y2 axes can also be made nonlinear by a "set link" command.
Ethan
> At the beginning of this thread I was plotting the rgbimage into x2y2,
> this was nonlinearly mapped to x1y1, which was in turn linearly mapped
> into the terminal coords. I.e. the bitmap was being nonlinearly warped.
>
> With the variation above, the rgbimage goes into x1y1, so the terminal
> output warps the image linearly-only, and the x1y2 axes instead show the
> nonlinearity.
>
> The distortion you're seeing is likely due to the linear scaling of the
> x1y1 axes. If you 'set size ratio -1' then you should see exactly the
> images you see at openstreetmap.org, which look "normal" to me since
> I've been looking at them for many years.
>
> Thanks for the help.
|
|
From: Dima K. <gn...@di...> - 2017-09-04 20:00:19
|
Thanks much for the help, Ethan. The end-product of this discussion: https://github.com/dkogan/osmgnuplot I found another bug: the check for an invalid axis-linking function was behaving poorly, and would be over-eager to report errors due to numerical inaccuracies. A patch (with further description of problem and solution): |
