#419 Images do not affect hidden3d
An image don't seem to obscure the hidden portion of lines when it should.
This demo from image2.dem illustrates:
set title "Simple extension of a two dimensional image into three dimensions"
set xrange [-10:137]
set yrange [-10:157]
set cbrange [0:255]
splot 'blutux.rgb' binary array=(128,128) flip=y format='%uchar%uchar%uchar' using ($1+$2+$3)/3 with image
Use the X11 window for its speed to get a better understanding of behavior. As entered above, at first it seems like hidden3d is active because the image covers the lines of the axes. However, I think that might just be the image shown last. Rotate the 3D plot and flip the Z axis to see that some lines can appear above the image.
Now turn on hidden lines
set hidden3d
replot
Notice that the X11 redraw slows down noticeably--which I'm taking as an indicator that hidden 3D is active. However, the axis lines appear on top of the image now when they should be broken to pieces, some of which discarded (i.e., the hidden pieces).
I've tried finding this one, but didn't have much time. Is there a setting for the plot object that is overlooked--one that indicates whether the image should be able to hide or obscure? Are the axes lines not able to be hidden in some type of mode?
hidden3d originally only affected surfaces
As per the documentation:
As of gnuplot version 4.6, hidden3d also affects 3D plotting styles
points,labels,vectors, andimpulseseven if no surface is present in the graph.Other plotting styles, including image and rgbimage, are ignored by the hidden3d code. In some cases you can dummy it up by describing a plane surface that lies exactly on (or just behind) the projected image. Then the hidden3d calculation is done for the surface but just happens to be approximately correct for the image as well.
Oh.
Well, it shouldn't be too difficult to make hidden3d work for an image, at least to some degree, because the image is actually a surface in 3D (i.e., MxN tiny pixels)...unless the viewing plane coincides with the image plane in which case the faster image is used. So, I think a small mod should make this work reasonably well.
There is the issue of the image changing from a "surface" to an "image" based upon viewing angle, but in the long run that might still work out as far as artifacts. That is, if the image plane matches the viewing plane, and the image is treated as one large rectangle in the hidden3d algorithm it seems to me there is no depth to the image so it can't cause any strange artifacts like what is now seen with the painter's algorithm for simulated hidden 3D.
Please leave this report open for now, and I will take a look at this when I get a chance sometime down the road.
It might be sufficient to add the hidden3d awareness only to the fallback one-pixel-at-a-time code "with image {pixels}". That code really does make a separate call for each pixel in all orientations, even if the optimized image code wouldn't.
I'm pretty sure it would be quite a bit more involved than that. Teaching plot_image_or_update_axes() about hidden3d would be by far the smaller job. The real job would be teaching hidden3d about objects that aren't (composed of) just lines. That's been what kept us from integrating hidden3d with pm3d, and it'll be what blocks this idea of integrating it with 'image' mode just as effectively.
Well, I think the idea is to get something for images that is on par with current pm3d behavior as a reasonable simple mod. For example,
set border 4095
splot (x+y) with pm3d
does seem to effectively hide axis lines, if one rotates the image some. (There are some "upside down" angles that create strange effects.) Maybe it is more work than I'm imagining, but I'll take a look.
I'd much rather add surface triangulation and intersecting triangle decomposition to hidden3d, but it's just too much work at the moment. The hard work is list/object management inside hidden3d for the new elements.
Ticket moved from /p/gnuplot/bugs/1601/
Can't be converted:
Attached is a patch that will add a phantom, four-point quadrangle outlining the edges of an 'image' or 'rgbimage' so that it hides. It's quite simple; basically if there is an image in the list of surface point plots, the code will create a new surface point plot with LT_NODRAW line type (i.e., invisible) and place it just after the image surface point plot. The new plot is made to be two isolines with two points per line. Thus, the phantom hides what is behind it but can't be seen. I also force the image to then opt out of hidden3d analysis, which would save otherwise wasted CPU.
This behaves as expected for me, and I've attached a few plots illustrating the behavior. I tested "nohidden3d with image" and got this result:
gnuplot> load 'fewpix_set_hidden3d_nohidden3d.gp'
"fewpix_set_hidden3d_nohidden3d.gp", line 74: All edges undefined or out of range, thus no plot.
which is simply because there is nothing in the plot that will hide. (I'd sort of prefer that gnuplot just draw the plot having no hiding objects rather than give an error.) To make the plot work I then added a simple surface to the plot and the result is in file Screenshot-set_hidden3d_nohidden3d.png. The image object is, in fact, not processed in the hidden3d algorithm as expected. The axes lines are still visible but the added surface is "hidden" by the image object because it is drawn before the image. So, the "nohidden3d" option is doing what it should. (Also, is there a long-standing bug that the hidden lines at the edges of a plot are not handled correctly? Hence the jagged appearance of the new surface? I seem to recall that.)
So, that's the patch. Really only a couple new hunks of code and then a bunch of format cleanup or function renaming to reflect.
Some general comments on the hidden3d code...
EXTRA DOCUMENTATION NEEDED? (Probably not)
" In contrast to the usual display in gnuplot, hidden line removal actually
treats the given function or data grids as real surfaces that can't be seen
through, so plot elements behind the surface will be hidden by it. For this
to work, the surface needs to have 'grid structure' (see
splot datafileabout this), and it has to be drawn
with linesorwith linespoints."This is true, I suppose, but not generally the case. I.e., the hidden3d algorithm isn't restricted to working with a grid. What I just added is testament to this. I just created the quadrangle as the outline of the image and it appears to work fine in the hidden3d algorithm.
Should I add some kind of notice in the hidden3d documentation that images now work? I'd prefer not to, simply because in some sense the image object in hidden3d behaves just as expected given the documentation description already present. That is, images will hide objects behind them, and can be excluded with the "nohidden3d" option.
FUTURE ORGANIZATION
There is another way that I could have done this. I could have changed the surface-plot definition of the IMAGE and RGBIMAGE internal memory so that the first group of data looks exactly like the way the new phantom is defined. Since it is only the "process_image()" function that utilizes the image data for drawing, I could have then placed the image data in the surface point data after the phantom image. In other words, when allocating the memory, make it large enough to contain the phantom isocurves and the image data after it. However, I thought this would be much too confusing for some other programmer to follow. But I would say if moving forward the plot3d/hidden3d code were reorganized slightly it might make sense for the surface_points structure to contain the isometric description of the object as it currently does along with extra data. The reason is that going forward, hidden3d lines and surfaces would be a little more sophisticated. IMAGE and RGBIMAGE are two examples where the overall shape is more important than the (possibly voluminous) data within the object. There could be more. This ramification is increased speed, if internally more complex objects are described simple geometric representation.
That being said, I'd like to point out the difference here between normal "image" mode and the "failsafe" option added some versions ago. For the hidden3d algorithm, the attached patch adds a construct that treats the image as a quadrangle. However, that really can't be done for "failsafe" mode, which treats the image as made up of many individual quadrangle pixels. By doing so, "failsafe" can treat NaN values as invisible or whatnot. If there are some pixels within the image that do not hide, the whole image can't be treated overall as a quadrangle. I'm not against "failsafe", just that I think it better to categorize that with its own style type (e.g., PIXMAP, RGBPIXMAP). It's conceptually much easier to separate the two ideas, and it also lends itself to underlying organization as far coding. Basically, IMAGE attempts to utilize the terminal's image capabilities first, and fall's back if it can't. On the other hand, PIXMAP would never attempt to use the terminal's image capabilities.
WHAT TO DO ABOUT ALPHA BLENDING?
And that brings me to an interesting quandary for hidden3d. Notice in this patch that I have left RGBA_IMAGE out of the mix. I figure that the user has a better chance of getting something alpha blended by having RGBA_IMAGE not hide lines. And this is just something for you to think about with regards to integrating surface objects into hidden3d down the road, be it line removal, triangulizing surfaces, etc. How would alpha blending be treated in hidden3d? If hidden lines and hidden surfaces are removed and the object that is obscuring them is an alpha blend, then the result won't be correct because the obscured line really should be blended.
Plot of an image using "set hidden3d"...
Plot of an image using "unset hidden3d"...
Plot of an image using "set hidden3d" and "nohidden3d with image"...
Looks promising. I'll run through some testing.
But I have a couple of questions.
1) I don't understand your comments about hidden3d processing of image plots taking extra time, and related to that it looks like the use of the opt_out_of_hidden3d flag is exactly backwards. So far as I know, up until now image plots were ignored by hidden3d processing so opting in or out is irrelevant and no extra time should be consumed. Am I missing something there? Is there a bug in the current code? I.e. up till now the plot was passed to hidden3d but ignored (I thought) once there. Should we not be passing it at all?
But now your patch does cause the hidden3d code to have an effect, but it sets the opt_out_of_hidden3d flag exactly for those plots which are now being processed by hidden3d. Isn't that backwards? We want to be able to set that flag in the command line so that they are indeed not processed.
2) Why is the code in process_image(..., IMG_UPDATE_CORNERS) needed at all? Not that it's all that much code, but isn't it just manually recalculating the same thing that the 3D rotation code will automatically do based on the corners of your dummy rectangle? Or maybe it's only relevant to 2D plots?
Answering one of my own questions...
Currently the hidden3d doesn't truly ignore image plots. It falls through to the default (POINTSTYLE) case. I doubt that does anything useful, but it does burn some wasted cycles. Let's fix that and proceed from there. Patch shown here and attached below.
Thanks. Yes, I recall now that the image was using the default hidden3d handling, and when hidden3d was turned on when just an image plotted it had slowed down--something had to be happening there. The patch makes sense to me. I'd say apply it. I'll then adjust the first patch because I also want to remove this comment from the patch:
I was thinking this evening the creation of the phantom is best just where it is. There's a reason sp_alloc() isn't made global. Calling it for every replot would slow things down and possibly be a memory leak. Best to use it only once on the creation of the plot constituents rather than when displaying the plot.
The reason for the process_image(..., IMG_UPDATE_CORNERS) is pretty much what you pointed out. Just use the corners, and what's more with the image the data points are pixel centers, so really the quadrangle phantom outline has to be expanded outward just a tad. That is why I tested using a plot with a few number of pixels so that it's clear that clipping is right at the outer edge of the image, not at the center of its outermost pixels. (Could have defined things differently, I know...can maybe revisit that some day.)
Using just the corners is for benefit of speed in the hidden3d algorithm. We know if we send an image to a terminal, the whole thing is displayed, not just a portion. So, as I see it, the four point quadrangle covers it. Even though the image routine falls back on small pixels for non-orthogonal views, I still like to think of that as a whole image because it may be that some day we come across a terminal that handles 3D projection of images (OpenGL?). This is sort of why I like the idea of "pixmap" (i.e., failsafe) being it's own style. In that case it would be as you described, no phantom quadrangle would be necessary because that would be taken care of automatically by the pixmap elements themselves (some possibly invisible or missing, e.g. NaN).
Oh yeah, opt-out. What I did was pass the command line setting onto the phantom and then set the IMAGE/RGBIMAGE opt-out to TRUE. Just the phantom needs the capability to do hidden3d or nohidden3d. That was the idea anyway.
I get that the corners of the phantom rectangle are slightly displaced from the nominal corners of the image to allow for the pixel size. But I still don't understand why they ever have to be updated as the plot is redrawn or rotated. If they are calculated correctly when the phantom is first created, isn't that sufficient?
FWIW when I comment out that UPDATE_CORNERS code I see no change at all in the test case I'm using. Is there something in particular I should test to understand what it does?
Oh - and 1 actual bug from testing. If I do
set hidden3d
splot FOO with image nohidden, BAZ with lines
It seems to have the effect of turning off the processing of BAZ, not FOO.
Ah, good point. It's not the projected points that is used with the UPDATE_CORNERS algorithm, so once is enough. I'll keep that in mind.
"FWIW when I comment out that UPDATE_CORNERS code I see no change at all in the test case I'm using."
Your example probably has the image drawn last so that looks like it is hiding lines (well, it is, but the hidden3d algorithm might not be causing an effect). I think there are some combinations of options that have the axes drawn last. Some views, perhaps? Or maybe the order that elements are plotted (image first in the list?). I'll see if I can put together a demo with unset/set hidden3d side by side in a multiplot.
Sorry, I didn't describe that clearly. I commented out the loop in do_3dplot which calls process_image(this_plot, IMG_UPDATE_CORNERS) every time the screen view is refreshed, and replaced it with a one-time call made immediately after the phantom rectangle plot is allocated and initialized. I don't see any difference in the result, which is what I would expect.
I've applied the "ignore images in hidden3d" patch, and added a demo/test case to hidden2.dem
Your hidden3d demo looks good. Before and after the patch makes clear that the image is hiding as is expected. I will create another simple demo this weekend.
"
If I do
set hidden3d
splot FOO with image nohidden, BAZ with lines
It seems to have the effect of turning off the processing of BAZ, not FOO.
"
Actually, I think that example is as expected. If one looks at the lines of the axis, say,
set border 4095;
set border lw 0.500000;
set title "Simple extension of a two dimensional image into three dimensions"
set xrange [-10:137]
set yrange [-10:157]
set cbrange [0:255]
set hidden3d
splot 'blutux.rgb' binary array=(128,128) flip=y format='%uchar%uchar%uchar' using ($1+$2+$3)/3 with image nohidden3d, x+y with lines
one can tell it is the surface that is hiding the axis lines not the image. But the image is plotted last in all viewing angles. Here is what the documentation indicates:
"
To mix pm3d surfaces with normal
hidden3dprocessing, use the optionset hidden3d frontto force all elements included in hidden3d processing tobe drawn after any remaining plot elements, including the pm3d surface.
"
So, the segmented lines are either drawn first or last. Presumably the hidden3d code doesn't do depth ordering on the lines, but drawing the lines last (i.e., set hidden3d front) effectively accomplishes that. Turn on 'set hidden3d front' and replot and the plot looks correct, similar to your hidden2.dem example. Applying depth sorting to the processed line segments actually would be an easier sub-problem of the whole hidden3d feature (compared to hidden surfaces), but the "front" option works OK.
I've run into a bug when trying to do a multiplot demo just changing "unset hidden3d" to "set hidden3d" to "set hidden3d back" to "set hidden3d front". The first plot of multiplot is fine, but the second then fails:
Hidden line behavior for images acts similar to pm3d behavior. The surface of the image may hide lines depending upon the configuration of hidden3d. Warning: empty z range [0:0], adjusting to [-1:1] Warning: empty z range [0:0], adjusting to [-1:1] "image2.dem", line 214: nextfrom_dynarray: dynarray wasn't initialized!I'll have to work on this later this evening, but I'll try finishing the patch update sometime tonight.
Attached is the most recent patch. Mainly, there is a new demo in 'image2.dem' that shows the hidden3d behavior for the various settings. I'm indifferent about its necessity--given there is a working example in 'hidden2.dem' and the hidden3d behavior is pretty consistent with pm3d and other hiding objects--but it does cover things and might avoid some confusion on part of the user.
Really, I'm not even sure why hidden3d has the option "back" and why "back" is the default. It seems to me that users will only ever want the "front" option if they are choosing hidden3d. I suppose one could imagine some scenario where "back" is really the user's choice, but if that is the case maybe a more thorough "draw order" or "depth order" feature makes sense. I believe plots are drawn in order of first listed to last listed, so there is already a way to control plotting.
Well, if there is one small thing to change, it would be that hidden3d default be "front" instead of "back".
I've added your 2015may07 patch to version 5.1 cvs.
I made one change: the interpretation of the opt_out_of_hidden3d flag.
When an image is viewed edge on I sometimes get warnings printed to the terminal:
Is that the bug you mentioned earlier? Weird that N in the second warning is 2^14.
The patch works for version 5.0 also but I'll wait to apply until these spurious warning are fixed or suppressed.