From: Amitha P. <pe...@cs...> - 2002-10-08 21:46:31
|
Hi there Disclaimer: I haven't studied the vil2 source in detail. I'm just going on Ian's presentation at the VXL meeting, from the discussions on this list, and from reading the occasional commit log What is The Correct Way to think about RGB images in vil2? The existence of vil2_image_view<vil_rgb<vxl_byte> > and vil2_image_view<vxl_byte> with nplanes()==3 creates some confusion. Consider the following vil code snippet: vil_memory_image_of<vxl_uint_8> img( vil_image_as<vxl_uint_8>( vil_load("file") ); // ...grey byte image processing code... It doesn't matter what image type "file" contains; my code will get a appropriately converted byte image. I think this is extremely useful. What is the similar code in vil2? In case there isn't one (yet), perhaps a something like vil2_flatten( my_multi_plane_image, vil2_flatten_params().rgb_to_grey() ) could be useful? This could be used for other squashings: vil2_flatten( my_multi_plane_image, vil2_flatten_params().average() ) vil2_flatten( my_multi_plane_image, vil2_flatten_params().add() ) Related topic: it seems to me that there is no real need to image views of type vil_rgb<vxl_uint_8> *provided* the backends (file format code) does the appropriate conversion automatically. I would argue that vil2 doesn't have vil_rgb *at all*. Is someone willing to convince me otherwise? [ The text below is somewhat disjointed. I apologise. I wrote things down as I thought of them, and don't really want to write a paper on it... ] If we don't have a vil_rgb, then introducing vil2 to a new user is easy: if you load an RGB image into img, then img(i,j,c) gets the c-th "component" of the image at pixel (i,j). For a greyscale image, use img(i,j,0) or simply img(i,j). Done. If we never think in terms of vil_rgb, then we don't think img(i,j).r = 5 is any better than img(i,j,0) = 5 Especially if we make it clear that an RGB image is loaded as a 3 plane image where plane 0 is R, plane 1 is G and plane 2 is B, unless explicitly specified otherwise. My basic assumption is that there are no images with multiple components *and* multiple planes. Then, a multi-component image is equivalent to a multi-plane image. You only really need multiple components when the component types have different types. I can understand struct my_data_type { int lum; float dist; float phase_of_moon; }; ... vil_image_view< my_data_type > img; for some exotic application. However, when each component has the same type, as is nearly always the case, each component can be indexed using the plane index. As I understand it, the original vil tried to allow for an image view that matches the on-disk view of the data, hence the multiple planes, multiple components thing. With the arithmetic indexing in vil2, it doesn't matter. Swap istep with plane_step and voila! I (the user) sees multiple planes, the back-end sees multiple components. If your code really needs to know which way the data is ordered on disk so it can optimize its access patterns, then I suggest that it should get an vil_image_resource and ask it. An image view is not the generalization it should work with. As for loading and saving images, I think the back-ends can do appropriate and automatic conversions. E.g. with PNM files, 1-plane bool images are PBM, 1-plane other images are PGM and 3-plane images PPM. If the file formats support other options (tiff?), and you care about the final layout, then you need to specify take care of it yourself: vil2_tiff_image_file tiff_img( "myoutput", vil2_tiff_image_file_params().save_5_components_and_then_switch_to_planes() ); tiff_img.put_view( my_view ); And if you do need a multi-component, multi-plane image, you will need to write the loader for it anyway, so you can write the appropriate conversion stuff, and most algorithms in vil2 wouldn't apply anyway, so why clutter vil2 with it? Of course, I start thinking about this *after* Ian, Tim and Peter have spent so much time on vil2 and vil_rgb... :-) Amitha. |
From: Peter V. <Pet...@es...> - 2002-10-09 05:52:44
|
> What is The Correct Way to think about RGB images in vil2? It's indeed as you say: RGB images are normally always planar (3 planes). That's e.g. what all loaders (currently just PNM :-) will return. And since the istep(), jstep() and planestep() values have to be used in user code, it is indeed correct that interleaved RGB images can (and normally will) present themselves as 3-plane colour images. Be it with istep()==3 and planestep()==1. But since vil2_image_view is a templated class, users are free to template over vil_rgb (a 3-byte pixel type). Although we would strongly discourage this. The only trace of this is the fact that vil2_pixel_type can assume the value VIL2_PIXEL_TYPE_RGB_BYTE, meaning that's a 3-byte pixel at the lowest level. I don't think this has to be removed, only its use should be discouraged. Peter. |
From: Amitha P. <pe...@cs...> - 2002-10-09 14:18:35
|
Peter Vanroose writes: > But since vil2_image_view is a templated class, users are free to template > over vil_rgb (a 3-byte pixel type). Although we would strongly discourage > this. > > The only trace of this is the fact that vil2_pixel_type can assume the > value VIL2_PIXEL_TYPE_RGB_BYTE, meaning that's a 3-byte pixel at the lowest > level. I don't think this has to be removed, only its use should be > discouraged. I would disagree somewhat here on the actual implementation. The pixel_format type is currently in vil_image_view. And I suggest that the view need not support vil_rgb pixel formats at the view level. I can understand such a flag at the image_resource level; here, it would be a description of how the data *actually* is. And this could be RGB. But that aside, my main concern is with vil_rgb appearing in the vil2-supplied conversion, print and math routines. I don't think they need to be, and I think those routines can easily be generalized to planes!=3. I don't think a user, using just a view, needs to know anything other than the number of planes and the type of each "pixel". (That is, typeof(img(i,j,p)) ). I am concerned that when there is even some trace of vil_rgb, users will latch onto that as the way to handle RGB images, because they have a mental separation between greyscale and RGB. It won't be obvious to many that it's more consistent to just deal with 3 planes instead. And they will write algorithms for vil_image_view< vil_rgb<...> > instead of writing them for vil_image_view<...> and 3-planes. Unfortunately, this kind of thing happens too often, and it creates problems in the future with system integration and such.[1] I really can't think of any place in vil2 where explicit support for vil_rgb would be useful. I offer to pull out all references to vil_rgb from vil2, while trying to maintain the existing functionality. I will begin to do this on Friday, unless someone has objections. Amitha. [1] Yes, user education is the key to most of these problems, but limited time and limited means conspire to make this not always possible. |
From: Ian S. <ian...@st...> - 2002-10-10 09:30:33
|
Short comment: I disagree with Amitha's proposed removal of vil_rgb support from vil2. Long comment: This is the way we have been thinking about rgb images, during the design. 1. All component-wise images vil2_image_view<vil_rgb<float>> can be represented as multi-plane scalar components images, but not vice-versa. This (and the fact that there are limited number of scalar pixel types) means that it is much simpler to do image manipulation in terms of planes. All vil2_image_resource (e.g. file images, etc,) which cannot know their pixel type until run-time, only deal with multi-planar views. (see caveat A) 2. If you are actually interested in playing with vil_rgb pixels, then that is no problem. Most image processing code on vil2_image_views is fully templated on pixel type and so will support vil_rgb pixels. There may be cases where some algorithms will only work on multi-planar views (e.g. where there is a switch statement on the pixel type.) 3. There are functions which will convert between multi-planar and multi-component views, some quietly and very quickly (e.g. vil2_image_view<T>::operator=().) It is relatively trivial to provide and maintain this level of support for vil2_image_view<vil_rgb<T>> and lots of people want it. 4. If you want to convert from rgb to greyscale, then you can do so explicitly at the moment, using vil2_convert* 5. If you want to load any image file, and force it into a particular view, then you can sort of do that now, but not easily. You will be able to do it more easily when we have written the function. I foresee the function looking like vil2_force_convert(const vil2_image_view<T1> &, const vil2_image_view<T2>) which will look at the pixel types and decide what to do in each case. *Caveat A: If there is some file type that natively provides a multi-component image where each compound pixel type contains differently typed components (e.g. vil2_depth_pixel{ vxl_byte r, g,b; float depth;} ) then that file_format (And that file_format only) may explicitly support this compound pixel type. Further: About Amitha's described problem of users latching on to vil_rgb image types, I'll go with his proposed solution and write a good chapter for the VXL book. I was reminded at the last meeting that whilst VXL libraries may have default ways of writing code and interfaces, we should not discourage users from doing what they want to do. If users really want to think about their images as vil_rgb then let them, it is no major problem for me. If somebody proposes adding code to vil2/algo which is specialised to vil2_image_view<vil_rgb<T>> without a very good reason, then we can disagree and point at the agreed standards (mul/vil2/notes.html) Ian. > -----Original Message----- > From: Amitha Perera [mailto:pe...@cs...] > Sent: Tuesday, October 08, 2002 10:46 PM > To: vxl...@li... > Subject: [Vxl-maintainers] vil2 and RGB images > > > Hi there > > Disclaimer: I haven't studied the vil2 source in detail. I'm just > going on Ian's presentation at the VXL meeting, from the discussions > on this list, and from reading the occasional commit log > > > What is The Correct Way to think about RGB images in vil2? The > existence of vil2_image_view<vil_rgb<vxl_byte> > and > vil2_image_view<vxl_byte> with nplanes()==3 creates some > confusion. Consider the following vil code snippet: > > vil_memory_image_of<vxl_uint_8> ( > vil_image_as<vxl_uint_8>( vil_load("file") ); > // ...grey byte image processing code... > > It doesn't matter what image type "file" contains; my code will get a > appropriately converted byte image. I think this is extremely > useful. What is the similar code in vil2? > > In case there isn't one (yet), perhaps a something like > > vil2_flatten( my_multi_plane_image, > vil2_flatten_params().rgb_to_grey() ) > > could be useful? This could be used for other squashings: > > vil2_flatten( my_multi_plane_image, > vil2_flatten_params().average() ) > > vil2_fla tten( my_multi_plane_image, vil2_flatten_params().add() ) > > > Related topic: it seems to me that there is no real need to image > views of type vil_rgb<vxl_uint_8> *provided* the backends (file format > code) does the appropriate conversion automatically. I would argue > that vil2 doesn't have vil_rgb *at all*. Is someone willing to > convince me otherwise? > > [ The text below is somewhat disjointed. I apologise. I wrote things > down as I thought of them, and don't really want to write a paper on > it... ] > > If we don't have a vil_rgb, then introducing vil2 to a new user is > easy: if you load an RGB image into img, then img(i,j,c) gets the c-th > "component" of the image at pixel (i,j). For a greyscale image, use > img(i,j,0) or simply img(i,j). Done. If we never think in terms of > vil_rgb, then we don't think > img(i,j).r = 5 > is any better than > img(i,j,0) = 5 > Especially if we make it clear that an RGB image is loaded as a 3 > plane image where plane 0 is R, plane 1 is G and plane 2 is B, unless > explicitly specified otherwise. > > > My basic assumption is that there are no images with multiple > components *and* multiple planes. Then, a multi-component image is > equivalent to a multi-plane image. You only really need multiple > components when the component types have different types. I can > understand > > struct my_data_type { int lum; float dist; float phase_of_moon; }; > ... > vil_image_view< my_data_type > img; > > for some exotic application. However, when each component has the same > type, as is nearly always the case, each component can be indexed > using the plane index. > > As I understand it, the original vil tried to allow for an image view > that matches the on-disk view of the data, hence the multiple planes, > multiple components thing. With the arithmetic indexing in vil2, it > doesn't matter. Swap istep with plane_step and voila! I (the user) > sees multiple planes, the back-end sees multiple components. > > If your code really needs to know which way the data is ordered on > disk so it can optimize its access patterns, then I suggest that it > should get an vil_image_resource and ask it. An image view is not the > generalization it should work with. > > As for loading and saving images, I think the back-ends can do > appropriate and automatic conversions. E.g. with PNM files, 1-plane > bool images are PBM, 1-plane other images are PGM and 3-plane images > PPM. If the file formats support other options (tiff?), and you care > about the final layout, then you need to specify take care of > it yourself: > > vil2_tiff_image_file tiff_img( "myoutput", > > vil2_tiff_image_file_params().save_5_components_and_then_switc > h_to_planes() > ); > tiff_img.put_view( my_view ); > > > And if you do need a multi-component, multi-plane image, you will need > to write the loader for it anyway, so you can write the appropriate > conversion stuff, and most algorithms in vil2 wouldn't apply anyway, > so why clutter vil2 with it? > > Of course, I start thinking about this *after* Ian, Tim and Peter have > spent so much time on vil2 and vil_rgb... :-) > > Amitha. > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Vxl-maintainers mailing list > Vxl...@li... > https://lists.sourceforge.net/lists/listinfo/vxl-maintainers > |
From: Amitha P. <pe...@cs...> - 2002-10-10 12:46:07
|
"Ian Scott" <ian...@st...> writes: > Short comment: I disagree with Amitha's proposed removal of vil_rgb support > from vil2. [...] > About Amitha's described problem of users latching on to vil_rgb image > types, I'll go with his proposed solution and write a good chapter for the > VXL book. Fair enough. If users read it and still use vil_rgb, well, they're lost causes. :-) > I was reminded at the last meeting that whilst VXL libraries may have > default ways of writing code and interfaces, we should not discourage users > from doing what they want to do. If users really want to think about their > images as vil_rgb then let them, it is no major problem for me. If somebody > proposes adding code to vil2/algo which is specialised to > vil2_image_view<vil_rgb<T>> without a very good reason, then we can disagree > and point at the agreed standards (mul/vil2/notes.html) Good enough solution for me. I guess we at RPI just have to be more vigilant with our internal code. Amitha. |
From: Ian S. <ian...@st...> - 2002-10-11 09:06:35
|
> From: Amitha Perera [mailto:pe...@cs...] > Sent: Tuesday, October 08, 2002 10:46 PM >... > > vil_memory_image_of<vxl_uint_8> img( > vil_image_as<vxl_uint_8>( vil_load("file") ); > // ...grey byte image processing code... > .. > What is the similar code in vil2? Now that I have written it, the code to do this is vil2_image_view<vxl_byte> img = vil2_convert_to_grey_using_average( vil_load("file"), vxl_byte() ); // ...grey byte image processing code... |