The VXL Project

My thoughts are that handling image blocking is a file read implementation
detail that is specific to certain file types.  If a filetype supports
blocking then the subclass that reads that type could/should handle it
transparently (note that the block related NITF functions below are all
protected).  I can't envision a scenario where client code would want to
read just the (i,j)-th block or even really a scenario where the client code
would need to know the blocking details of a particular file.  Well, I guess
the latter case is reasonable just in terms of providing file meta-data.  In
the NITF case, one could access this information in one of two ways:
 
- Use the vil_nitf2 specific get_tree() function which would provide them
with all of the support data relevant to either the whole file, or a
specific header.
- Use vil_image_resource::get_property() function with the appropriate tags
from the NITF specification ("NPPBH", "NPPBV", "NBPR", "NBPC" respectively).

 
Assuming I'm not missing something about how TIFF files use blocking, I
don't see the need for either solution: moving the blocking API up to
vil_image_resource or making a block-aware subclass.  
 
However, while we're on the topic of moving functionality up to
vil_image_resource, I have another question.  Does anyone object to my
moving these functions up to vil_image_resource (from vil_nitf2_image)?
 
  void set_current_image( unsigned int index );
  unsigned int nimages() const;
 
All filetypes that don't support multiple images in a single file won't be
affected; nimages() will return 1 and they will only be allowed to pass one
into set_current_image().   As far as I know, vil_nitf2_image is the only
subclass for which this functionality will be useful, but this change should
make it easier for client code to access this capability.  
 
Rob
 



  _____  

From: vxl-maintainers-admin@...
[mailto:vxl-maintainers-admin@...] On Behalf Of Joseph L
Mundy
Sent: Monday, December 19, 2005 6:33 AM
To: vxl-maintainers@...
Subject: [Vxl-maintainers] Blocked Images?



Dear All,

The vil image library does not currently recognize blocked images. For
example TIFF6.0 extends the concept of  "strips" to allow general
rectangular "tiles".  The NITF2.0 and 2.1 file formats also provides
blocking.  Indeed, the vil_nitf2_image class has the following methods:

  unsigned int get_pixels_per_block_x() const;

  unsigned int get_pixels_per_block_y() const;

  unsigned int get_num_blocks_x() const;

  unsigned int get_num_blocks_y() const;

 

I plan to extend vil_tiff  to handle blocks as well. Should we have block
parameters and methods on vil_image_resource/view or should there be a
sub-class of these that supports blocking?  Also the get_... should be
removed in the new block method names according to our vxl coding standards.

Joe

o----o----o----o----o----o----o----o

Professor of Engineering(Research)

Room 351

Barus and Holley

184 Hope Street

Providence, RI

401-863-2655

Rob,

The need for knowledge of blocking comes up in at least two contexts:

1)    Processing large images, e.g. filtering kernels that span block
boundaries.

2)    Displaying large images

Both of these applications will benefit from a smart block cache. The
cache anticipates what blocks need to be retained since they are likely
to be reused.

If we want to build generic mechanisms that work for both blocked and
unblocked images, wouldn't it be difficult to tell which formats are
blocked without methods on vil_image_resource/view?  Also there is no
guarantee that the method signatures would be consistent from one
blocked image file type to another. 

 

I have no objection to adding the multiple image/file support to
vil_image_resource.  

Joe

 

-----Original Message-----
From: vxl-maintainers-admin@...
[mailto:vxl-maintainers-admin@...] On Behalf Of Rob
Radtke
Sent: Monday, December 19, 2005 10:29 AM
To: 'Joseph L Mundy'; vxl-maintainers@...
Subject: RE: [Vxl-maintainers] Blocked Images?

 

My thoughts are that handling image blocking is a file read
implementation detail that is specific to certain file types.  If a
filetype supports blocking then the subclass that reads that type
could/should handle it transparently (note that the block related NITF
functions below are all protected).  I can't envision a scenario where
client code would want to read just the (i,j)-th block or even really a
scenario where the client code would need to know the blocking details
of a particular file.  Well, I guess the latter case is reasonable just
in terms of providing file meta-data.  In the NITF case, one could
access this information in one of two ways:

 

- Use the vil_nitf2 specific get_tree() function which would provide
them with all of the support data relevant to either the whole file, or
a specific header.

- Use vil_image_resource::get_property() function with the appropriate
tags from the NITF specification ("NPPBH", "NPPBV", "NBPR", "NBPC"
respectively).  

 

Assuming I'm not missing something about how TIFF files use blocking, I
don't see the need for either solution: moving the blocking API up to
vil_image_resource or making a block-aware subclass.  

 

However, while we're on the topic of moving functionality up to
vil_image_resource, I have another question.  Does anyone object to my
moving these functions up to vil_image_resource (from vil_nitf2_image)?

 

  void set_current_image( unsigned int index );
  unsigned int nimages() const;

 

All filetypes that don't support multiple images in a single file won't
be affected; nimages() will return 1 and they will only be allowed to
pass one into set_current_image().   As far as I know, vil_nitf2_image
is the only subclass for which this functionality will be useful, but
this change should make it easier for client code to access this
capability.  

 

Rob

 

 


  _____  


From: vxl-maintainers-admin@...
[mailto:vxl-maintainers-admin@...] On Behalf Of Joseph
L Mundy
Sent: Monday, December 19, 2005 6:33 AM
To: vxl-maintainers@...
Subject: [Vxl-maintainers] Blocked Images?

Dear All,

The vil image library does not currently recognize blocked images. For
example TIFF6.0 extends the concept of  "strips" to allow general
rectangular "tiles".  The NITF2.0 and 2.1 file formats also provides
blocking.  Indeed, the vil_nitf2_image class has the following methods:

  unsigned int get_pixels_per_block_x() const;

  unsigned int get_pixels_per_block_y() const;

  unsigned int get_num_blocks_x() const;

  unsigned int get_num_blocks_y() const;

 

I plan to extend vil_tiff  to handle blocks as well. Should we have
block parameters and methods on vil_image_resource/view or should there
be a sub-class of these that supports blocking?  Also the get_... should
be removed in the new block method names according to our vxl coding
standards.

Joe

o----o----o----o----o----o----o----o

Professor of Engineering(Research)

Room 351

Barus and Holley

184 Hope Street

Providence, RI

401-863-2655

Hi Joe,
Thanks for explaining.  I think I see what you're getting at.  Assuming that
the image blocks were individually compressed or that random access to a
given location within a block was slow (for some other reason), one could
realize great benefit from caching entire blocks at a time.  For the case of
uncompressed NITF files, block boundaries have always seemed random and
insignificant to me.  That is, I wouldn't see much advantage to aligning
anything along block boundaries; you could still cache pixels, but there
wouldn't seem to be much advantage to doing so along block boundaries.  At
any rate, I think I see where you're going with this now.  I suppose I'd
vote for your first suggestion: move the blocking API directly onto
vil_image_resource.
 
Rob
 
 


  _____  

From: Joseph L Mundy [mailto:mundy@...] 
Sent: Monday, December 19, 2005 9:09 AM
To: 'Rob Radtke'; vxl-maintainers@...
Subject: RE: [Vxl-maintainers] Blocked Images?



Rob,

The need for knowledge of blocking comes up in at least two contexts:

1)    Processing large images, e.g. filtering kernels that span block
boundaries.

2)    Displaying large images

Both of these applications will benefit from a smart block cache. The cache
anticipates what blocks need to be retained since they are likely to be
reused.

If we want to build generic mechanisms that work for both blocked and
unblocked images, wouldn't it be difficult to tell which formats are blocked
without methods on vil_image_resource/view?  Also there is no guarantee that
the method signatures would be consistent from one blocked image file type
to another. 

 

I have no objection to adding the multiple image/file support to
vil_image_resource.  

Joe

 

-----Original Message-----
From: vxl-maintainers-admin@...
[mailto:vxl-maintainers-admin@...] On Behalf Of Rob Radtke
Sent: Monday, December 19, 2005 10:29 AM
To: 'Joseph L Mundy'; vxl-maintainers@...
Subject: RE: [Vxl-maintainers] Blocked Images?

 

My thoughts are that handling image blocking is a file read implementation
detail that is specific to certain file types.  If a filetype supports
blocking then the subclass that reads that type could/should handle it
transparently (note that the block related NITF functions below are all
protected).  I can't envision a scenario where client code would want to
read just the (i,j)-th block or even really a scenario where the client code
would need to know the blocking details of a particular file.  Well, I guess
the latter case is reasonable just in terms of providing file meta-data.  In
the NITF case, one could access this information in one of two ways:

 

- Use the vil_nitf2 specific get_tree() function which would provide them
with all of the support data relevant to either the whole file, or a
specific header.

- Use vil_image_resource::get_property() function with the appropriate tags
from the NITF specification ("NPPBH", "NPPBV", "NBPR", "NBPC" respectively).


 

Assuming I'm not missing something about how TIFF files use blocking, I
don't see the need for either solution: moving the blocking API up to
vil_image_resource or making a block-aware subclass.  

 

However, while we're on the topic of moving functionality up to
vil_image_resource, I have another question.  Does anyone object to my
moving these functions up to vil_image_resource (from vil_nitf2_image)?

 

  void set_current_image( unsigned int index );
  unsigned int nimages() const;

 

All filetypes that don't support multiple images in a single file won't be
affected; nimages() will return 1 and they will only be allowed to pass one
into set_current_image().   As far as I know, vil_nitf2_image is the only
subclass for which this functionality will be useful, but this change should
make it easier for client code to access this capability.  

 

Rob

 

 


  _____  


From: vxl-maintainers-admin@...
[mailto:vxl-maintainers-admin@...] On Behalf Of Joseph L
Mundy
Sent: Monday, December 19, 2005 6:33 AM
To: vxl-maintainers@...
Subject: [Vxl-maintainers] Blocked Images?

Dear All,

The vil image library does not currently recognize blocked images. For
example TIFF6.0 extends the concept of  "strips" to allow general
rectangular "tiles".  The NITF2.0 and 2.1 file formats also provides
blocking.  Indeed, the vil_nitf2_image class has the following methods:

  unsigned int get_pixels_per_block_x() const;

  unsigned int get_pixels_per_block_y() const;

  unsigned int get_num_blocks_x() const;

  unsigned int get_num_blocks_y() const;

 

I plan to extend vil_tiff  to handle blocks as well. Should we have block
parameters and methods on vil_image_resource/view or should there be a
sub-class of these that supports blocking?  Also the get_... should be
removed in the new block method names according to our vxl coding standards.

Joe

o----o----o----o----o----o----o----o

Professor of Engineering(Research)

Room 351

Barus and Holley

184 Hope Street

Providence, RI

401-863-2655

I think the real solution is to have various image iterators that can
encode the type of access required by the algorithm. Off the top of
my head,
- unordered_pixel_iterator
- ordered_pixel_iterator
- unordered_block_iterator.

However, this is unlikely to happen soon, so I'd be okay with a
block-access API at the view/resource level.

BTW, it seems that even a pixel_iterator cannot be implemented as
efficiently as two nested loops because of C/C++ aliasing rules and
optimizer limitations. Even with agressive inlining. We should go to
FORTRAN. Or back to LISP. :-)

Amitha.

Amitha Perera wrote:
> BTW, it seems that even a pixel_iterator cannot be implemented as
> efficiently as two nested loops because of C/C++ aliasing rules and
> optimizer limitations. Even with agressive inlining. We should go to
> FORTRAN. Or back to LISP. :-)

I'm pretty sure it is possible to convince the compiler to do certain 
optimizations, at least in C++ with its more strict type system.  Can 
you point me to any references that describe this problem?

Thanks,
-Brad

Hi,

I'm sorry I didn't reply to this earlier - been on holiday.

As part of the original design for vil, I was asked to consider how to=20
provide support for blocks and very large images. Whilst there is no=20
explicit provision, it was designed to be fairly easy to add such=20
support to the vil_image_resource interface.

The basic design idea for VIL was the vil_image_view<T> would be the=20
preferred type for algorithms working on simple in-memory images. An=20
overloaded version of each algorithm operating on=20
vil_image_resource_sptr would then handle more complicated imaging=20
situations - images on disks, images whose pixel type isn't immediately=20
known, AND images which have prefered blocking parameters.

$SRC/core/vil/notes.html has my suggestion on how to implement blocking:
> Add block processing implementation to the vil_image_resource
 > derivatives for very large images.
 >
> The basic idea of dealing with large images was designed into vil1.
 > It deferred loading the whole image into memory until actually needed.
 > You could then load only one block of the image as required. This
 > basic idea has been passed into vil2/vil. However, as with vil1, this
 > is only a design outline and API. A lot of implementation to do
 > block-by-block processing efficiently is missing.
>=20
> The first step is to add preferred_block_size and
 > preferred_block_origin to vil_property.h. Both of these unsigned[2]
 > properties would be provided by file resources and used by the
 > processing resources to do the work one block at a time. I guess many
 > of the existing file resources will specify blocks that are one or
 > more full rasters. Those file formats designed to deal with large
 > images (including TIFF) can store their pixels in rectangular tiles.
 > The processing resources should be able to use several of their
 > supplier's blocks together up to some locally decided (or globally
 > specified) limit. The processing resources would need to export these
 > properties including any modification (e.g. vil_transpose().)
>=20
> Quite how vil_convolve_1d_resource should deal with a single column
 > for a preferred block size (when vil_convolve_1d_resource operates on
 > rows) - I don't know. It would take some experimentation to get an
 > efficient solution. Until then we can always rely on the fact that the
 > preferred_block_size, etc are just hints and can safely be ignored at
 > the loss of efficiency.

I guess that finding this documentation has been made difficult with the=20
failure in our machine room.

If anyone has any questions, let me know.

Ian.


Joseph L Mundy wrote:
> Dear All,
>=20
> The vil image library does not currently recognize blocked images. For=20
> example TIFF6.0 extends the concept of  =93strips=94 to allow general=20
> rectangular =93tiles=94.  The NITF2.0 and 2.1 file formats also provide=
s=20
> blocking.  Indeed, the vil_nitf2_image class has the following methods:
>=20
>   unsigned int get_pixels_per_block_x() const;
>=20
>   unsigned int get_pixels_per_block_y() const;
>=20
>   unsigned int get_num_blocks_x() const;
>=20
>   unsigned int get_num_blocks_y() const;
>=20
> =20
>=20
> I plan to extend vil_tiff  to handle blocks as well. Should we have=20
> block parameters and methods on vil_image_resource/view or should there=
=20
> be a sub-class of these that supports blocking?  Also the get_... shoul=
d=20
> be removed in the new block method names according to our vxl coding=20
> standards.
>=20
> Joe

Rob Radtke wrote:

> However, while we're on the topic of moving functionality up to 
> vil_image_resource, I have another question.  Does anyone object to my 
> moving these functions up to vil_image_resource (from vil_nitf2_image)?
>  
>   void set_current_image( unsigned int index );
>   unsigned int nimages() const;
>  
> All filetypes that don't support multiple images in a single file won't 
> be affected; nimages() will return 1 and they will only be allowed to 
> pass one into set_current_image().   As far as I know, vil_nitf2_image 
> is the only subclass for which this functionality will be useful, but 
> this change should make it easier for client code to access this 
> capability. 

Sorry I didn't get back to your earlier about this - been on holiday.

I think there are neater ways of handling this. The vil_properties 
interface was designed for handling the read-only aspects of this 
situation. Have a look in vil_properties.h for a full explanation. As 
for setting the parameter, it has always seemed preferable to require 
explicit down-casting to get to vil_nitf2_image::set_current_image() 
than clutter the base-class API.

The beauty of this design is more visible in vil3d which follows an 
identical design. In medical image formats (the main domain of 3d 
images) there are often all sorts of information stored in the file 
format - patient name, consulting physician's name, billing record 
number... Some of this stuff is genuinely useful, but it can't all 
reasonably be added to the base class API. For a start every function in 
the base class needs to be understood by every user who wants to derive 
from that base class. Answering mailing-list questions about vil takes 
enough of my time without try to explain the purpose of rarely-used 
methods in the base class. My experience has been that clean, simple, 
base-class APIs are the basis for long-term stability and ease of use.


To avoid having never-ending discussions about which parameters are 
useful for lots of people, and easy to understand by everybody, we use 
an alternative criterion - is the parameter provided by the majority of 
image formats. So far only one parameter has passed that criterion - 
voxel size. We still use the vil_property API for reading that value, 
but we have added a set_voxel_size(...) method to the base 
vil3d_image_resource class.

I suggest that a similar criterion be applied to vil, and these changes 
removed from the base class API.

Ian.

Ian,
I guess the main question is whether or not block attributes are
important enough to rise to their own hallowed spot on the
vil_image_resource interface.  I would argue that getting and putting
blocks is a reasonable thing for a resource to do. It seems pretty
inconvenient to have to get the blocking properties via get_property
then figure out what view corresponds to a block of interest. 

Also, did you have any thoughts on multiple images per resource?  Both
tiff and nitf  support multiple images in a single file by indexing
through separate image headers and data blocks. This capability in tiff
was designed to support resolution pyramids and fax transmissions
consisting of a number of pages. Rob Radtke has added  methods for this
case to the image_resource interface,  n_images() and
set_current_image(unsigned i) .  I think we also need add_image(unsigned
ni, unsigned nj, unsigned nplanes, vil_pixel_format fmt).

Joe
-----Original Message-----
From: Ian Scott [mailto:ian.m.scott@...] 
Sent: Wednesday, January 04, 2006 5:47 AM
To: Joseph L Mundy
Cc: vxl-maintainers@...
Subject: Re: [Vxl-maintainers] Blocked Images?

Hi,

I'm sorry I didn't reply to this earlier - been on holiday.

As part of the original design for vil, I was asked to consider how to 
provide support for blocks and very large images. Whilst there is no 
explicit provision, it was designed to be fairly easy to add such 
support to the vil_image_resource interface.

The basic design idea for VIL was the vil_image_view<T> would be the 
preferred type for algorithms working on simple in-memory images. An 
overloaded version of each algorithm operating on 
vil_image_resource_sptr would then handle more complicated imaging 
situations - images on disks, images whose pixel type isn't immediately 
known, AND images which have prefered blocking parameters.

$SRC/core/vil/notes.html has my suggestion on how to implement blocking:
> Add block processing implementation to the vil_image_resource
 > derivatives for very large images.
 >
> The basic idea of dealing with large images was designed into vil1.
 > It deferred loading the whole image into memory until actually
needed.
 > You could then load only one block of the image as required. This
 > basic idea has been passed into vil2/vil. However, as with vil1, this
 > is only a design outline and API. A lot of implementation to do
 > block-by-block processing efficiently is missing.
> 
> The first step is to add preferred_block_size and
 > preferred_block_origin to vil_property.h. Both of these unsigned[2]
 > properties would be provided by file resources and used by the
 > processing resources to do the work one block at a time. I guess many
 > of the existing file resources will specify blocks that are one or
 > more full rasters. Those file formats designed to deal with large
 > images (including TIFF) can store their pixels in rectangular tiles.
 > The processing resources should be able to use several of their
 > supplier's blocks together up to some locally decided (or globally
 > specified) limit. The processing resources would need to export these
 > properties including any modification (e.g. vil_transpose().)
> 
> Quite how vil_convolve_1d_resource should deal with a single column
 > for a preferred block size (when vil_convolve_1d_resource operates on
 > rows) - I don't know. It would take some experimentation to get an
 > efficient solution. Until then we can always rely on the fact that
the
 > preferred_block_size, etc are just hints and can safely be ignored at
 > the loss of efficiency.

I guess that finding this documentation has been made difficult with the

failure in our machine room.

If anyone has any questions, let me know.

Ian.


Joseph L Mundy wrote:
> Dear All,
> 
> The vil image library does not currently recognize blocked images. For

> example TIFF6.0 extends the concept of  "strips" to allow general 
> rectangular "tiles".  The NITF2.0 and 2.1 file formats also provides 
> blocking.  Indeed, the vil_nitf2_image class has the following
methods:
> 
>   unsigned int get_pixels_per_block_x() const;
> 
>   unsigned int get_pixels_per_block_y() const;
> 
>   unsigned int get_num_blocks_x() const;
> 
>   unsigned int get_num_blocks_y() const;
> 
>  
> 
> I plan to extend vil_tiff  to handle blocks as well. Should we have 
> block parameters and methods on vil_image_resource/view or should
there 
> be a sub-class of these that supports blocking?  Also the get_...
should 
> be removed in the new block method names according to our vxl coding 
> standards.
> 
> Joe

Joseph L Mundy wrote:
> Ian,
> I guess the main question is whether or not block attributes are
> important enough to rise to their own hallowed spot on the
> vil_image_resource interface.  I would argue that getting and putting
> blocks is a reasonable thing for a resource to do. It seems pretty
> inconvenient to have to get the blocking properties via get_property
> then figure out what view corresponds to a block of interest. 

Assuming that the use of block-optimised support were to become 
widespread through out the vil_image_resource derivatives, then it would 
indeed make sense to provide it as a public method on the base class. I 
suspect the implementation of algorithms that efficiently handle 
multiple blocks is a more interesting question, and a lot trickier.


> Also, did you have any thoughts on multiple images per resource?  Both
> tiff and nitf  support multiple images in a single file by indexing
> through separate image headers and data blocks. This capability in tiff
> was designed to support resolution pyramids and fax transmissions
> consisting of a number of pages. Rob Radtke has added  methods for this
> case to the image_resource interface,  n_images() and
> set_current_image(unsigned i) .  I think we also need add_image(unsigned
> ni, unsigned nj, unsigned nplanes, vil_pixel_format fmt).
> 


As I wrote in another email to Rob (that should have been cc'ed to the 
list), I don't like adding functions to a base class that aren't 
meaningful to  most of the derived classes, and in general I prefer 
class designs with the minimal functional API.

The multiple images functionality is currently only meaningful to two 
derived file loaders out of tens of derivatives of vil_image_resource 
(only one of which has any implementation). I think that adding multiple 
image support to the base class would suggest that all the algorithm 
wrappers (eg vil_transpose_image_resource and vil_convolve_1d_resource) 
should know how to handle multiple images.

It could be that I don't understand the use of multiple images here - 
but it appears that the meaning of multiple images may not be 
consistent. If it means both multi-resolution layers, and multiple pages 
  in tiff, then how are users going to write correct programs supporting 
multiple images if the two meanings share the same API?

As an completely different approach - have you thought about handling 
multi-image NITF or TIFF files transparently using a 
vil_image_resource_plugin? When trying to read the 3rd image from 
my_image.nitf you would just pass the name "my_image.nitf:2" to the one 
of the vil_load_*() functions. When the normal loaders fail to load the 
file, the plugins would get to handle it. They can interpret the 
filename anyway they like - in this case, passing "my_image.nitf" back 
to the vil loader framework to load the file, and then downcasting to 
one of the vil_image_resource that knows about multiple images, and 
extract the appropriate image. A colleague of mine used this approach to 
  work on video files, without having to make any other code aware of 
the fact that it wasn't playing with ordinary images. See 
$src/contrib/mul/mvl2/mvl2_image_format_plugin.* for more details. This 
approach has the advantage of delegating worry about the meaning of 
multiple images to the user, when they specify the indexed filename.

Ian.

Ian,
Thanks for the ideas.  I will think about the plug-in concept. While we
are on a roll, what are your thoughts on a pyramid image resource?  In
order to display large images, it is necessary to maintain a resolution
pyramid and the gui has to extract the appropriate view according to
scale.  I know that vimt handles pyramids with associated 2-d transforms
but if we want have a very basic pyramid that could go into core vil, we
can't depend on vimt. 
Joe

-----Original Message-----
From: Ian Scott [mailto:ian.m.scott@...] 
Sent: Wednesday, January 04, 2006 8:28 AM
To: Joseph L Mundy
Cc: Rob Radtke; vxl-maintainers@...
Subject: Re: [Vxl-maintainers] Blocked Images?



Joseph L Mundy wrote:
> Ian,
> I guess the main question is whether or not block attributes are
> important enough to rise to their own hallowed spot on the
> vil_image_resource interface.  I would argue that getting and putting
> blocks is a reasonable thing for a resource to do. It seems pretty
> inconvenient to have to get the blocking properties via get_property
> then figure out what view corresponds to a block of interest. 

Assuming that the use of block-optimised support were to become 
widespread through out the vil_image_resource derivatives, then it would

indeed make sense to provide it as a public method on the base class. I 
suspect the implementation of algorithms that efficiently handle 
multiple blocks is a more interesting question, and a lot trickier.


> Also, did you have any thoughts on multiple images per resource?  Both
> tiff and nitf  support multiple images in a single file by indexing
> through separate image headers and data blocks. This capability in
tiff
> was designed to support resolution pyramids and fax transmissions
> consisting of a number of pages. Rob Radtke has added  methods for
this
> case to the image_resource interface,  n_images() and
> set_current_image(unsigned i) .  I think we also need
add_image(unsigned
> ni, unsigned nj, unsigned nplanes, vil_pixel_format fmt).
> 


As I wrote in another email to Rob (that should have been cc'ed to the 
list), I don't like adding functions to a base class that aren't 
meaningful to  most of the derived classes, and in general I prefer 
class designs with the minimal functional API.

The multiple images functionality is currently only meaningful to two 
derived file loaders out of tens of derivatives of vil_image_resource 
(only one of which has any implementation). I think that adding multiple

image support to the base class would suggest that all the algorithm 
wrappers (eg vil_transpose_image_resource and vil_convolve_1d_resource) 
should know how to handle multiple images.

It could be that I don't understand the use of multiple images here - 
but it appears that the meaning of multiple images may not be 
consistent. If it means both multi-resolution layers, and multiple pages

  in tiff, then how are users going to write correct programs supporting

multiple images if the two meanings share the same API?

As an completely different approach - have you thought about handling 
multi-image NITF or TIFF files transparently using a 
vil_image_resource_plugin? When trying to read the 3rd image from 
my_image.nitf you would just pass the name "my_image.nitf:2" to the one 
of the vil_load_*() functions. When the normal loaders fail to load the 
file, the plugins would get to handle it. They can interpret the 
filename anyway they like - in this case, passing "my_image.nitf" back 
to the vil loader framework to load the file, and then downcasting to 
one of the vil_image_resource that knows about multiple images, and 
extract the appropriate image. A colleague of mine used this approach to

  work on video files, without having to make any other code aware of 
the fact that it wasn't playing with ordinary images. See 
$src/contrib/mul/mvl2/mvl2_image_format_plugin.* for more details. This 
approach has the advantage of delegating worry about the meaning of 
multiple images to the user, when they specify the indexed filename.

Ian.

> Both tiff and nitf support multiple images in a single file
> by indexing through separate image headers and data blocks.

Although support for this certainly belongs in vil_tiff and vil_nitf2,
I don't think it belongs in vil_image since an image is (by definition)
a single image.
So, at the core vil level, one should only be able to access a single
image from the several ones in a multi-image TIFF file.
At a "higher" (algorithmic?) level, of course, multiple images might be
combined into either a pyramid, or a patched image, or an image (time)
sequence. But I would keep that out of vil.


--	Peter.

Joseph L Mundy wrote:
> Ian,
> Thanks for the ideas.  I will think about the plug-in concept. While we
> are on a roll, what are your thoughts on a pyramid image resource?  In
> order to display large images, it is necessary to maintain a resolution
> pyramid and the gui has to extract the appropriate view according to
> scale.  I know that vimt handles pyramids with associated 2-d transforms
> but if we want have a very basic pyramid that could go into core vil, we
> can't depend on vimt. 
> Joe
> 

I don't know if it is what you need, but how about
struct vil_image_resource_pyramid
{
   vcl_vector<vil_image_resource_sptr> images;
   double scale_step;
}

If there are some useful functions that could operate on this struct, 
then I don't see any reason why they couldn't go in vil.

At the time we wrote vil and vimt, AWF suggested out that our designs 
for the vimt_image hierarchy weren't that relevant to many users, and we 
should avoid polluting the vil workspace - which is one reason why vimt 
ended up as a small extra library. I have always had reason to rethink 
that decision - it would be very useful to have transforms and 
polymorphically typed images of the vimt_image style in the core vil 
library - but always the idea of having to explain three separate image 
class hierarchies in vil would be too much - we have enough trouble 
explaining two :-)

Ian.

Peter Vanroose wrote:

>>Both tiff and nitf support multiple images in a single file
>>by indexing through separate image headers and data blocks.
>>    
>>
>Although support for this certainly belongs in vil_tiff and vil_nitf2,
>I don't think it belongs in vil_image since an image is (by definition)
>a single image.
>So, at the core vil level, one should only be able to access a single
>image from the several ones in a multi-image TIFF file.
>At a "higher" (algorithmic?) level, of course, multiple images might be
>combined into either a pyramid, or a patched image, or an image (time)
>sequence. But I would keep that out of vil.
>  
>
Since the purpose of an image pyramid is to allow the client application 
to read pixels at one resolution without necessarily loading the finest 
resolution data, and since some file formats support storing a pyramid 
as a single file, it seems to me that classes like vil_nitf2_image need 
to behave both as as pyramid and a simple image. Therefore, why not 
define vil_image_pyramid_resource as part of this inheritance hierarchy:

   vil_image_resource  <-- vil_image_pyramid_resource  <-- 
vil_nitf2_image (and other file formats that support pyramids)

where vil_image_pyramid_resource implements vil_image_resource methods 
to access the base level of the pyramid and adds methods to access other 
levels. Clients that don't care about pyramids can just use the 
vil_image_resource interface.

Harry Voorhees
________________________________________________________

 Harry L. Voorhees, hlv@..., 978-475-5279
    Stellar Science Ltd. Co., http://www.stellarscience.com
         "Stellar Scientific Software Solutions"

Hi Ian,
I can definitely appreciate your desire to keep the API simple.  In this
particular case though, it is relatively simple and effective for the base
class (vil_image_resource) to provide default implementations of the
current_image* functions.  These implementations are below.  By doing this,
subclasses that don't support multiple images per file don't have to do
anything.

Regardless, I'm more than happy to defer to your judgement on this issue and
do not mind removing these functions from vil_image_resource.  I'd do it
immediately, but don't want to interfere with Joe's on-going development.

Rob

void vil_image_resource::set_current_image( unsigned int index )
{
  assert( index < nimages() );  
}

unsigned int vil_image_resource::current_image() const
{
  // by default, we only support one image per file.
  // vil_image_resource subclasses can override that
  return 0;
}

unsigned int vil_image_resource::nimages() const
{
  // by default, we only support one image per file.
  // vil_image_resource subclasses can override that
  return 1;
} 

->-----Original Message-----
->From: Ian Scott [mailto:ian.m.scott@...] 
->Sent: Wednesday, January 04, 2006 4:09 AM
->To: Rob Radtke
->Cc: vxl-maintainers@...
->Subject: Re: [Vxl-maintainers] Blocked Images?
->
->Rob Radtke wrote:
->
->> However, while we're on the topic of moving functionality up to 
->> vil_image_resource, I have another question.  Does anyone 
->object to my 
->> moving these functions up to vil_image_resource (from 
->vil_nitf2_image)?
->>  
->>   void set_current_image( unsigned int index );
->>   unsigned int nimages() const;
->>  
->> All filetypes that don't support multiple images in a 
->single file won't 
->> be affected; nimages() will return 1 and they will only be 
->allowed to 
->> pass one into set_current_image().   As far as I know, 
->vil_nitf2_image 
->> is the only subclass for which this functionality will be 
->useful, but 
->> this change should make it easier for client code to access this 
->> capability. 
->
->Sorry I didn't get back to your earlier about this - been on holiday.
->
->I think there are neater ways of handling this. The vil_properties 
->interface was designed for handling the read-only aspects of this 
->situation. Have a look in vil_properties.h for a full explanation. As 
->for setting the parameter, it has always seemed preferable to require 
->explicit down-casting to get to vil_nitf2_image::set_current_image() 
->than clutter the base-class API.
->
->The beauty of this design is more visible in vil3d which follows an 
->identical design. In medical image formats (the main domain of 3d 
->images) there are often all sorts of information stored in the file 
->format - patient name, consulting physician's name, billing record 
->number... Some of this stuff is genuinely useful, but it can't all 
->reasonably be added to the base class API. For a start every 
->function in 
->the base class needs to be understood by every user who wants 
->to derive 
->from that base class. Answering mailing-list questions about 
->vil takes 
->enough of my time without try to explain the purpose of rarely-used 
->methods in the base class. My experience has been that clean, simple, 
->base-class APIs are the basis for long-term stability and ease of use.
->
->
->To avoid having never-ending discussions about which parameters are 
->useful for lots of people, and easy to understand by 
->everybody, we use 
->an alternative criterion - is the parameter provided by the 
->majority of 
->image formats. So far only one parameter has passed that criterion - 
->voxel size. We still use the vil_property API for reading that value, 
->but we have added a set_voxel_size(...) method to the base 
->vil3d_image_resource class.
->
->I suggest that a similar criterion be applied to vil, and 
->these changes 
->removed from the base class API.
->
->Ian.
->

> vil_image_resource  <-- vil_image_pyramid_resource  <-- 
> vil_nitf2_image

Exactly.
So pyramid "images" are at a higher level of complexity, i.e., they use
the basic image layer (vil_image_resource) but do not pollute that
representation level.


--	Peter.

Rob Radtke wrote:
> Hi Ian,
> I can definitely appreciate your desire to keep the API simple.  In this
> particular case though, it is relatively simple and effective for the base
> class (vil_image_resource) to provide default implementations of the
> current_image* functions.

Sorry about this: My problem is that most additions to the base class 
API can have an "in this case" argument. If you want to have a common 
base class for handling multiple images I would much prefer one of the 
solutions below. Your solution may make life slightly easier for you - 
but at the expense of complicating a simple representation of an image 
and confusing it with videos, pyramids, etc.



1. An additional class that contains a vil_image_resource  -  generally 
the base way of solving this problem - but it doesn't play well with the 
vil loading framework.
2. Use a mix-in pure abstract class (e.g. vil_multi_image) with the 
relevant API (in the style of Java interfaces) If the unwrapped 
vil_image_resource_sptr can be dynamically cast to a (vil_multi_image*) 
then you have you control - if it can't then there is no multi_image 
support - This approach is elegant and can be used to add multi-image 
support to other objects. - on the downside I believe VXL core currently 
has a prohibition against multiple inheritance which would need to be 
rewritten to allow interface style mixins.
3. Use an intermediate base class
vil_image_resource -> vil_multi_image_resource -> vil_nitf_image
                                                -> vil_tiff_image
Again, if you can successfully dynamically cast the unwrapped 
vil_image_resource_sptr to a vil_multi_image_resource then you can use 
that API.

Although I still believe that any of these solutions are premature in 
the face of only one vil file format actually handling multiple images.

  These implementations are below.  By doing this,
> subclasses that don't support multiple images per file don't have to do
> anything.
> 
> Regardless, I'm more than happy to defer to your judgement on this issue and
> do not mind removing these functions from vil_image_resource.  I'd do it
> immediately, but don't want to interfere with Joe's on-going development.
> 
> Rob
>