Thank you for getting back with me. I'm glad to hear that you're interested
in getting this resolved. I'll send the patch to you when I've finished.
One major question though:
You had mentioned that MSVC 7.1 uses a 64 bit streamoff, but my research
seems to contradict this. My Visual Studio documentation said only that:
"[streamoff's] representation has at least 32 value bits." That wasn't very
definitive, so I checked the return value of 'std::numeric_limits<
vcl_ios::streamoff >::max()' and it was 2,147,483,647 (2^31 - 1).
Now, this isn't to say that solution #1 can't work. I would just also have
to modify vil_stream_fstream::seek() to do multiple seeks when the desired
position exceeds 'std::numeric_limits< vcl_ios::streamoff >::max()'. The
only problem I see with this is that the function could end up executing as
many as ~2^32 seeks (for the worst case of passing in a desired position of
2^63) -- that might be prohibitively slow. I'm not sure where someone would
get access to a stream that big though (especially on a 32 bit system), so
it should be ok. I.e. this approach would work nicely for all reasonable
cases. What do you think?
->From: Ian Scott [mailto:ian.m.scott@...]
->Sent: Friday, June 24, 2005 4:55 AM
->To: Rob Radtke
->Subject: Re: [Vxl-maintainers] VIL and large files
->Rob Radtke wrote:
->> Hi everyone,
->> I've been using VIL for a while now and had a
->question/comment for you
->> Unless I'm missing something, it appears as if VIL is
->unable to handle
->> image files larger than 2^31-1 bytes (i.e.. the largest
->number that can
->> be represented by vil_streampos which is typedef'd to be a long
->> int). The reason I noticed this in the first place was
->because I ran
->> across a NITF file that exceeded these bounds. The NITF
->spec allows for
->> files as big as 10^12-1 (999,999,999,999) bytes.
->> So, it seems as if this limitation is caused by the
->combination of two
->> 1) vil_streampos is defined to be long int -- long int is
->32 bits on my
->> platform (Windows).
->> 2) vil_stream has no API for iterative seeking (i.e..
->seeking from the
->> current position as opposed to the start of the stream).
->> It seems like we could overcome this limitation by
->remedying either #1
->> or #2. I took a crack at #1 in my working directory by
->> typedef (of vil_streampos) to this:
->> #if VXL_HAS_INT_64
->> typedef vxl_int_64 vil_streampos;
->> #else //VXL_HAS_INT_64
->> typedef long int vil_streampos;
->> #endif //VXL_HAS_INT_64
->> That caused lots of compiler warnings elsewhere in VIL code
->> int, long int and vil_streampos are occasionally used
->> Some of these issues were easy to resolve and other would
->> require API changes. Also, vil_stream_fstream::seek() (which uses
->> std::fstream under the hood) would have to be modified. My
->> that was to have it do multiple seeks (from current
->position) for cases
->> where the desired seek offset exceeded the size of
->> I never got this far.
->> The other option would be to attack #2 by adding a
->parameter similar to
->> ios_base::seekdir to vil_stream::seek(). Then client code
->could get a
->> larger seek, by iteratively seeking. This would solve the
->> somewhat nicely (files up to 10^12 bytes) but would be
->> REALLY large files where you may have to seek a ridiculous
->> times (worst case: 2^32) to get to the end of the file. Of
->> cares about files that are that big if they even exist.
->> requires a (small) change to the core API.
->> I suppose a third option would be to add a virtual seek64()
->> that did the right thing for all subclasses. I haven't
->> this idea through too thoroughly.
->> At any rate, does anyone have any opinions on this? Is
->> (easier) option I'm missing. If I were to pursue a
->solution, would you
->> want it incorporated into VIL? Thank you all for your
->> Rob Radtke
->I think 1 sounds like the least hassle, and the best long
->However, the code currently assumes that vil_streampos has
->the same type
->as vcl_ios::streamoff. If it is generally the case that
->64 bit will define vcl_ios::streamoff as 64 bit, then there
->a problem. This is true for MSVC 7.1, and (according to gcc-bugzilla)
->for gcc 3.4.
->Alternatively, just to be careful, vil_stream could
->and typedef vcl_ios::streamoff vil_streampos.
->I think the API changes you describe are worth doing - just
->the ints that give errors/warnings to vil_streampos, and they
->away. This should include the calls to vil_stream_fstream::seek.
->Now, the next question is, are you okay with only reading
->files on platforms with 64 bits?
->Anyway, I'd be happy to accept such a patch with those fixes.