geotools-gt2-users

[Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

I need to process huge rasters with GeoTools, but it seems I've found
its limit :-(

INFO: Unable to find crs, continuing with default WGS4 CRS
Exception in thread "main" java.lang.IllegalArgumentException:
Dimensions (width=59000 height=46000) are too large
	at java.awt.image.SampleModel.<init>(SampleModel.java:112)
	at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
	at javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
	at javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
	at org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
	at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
	at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
	at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24)


The relevant code follows:

		File file = new File("d:\\bigtab\\mosaic.asc");
		ArcGridFormat agf = new ArcGridFormat();
		GridCoverageReader reader = agf.getReader(file);

Have you found a similar problem? Is there any workaround?
I see it is a problem with JAI... is there any chance to replace the
"limited" classes with another non-JAIs classes?

Regards,

César Martínez Izquierdo

-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[Christian M. <chr...@nv...>]

What do you want to do with this large image in memory ? 

I never worded with ArgGrid rasters but I would suggest you download 

http://fwtools.maptools.org/ 

and use gdal_retile.py to build tiles and pyramids. Afterwards you can use 
the geotools imagemosaic mdoule to process the tiles from the file system or 
imagemosaic-jdbc to do the same having the tiles in a jdbc database. 

 


César Martínez Izquierdo writes: 

> I need to process huge rasters with GeoTools, but it seems I've found
> its limit :-( 
> 
> INFO: Unable to find crs, continuing with default WGS4 CRS
> Exception in thread "main" java.lang.IllegalArgumentException:
> Dimensions (width=59000 height=46000) are too large
> 	at java.awt.image.SampleModel.<init>(SampleModel.java:112)
> 	at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
> 	at javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
> 	at javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
> 	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
> 	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
> 	at org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
> 	at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
> 	at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
> 	at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24) 
> 
> 
> The relevant code follows: 
> 
> 		File file = new File("d:\\bigtab\\mosaic.asc");
> 		ArcGridFormat agf = new ArcGridFormat();
> 		GridCoverageReader reader = agf.getReader(file); 
> 
> Have you found a similar problem? Is there any workaround?
> I see it is a problem with JAI... is there any chance to replace the
> "limited" classes with another non-JAIs classes? 
> 
> Regards, 
> 
> César Martínez Izquierdo 
> 
> -- 
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>    César Martínez Izquierdo
>    GIS developer
>    -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>    ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>    Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
> 
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay 
> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
> http://p.sf.net/sfu/devconf
> _______________________________________________
> Geotools-gt2-users mailing list
> Geo...@li...
> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
 

Re: [Geotools-gt2-users] Problems reading huge rasters

[Simone G. <sim...@ge...>]

Ciao Cesar,
you ran into a limitation of how images are handle in java. Long story
short width*height cannot be larger than the maximum positive Integer.
However I found a workaround a while ago, I am not sure I ever applied
it to the ascii arcgrid reader.
Which version of geotools are you working with.

Ciao,
Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041  Camaiore (LU)
Italy

phone: +39 0584983027
fax:      +39 0584983027
mob:    +39 333 8128928


http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------



2009/10/2 César Martínez Izquierdo <ces...@gm...>:
> I need to process huge rasters with GeoTools, but it seems I've found
> its limit :-(
>
> INFO: Unable to find crs, continuing with default WGS4 CRS
> Exception in thread "main" java.lang.IllegalArgumentException:
> Dimensions (width=59000 height=46000) are too large
>        at java.awt.image.SampleModel.<init>(SampleModel.java:112)
>        at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
>        at javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
>        at javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
>        at org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
>        at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
>        at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
>        at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24)
>
>
> The relevant code follows:
>
>                File file = new File("d:\\bigtab\\mosaic.asc");
>                ArcGridFormat agf = new ArcGridFormat();
>                GridCoverageReader reader = agf.getReader(file);
>
> Have you found a similar problem? Is there any workaround?
> I see it is a problem with JAI... is there any chance to replace the
> "limited" classes with another non-JAIs classes?
>
> Regards,
>
> César Martínez Izquierdo
>
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   César Martínez Izquierdo
>   GIS developer
>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>   Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
> http://p.sf.net/sfu/devconf
> _______________________________________________
> Geotools-gt2-users mailing list
> Geo...@li...
> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

El día 2 de octubre de 2009 10:53, Simone Giannecchini
<sim...@ge...> escribió:
> Ciao Cesar,
> you ran into a limitation of how images are handle in java. Long story
> short width*height cannot be larger than the maximum positive Integer.
> However I found a workaround a while ago, I am not sure I ever applied
> it to the ascii arcgrid reader.
> Which version of geotools are you working with.

Hello Simone,
now I'm trying with geotools 2.5.7 and I get the same error.
Any clue about this workaround?

Thanks,

César


-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

Hello Christian, and thanks for your quick reply.
I'm writing an application which should be able to deal with such big
data (either in GRID or in TIFF).

I know that there are tools like gdal which are able to build tiles
and so on, but I need my application (written in Java) to be able to
deal with the whole file.

Of course, I do not try to load it in memory, but I wish to read it to
make some geo-processing.

Regards,

César

2009/10/2 Christian Müller <chr...@nv...>:
> What do you want to do with this large image in memory ?
> I never worded with ArgGrid rasters but I would suggest you download
> http://fwtools.maptools.org/
> and use gdal_retile.py to build tiles and pyramids. Afterwards you can use
> the geotools imagemosaic mdoule to process the tiles from the file system or
> imagemosaic-jdbc to do the same having the tiles in a jdbc database.
>
>
>
> César Martínez Izquierdo writes:
>>
>> I need to process huge rasters with GeoTools, but it seems I've found
>> its limit :-(
>> INFO: Unable to find crs, continuing with default WGS4 CRS
>> Exception in thread "main" java.lang.IllegalArgumentException:
>> Dimensions (width=59000 height=46000) are too large
>>        at java.awt.image.SampleModel.<init>(SampleModel.java:112)
>>        at
>> java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
>>        at
>> javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
>>        at
>> javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
>>        at
>> it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
>>        at
>> it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
>>        at
>> org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
>>        at
>> org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
>>        at
>> org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
>>        at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24)
>>
>> The relevant code follows:
>>                File file = new File("d:\\bigtab\\mosaic.asc");
>>                ArcGridFormat agf = new ArcGridFormat();
>>                GridCoverageReader reader = agf.getReader(file);
>> Have you found a similar problem? Is there any workaround?
>> I see it is a problem with JAI... is there any chance to replace the
>> "limited" classes with another non-JAIs classes?
>> Regards,
>> César Martínez Izquierdo
>> --
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>   César Martínez Izquierdo
>>   GIS developer
>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>   Universitat Autònoma de Barcelona (SPAIN)
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>
>> ------------------------------------------------------------------------------
>> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
>> is the only developer event you need to attend this year. Jumpstart your
>> developing skills, take BlackBerry mobile applications to market and stay
>> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
>> http://p.sf.net/sfu/devconf
>> _______________________________________________
>> Geotools-gt2-users mailing list
>> Geo...@li...
>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>
>
>



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[Simone G. <sim...@ge...>]

Ciao Cesar,
I just checked the arcgrid reader and yeah I can apply the fix that
should make your code work.

We are on the edge of a new geotools release, so that fix should be
available soon, but if you want to test it right away, I can provide
you with a fixed jar.

Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041  Camaiore (LU)
Italy

phone: +39 0584983027
fax:      +39 0584983027
mob:    +39 333 8128928


http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------



2009/10/2 César Martínez Izquierdo <ces...@gm...>:
> Hello Christian, and thanks for your quick reply.
> I'm writing an application which should be able to deal with such big
> data (either in GRID or in TIFF).
>
> I know that there are tools like gdal which are able to build tiles
> and so on, but I need my application (written in Java) to be able to
> deal with the whole file.
>
> Of course, I do not try to load it in memory, but I wish to read it to
> make some geo-processing.
>
> Regards,
>
> César
>
> 2009/10/2 Christian Müller <chr...@nv...>:
>> What do you want to do with this large image in memory ?
>> I never worded with ArgGrid rasters but I would suggest you download
>> http://fwtools.maptools.org/
>> and use gdal_retile.py to build tiles and pyramids. Afterwards you can use
>> the geotools imagemosaic mdoule to process the tiles from the file system or
>> imagemosaic-jdbc to do the same having the tiles in a jdbc database.
>>
>>
>>
>> César Martínez Izquierdo writes:
>>>
>>> I need to process huge rasters with GeoTools, but it seems I've found
>>> its limit :-(
>>> INFO: Unable to find crs, continuing with default WGS4 CRS
>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>> Dimensions (width=59000 height=46000) are too large
>>>        at java.awt.image.SampleModel.<init>(SampleModel.java:112)
>>>        at
>>> java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
>>>        at
>>> javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
>>>        at
>>> javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
>>>        at
>>> it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
>>>        at
>>> it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
>>>        at
>>> org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
>>>        at
>>> org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
>>>        at
>>> org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
>>>        at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24)
>>>
>>> The relevant code follows:
>>>                File file = new File("d:\\bigtab\\mosaic.asc");
>>>                ArcGridFormat agf = new ArcGridFormat();
>>>                GridCoverageReader reader = agf.getReader(file);
>>> Have you found a similar problem? Is there any workaround?
>>> I see it is a problem with JAI... is there any chance to replace the
>>> "limited" classes with another non-JAIs classes?
>>> Regards,
>>> César Martínez Izquierdo
>>> --
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>   César Martínez Izquierdo
>>>   GIS developer
>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>   Universitat Autònoma de Barcelona (SPAIN)
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>
>>> ------------------------------------------------------------------------------
>>> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
>>> is the only developer event you need to attend this year. Jumpstart your
>>> developing skills, take BlackBerry mobile applications to market and stay
>>> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
>>> http://p.sf.net/sfu/devconf
>>> _______________________________________________
>>> Geotools-gt2-users mailing list
>>> Geo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>
>>
>>
>
>
>
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   César Martínez Izquierdo
>   GIS developer
>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>   Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
> http://p.sf.net/sfu/devconf
> _______________________________________________
> Geotools-gt2-users mailing list
> Geo...@li...
> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

El día 2 de octubre de 2009 11:04, Simone Giannecchini
<sim...@ge...> escribió:
> Ciao Cesar,
> I just checked the arcgrid reader and yeah I can apply the fix that
> should make your code work.
>
> We are on the edge of a new geotools release, so that fix should be
> available soon, but if you want to test it right away, I can provide
> you with a fixed jar.

Hello again,
It will be great if you could provide that fixed jar.

Regards,

César




>
> Simone.
> -------------------------------------------------------
> Ing. Simone Giannecchini
> GeoSolutions S.A.S.
> Founder - Software Engineer
> Via Carignoni 51
> 55041  Camaiore (LU)
> Italy
>
> phone: +39 0584983027
> fax:      +39 0584983027
> mob:    +39 333 8128928
>
>
> http://www.geo-solutions.it
> http://geo-solutions.blogspot.com/
> http://simboss.blogspot.com/
> http://www.linkedin.com/in/simonegiannecchini
>
> -------------------------------------------------------
>
>
>
> 2009/10/2 César Martínez Izquierdo <ces...@gm...>:
>> Hello Christian, and thanks for your quick reply.
>> I'm writing an application which should be able to deal with such big
>> data (either in GRID or in TIFF).
>>
>> I know that there are tools like gdal which are able to build tiles
>> and so on, but I need my application (written in Java) to be able to
>> deal with the whole file.
>>
>> Of course, I do not try to load it in memory, but I wish to read it to
>> make some geo-processing.
>>
>> Regards,
>>
>> César
>>
>> 2009/10/2 Christian Müller <chr...@nv...>:
>>> What do you want to do with this large image in memory ?
>>> I never worded with ArgGrid rasters but I would suggest you download
>>> http://fwtools.maptools.org/
>>> and use gdal_retile.py to build tiles and pyramids. Afterwards you can use
>>> the geotools imagemosaic mdoule to process the tiles from the file system or
>>> imagemosaic-jdbc to do the same having the tiles in a jdbc database.
>>>
>>>
>>>
>>> César Martínez Izquierdo writes:
>>>>
>>>> I need to process huge rasters with GeoTools, but it seems I've found
>>>> its limit :-(
>>>> INFO: Unable to find crs, continuing with default WGS4 CRS
>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>> Dimensions (width=59000 height=46000) are too large
>>>>        at java.awt.image.SampleModel.<init>(SampleModel.java:112)
>>>>        at
>>>> java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
>>>>        at
>>>> javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
>>>>        at
>>>> javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
>>>>        at
>>>> it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
>>>>        at
>>>> it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
>>>>        at
>>>> org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
>>>>        at
>>>> org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
>>>>        at
>>>> org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
>>>>        at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24)
>>>>
>>>> The relevant code follows:
>>>>                File file = new File("d:\\bigtab\\mosaic.asc");
>>>>                ArcGridFormat agf = new ArcGridFormat();
>>>>                GridCoverageReader reader = agf.getReader(file);
>>>> Have you found a similar problem? Is there any workaround?
>>>> I see it is a problem with JAI... is there any chance to replace the
>>>> "limited" classes with another non-JAIs classes?
>>>> Regards,
>>>> César Martínez Izquierdo
>>>> --
>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>   César Martínez Izquierdo
>>>>   GIS developer
>>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>>   Universitat Autònoma de Barcelona (SPAIN)
>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
>>>> is the only developer event you need to attend this year. Jumpstart your
>>>> developing skills, take BlackBerry mobile applications to market and stay
>>>> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
>>>> http://p.sf.net/sfu/devconf
>>>> _______________________________________________
>>>> Geotools-gt2-users mailing list
>>>> Geo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>>
>>>
>>>
>>
>>
>>
>> --
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>   César Martínez Izquierdo
>>   GIS developer
>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>   Universitat Autònoma de Barcelona (SPAIN)
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>
>> ------------------------------------------------------------------------------
>> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
>> is the only developer event you need to attend this year. Jumpstart your
>> developing skills, take BlackBerry mobile applications to market and stay
>> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
>> http://p.sf.net/sfu/devconf
>> _______________________________________________
>> Geotools-gt2-users mailing list
>> Geo...@li...
>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>
>



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

Hello Simone,
I've tried with the new JAR, but it still fails:

02-oct-2009 11:35:57 org.geotools.gce.arcgrid.ArcGridReader
getCoordinateReferenceSystem
INFO: Unable to find crs, continuing with default WGS4 CRS
Exception in thread "main" java.lang.IllegalArgumentException:
Dimensions (width=59000 height=46000) are too large
	at java.awt.image.SampleModel.<init>(SampleModel.java:112)
	at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
	at javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
	at javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
	at org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
	at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
	at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
	at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24)


Tell me if I can help in any way. I'm not used to GeoTools internals,
but it doesn't frighten me.

Best regards,

César



El día 2 de octubre de 2009 11:17, Simone Giannecchini
<sim...@ge...> escribió:
> It is not something you can do in your code, it must be done at a lower level.
> Do this, replace the imageio-ext-arcgrid jar from your project with
> the one attached and let meknow how it goes.
>
> Simone.
> -------------------------------------------------------
> Ing. Simone Giannecchini
> GeoSolutions S.A.S.
> Founder - Software Engineer
> Via Carignoni 51
> 55041  Camaiore (LU)
> Italy
>
> phone: +39 0584983027
> fax:      +39 0584983027
> mob:    +39 333 8128928
>
>
> http://www.geo-solutions.it
> http://geo-solutions.blogspot.com/
> http://simboss.blogspot.com/
> http://www.linkedin.com/in/simonegiannecchini
>
> -------------------------------------------------------
>
>
>
> 2009/10/2 César Martínez Izquierdo <ces...@gm...>:
>> El día 2 de octubre de 2009 10:53, Simone Giannecchini
>> <sim...@ge...> escribió:
>>> Ciao Cesar,
>>> you ran into a limitation of how images are handle in java. Long story
>>> short width*height cannot be larger than the maximum positive Integer.
>>> However I found a workaround a while ago, I am not sure I ever applied
>>> it to the ascii arcgrid reader.
>>> Which version of geotools are you working with.
>>
>> Hello Simone,
>> now I'm trying with geotools 2.5.7 and I get the same error.
>> Any clue about this workaround?
>>
>> Thanks,
>>
>> César
>>
>>
>> --
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>   César Martínez Izquierdo
>>   GIS developer
>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>   Universitat Autònoma de Barcelona (SPAIN)
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>
>> ------------------------------------------------------------------------------
>> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
>> is the only developer event you need to attend this year. Jumpstart your
>> developing skills, take BlackBerry mobile applications to market and stay
>> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
>> http://p.sf.net/sfu/devconf
>> _______________________________________________
>> Geotools-gt2-users mailing list
>> Geo...@li...
>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>
>



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

Hello again, list.

Just for the record, I could successfully create the ArcGrid reader
using Simone's new code.

Probably, the first JAR file he sent was already correct, but my
runtime environment was taking the old one (I still don't fully
understand why... Andrea suggested that it was a problem with maven
dependences [and probably it was], but this is weird for me as I was
launching my test code from Eclipse, from a Java project which
*didn't* have a pom.xml file, so I didn't expect maven interfering in
the runtime configuration of this project...)

I want to thank Simone for his support and patience.

Now I have another problem... although creating the reader succeeds, I
get a similar error when reading the file.
However, I guess it is not a problem with the driver itself, but
rather a wrong usage of it.

The code comes from Sextante-GeoTools bindings:

ArcGridFormat agf = new ArcGridFormat();
GridCoverageReader reader = agf.getReader(file);
GridCoverage2D gc = (GridCoverage2D) reader.read(null);			
PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
Raster raster = img.getAsBufferedImage().getRaster();

And then I get:

05-oct-2009 17:20:56 org.geotools.gce.arcgrid.ArcGridReader
getCoordinateReferenceSystem
INFO: Unable to find crs, continuing with default WGS4 CRS
java.lang.IllegalArgumentException: Dimensions (width=59000
height=46000) are too large
	at java.awt.image.SampleModel.<init>(SampleModel.java:112)
	at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:191)
	at javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:101)
	at javax.media.jai.ComponentSampleModelJAI.createCompatibleSampleModel(ComponentSampleModelJAI.java:219)
	at javax.media.jai.PlanarImage.getAsBufferedImage(PlanarImage.java:2517)
	at javax.media.jai.PlanarImage.getAsBufferedImage(PlanarImage.java:2546)
	at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:39)


The question is whether

  img.getAsBufferedImage().getRaster();

is the right way to access a huge raster, or there are better methods
to perform it.
I know that this is a generic JAI question (nothing Geotools
specifical), but I would really appreciate if someone can point me to
the right solution. A JAI tutorial on this subject would also be
appreciated :-D

We'd like to process huge data sets with Sextante + GeoTools, so I
would really appreciate any clue on this.

Best regards,


César Martínez Izquierdo



El día 2 de octubre de 2009 14:47, Simone Giannecchini
<sim...@ge...> escribió:
> ciao cesar,
> please join #geotools
>
> Simone.
> -------------------------------------------------------
> Ing. Simone Giannecchini
> GeoSolutions S.A.S.
> Founder - Software Engineer
> Via Carignoni 51
> 55041  Camaiore (LU)
> Italy
>
> phone: +39 0584983027
> fax:      +39 0584983027
> mob:    +39 333 8128928
>
>
> http://www.geo-solutions.it
> http://geo-solutions.blogspot.com/
> http://simboss.blogspot.com/
> http://www.linkedin.com/in/simonegiannecchini
>
> -------------------------------------------------------
>
>
>
> 2009/10/2 César Martínez Izquierdo <ces...@gm...>:
>> Hi again Simone,
>>
>> Sorry, but I don't usually connect to IRC. Which server/channel should
>> I join? #osgeo on Freenode, or there is a specifical one for GeoTools?
>> I've joined #osgeo as cesar_izq, and we'll be available until 5pm and
>> wishing to receive your hints for playing with the code.
>>
>> Regards,
>>
>> César
>>
>>
>> El día 2 de octubre de 2009 11:46, Simone Giannecchini
>> <sim...@ge...> escribió:
>>> Ouch, I sent you the wrong jar :-). Please try this one.
>>> If you want to chat about how to play with the code, please, jump on
>>> the IRC channel.
>>>
>>> Simone.
>>> -------------------------------------------------------
>>> Ing. Simone Giannecchini
>>> GeoSolutions S.A.S.
>>> Founder - Software Engineer
>>> Via Carignoni 51
>>> 55041  Camaiore (LU)
>>> Italy
>>>
>>> phone: +39 0584983027
>>> fax:      +39 0584983027
>>> mob:    +39 333 8128928
>>>
>>>
>>> http://www.geo-solutions.it
>>> http://geo-solutions.blogspot.com/
>>> http://simboss.blogspot.com/
>>> http://www.linkedin.com/in/simonegiannecchini
>>>
>>> -------------------------------------------------------
>>>
>>>
>>>
>>> 2009/10/2 César Martínez Izquierdo <ces...@gm...>:
>>>> Hello Simone,
>>>> I've tried with the new JAR, but it still fails:
>>>>
>>>> 02-oct-2009 11:35:57 org.geotools.gce.arcgrid.ArcGridReader
>>>> getCoordinateReferenceSystem
>>>> INFO: Unable to find crs, continuing with default WGS4 CRS
>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>> Dimensions (width=59000 height=46000) are too large
>>>>        at java.awt.image.SampleModel.<init>(SampleModel.java:112)
>>>>        at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
>>>>        at javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
>>>>        at javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
>>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
>>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
>>>>        at org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
>>>>        at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
>>>>        at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
>>>>        at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24)
>>>>
>>>>
>>>> Tell me if I can help in any way. I'm not used to GeoTools internals,
>>>> but it doesn't frighten me.
>>>>
>>>> Best regards,
>>>>
>>>> César
>>>>
>>>>
>>>>
>>>> El día 2 de octubre de 2009 11:17, Simone Giannecchini
>>>> <sim...@ge...> escribió:
>>>>> It is not something you can do in your code, it must be done at a lower level.
>>>>> Do this, replace the imageio-ext-arcgrid jar from your project with
>>>>> the one attached and let meknow how it goes.
>>>>>
>>>>> Simone.
>>>>> -------------------------------------------------------
>>>>> Ing. Simone Giannecchini
>>>>> GeoSolutions S.A.S.
>>>>> Founder - Software Engineer
>>>>> Via Carignoni 51
>>>>> 55041  Camaiore (LU)
>>>>> Italy
>>>>>
>>>>> phone: +39 0584983027
>>>>> fax:      +39 0584983027
>>>>> mob:    +39 333 8128928
>>>>>
>>>>>
>>>>> http://www.geo-solutions.it
>>>>> http://geo-solutions.blogspot.com/
>>>>> http://simboss.blogspot.com/
>>>>> http://www.linkedin.com/in/simonegiannecchini
>>>>>
>>>>> -------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>> 2009/10/2 César Martínez Izquierdo <ces...@gm...>:
>>>>>> El día 2 de octubre de 2009 10:53, Simone Giannecchini
>>>>>> <sim...@ge...> escribió:
>>>>>>> Ciao Cesar,
>>>>>>> you ran into a limitation of how images are handle in java. Long story
>>>>>>> short width*height cannot be larger than the maximum positive Integer.
>>>>>>> However I found a workaround a while ago, I am not sure I ever applied
>>>>>>> it to the ascii arcgrid reader.
>>>>>>> Which version of geotools are you working with.
>>>>>>
>>>>>> Hello Simone,
>>>>>> now I'm trying with geotools 2.5.7 and I get the same error.
>>>>>> Any clue about this workaround?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> César
>>>>>>
>>>>>>
>>>>>> --
>>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>>   César Martínez Izquierdo
>>>>>>   GIS developer
>>>>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>>>>   Universitat Autònoma de Barcelona (SPAIN)
>>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
>>>>>> is the only developer event you need to attend this year. Jumpstart your
>>>>>> developing skills, take BlackBerry mobile applications to market and stay
>>>>>> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
>>>>>> http://p.sf.net/sfu/devconf
>>>>>> _______________________________________________
>>>>>> Geotools-gt2-users mailing list
>>>>>> Geo...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>   César Martínez Izquierdo
>>>>   GIS developer
>>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>>   Universitat Autònoma de Barcelona (SPAIN)
>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>
>>>>
>>>
>>
>>
>>
>> --
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>   César Martínez Izquierdo
>>   GIS developer
>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>   Universitat Autònoma de Barcelona (SPAIN)
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>
>>
>



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[Simone G. <sim...@ge...>]

Ciao Cesar,
please, read below....
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041  Camaiore (LU)
Italy

phone: +39 0584983027
fax:      +39 0584983027
mob:    +39 333 8128928


http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------



2009/10/5 César Martínez Izquierdo <ces...@gm...>:
> Hello again, list.
>
> Just for the record, I could successfully create the ArcGrid reader
> using Simone's new code.
>
> Probably, the first JAR file he sent was already correct, but my
> runtime environment was taking the old one (I still don't fully
> understand why... Andrea suggested that it was a problem with maven
> dependences [and probably it was], but this is weird for me as I was
> launching my test code from Eclipse, from a Java project which
> *didn't* have a pom.xml file, so I didn't expect maven interfering in
> the runtime configuration of this project...)
>
> I want to thank Simone for his support and patience.

np

>
> Now I have another problem... although creating the reader succeeds, I
> get a similar error when reading the file.
> However, I guess it is not a problem with the driver itself, but
> rather a wrong usage of it.
>
> The code comes from Sextante-GeoTools bindings:
>
> ArcGridFormat agf = new ArcGridFormat();
> GridCoverageReader reader = agf.getReader(file);
> GridCoverage2D gc = (GridCoverage2D) reader.read(null);
> PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
> Raster raster = img.getAsBufferedImage().getRaster();
>

This is probably the worst way to work with a PlanarImage. I dag once
those bindings and I told Victor that he was loading everything into
memory doing that. Actually not only that, that code cobbles the
original raster into a single untiled raster in memory, performing a
full copy! Rather inefficient I would say :-)




> And then I get:
>
> 05-oct-2009 17:20:56 org.geotools.gce.arcgrid.ArcGridReader
> getCoordinateReferenceSystem
> INFO: Unable to find crs, continuing with default WGS4 CRS
> java.lang.IllegalArgumentException: Dimensions (width=59000
> height=46000) are too large
>        at java.awt.image.SampleModel.<init>(SampleModel.java:112)
>        at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:191)
>        at javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:101)
>        at javax.media.jai.ComponentSampleModelJAI.createCompatibleSampleModel(ComponentSampleModelJAI.java:219)
>        at javax.media.jai.PlanarImage.getAsBufferedImage(PlanarImage.java:2517)
>        at javax.media.jai.PlanarImage.getAsBufferedImage(PlanarImage.java:2546)
>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:39)
>
>
> The question is whether
>
>  img.getAsBufferedImage().getRaster();
>
> is the right way to access a huge raster, or there are better methods
> to perform it.
> I know that this is a generic JAI question (nothing Geotools
> specifical), but I would really appreciate if someone can point me to
> the right solution. A JAI tutorial on this subject would also be
> appreciated :-D

https://jaistuff.dev.java.net/docs/jaitutorial.pdf
http://java.sun.com/developer/releases/jai/

the goal is to use iterators to do the processing, or anyway to do it
tile by tile in order to avoid copies and/or loading everything into
memory.

Simone.

>
> We'd like to process huge data sets with Sextante + GeoTools, so I
> would really appreciate any clue on this.
>
> Best regards,
>
>
> César Martínez Izquierdo
>
>
>
> El día 2 de octubre de 2009 14:47, Simone Giannecchini
> <sim...@ge...> escribió:
>> ciao cesar,
>> please join #geotools
>>
>> Simone.
>> -------------------------------------------------------
>> Ing. Simone Giannecchini
>> GeoSolutions S.A.S.
>> Founder - Software Engineer
>> Via Carignoni 51
>> 55041  Camaiore (LU)
>> Italy
>>
>> phone: +39 0584983027
>> fax:      +39 0584983027
>> mob:    +39 333 8128928
>>
>>
>> http://www.geo-solutions.it
>> http://geo-solutions.blogspot.com/
>> http://simboss.blogspot.com/
>> http://www.linkedin.com/in/simonegiannecchini
>>
>> -------------------------------------------------------
>>
>>
>>
>> 2009/10/2 César Martínez Izquierdo <ces...@gm...>:
>>> Hi again Simone,
>>>
>>> Sorry, but I don't usually connect to IRC. Which server/channel should
>>> I join? #osgeo on Freenode, or there is a specifical one for GeoTools?
>>> I've joined #osgeo as cesar_izq, and we'll be available until 5pm and
>>> wishing to receive your hints for playing with the code.
>>>
>>> Regards,
>>>
>>> César
>>>
>>>
>>> El día 2 de octubre de 2009 11:46, Simone Giannecchini
>>> <sim...@ge...> escribió:
>>>> Ouch, I sent you the wrong jar :-). Please try this one.
>>>> If you want to chat about how to play with the code, please, jump on
>>>> the IRC channel.
>>>>
>>>> Simone.
>>>> -------------------------------------------------------
>>>> Ing. Simone Giannecchini
>>>> GeoSolutions S.A.S.
>>>> Founder - Software Engineer
>>>> Via Carignoni 51
>>>> 55041  Camaiore (LU)
>>>> Italy
>>>>
>>>> phone: +39 0584983027
>>>> fax:      +39 0584983027
>>>> mob:    +39 333 8128928
>>>>
>>>>
>>>> http://www.geo-solutions.it
>>>> http://geo-solutions.blogspot.com/
>>>> http://simboss.blogspot.com/
>>>> http://www.linkedin.com/in/simonegiannecchini
>>>>
>>>> -------------------------------------------------------
>>>>
>>>>
>>>>
>>>> 2009/10/2 César Martínez Izquierdo <ces...@gm...>:
>>>>> Hello Simone,
>>>>> I've tried with the new JAR, but it still fails:
>>>>>
>>>>> 02-oct-2009 11:35:57 org.geotools.gce.arcgrid.ArcGridReader
>>>>> getCoordinateReferenceSystem
>>>>> INFO: Unable to find crs, continuing with default WGS4 CRS
>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>> Dimensions (width=59000 height=46000) are too large
>>>>>        at java.awt.image.SampleModel.<init>(SampleModel.java:112)
>>>>>        at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:128)
>>>>>        at javax.media.jai.ComponentSampleModelJAI.<init>(ComponentSampleModelJAI.java:73)
>>>>>        at javax.media.jai.FloatDoubleColorModel.createCompatibleSampleModel(FloatDoubleColorModel.java:1003)
>>>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.initializeReader(AsciiGridsImageReader.java:344)
>>>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.setInput(AsciiGridsImageReader.java:320)
>>>>>        at org.geotools.gce.arcgrid.ArcGridReader.<init>(ArcGridReader.java:184)
>>>>>        at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:157)
>>>>>        at org.geotools.gce.arcgrid.ArcGridFormat.getReader(ArcGridFormat.java:106)
>>>>>        at etclusi.tests.gt.TestArcGrid.main(TestArcGrid.java:24)
>>>>>
>>>>>
>>>>> Tell me if I can help in any way. I'm not used to GeoTools internals,
>>>>> but it doesn't frighten me.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> César
>>>>>
>>>>>
>>>>>
>>>>> El día 2 de octubre de 2009 11:17, Simone Giannecchini
>>>>> <sim...@ge...> escribió:
>>>>>> It is not something you can do in your code, it must be done at a lower level.
>>>>>> Do this, replace the imageio-ext-arcgrid jar from your project with
>>>>>> the one attached and let meknow how it goes.
>>>>>>
>>>>>> Simone.
>>>>>> -------------------------------------------------------
>>>>>> Ing. Simone Giannecchini
>>>>>> GeoSolutions S.A.S.
>>>>>> Founder - Software Engineer
>>>>>> Via Carignoni 51
>>>>>> 55041  Camaiore (LU)
>>>>>> Italy
>>>>>>
>>>>>> phone: +39 0584983027
>>>>>> fax:      +39 0584983027
>>>>>> mob:    +39 333 8128928
>>>>>>
>>>>>>
>>>>>> http://www.geo-solutions.it
>>>>>> http://geo-solutions.blogspot.com/
>>>>>> http://simboss.blogspot.com/
>>>>>> http://www.linkedin.com/in/simonegiannecchini
>>>>>>
>>>>>> -------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2009/10/2 César Martínez Izquierdo <ces...@gm...>:
>>>>>>> El día 2 de octubre de 2009 10:53, Simone Giannecchini
>>>>>>> <sim...@ge...> escribió:
>>>>>>>> Ciao Cesar,
>>>>>>>> you ran into a limitation of how images are handle in java. Long story
>>>>>>>> short width*height cannot be larger than the maximum positive Integer.
>>>>>>>> However I found a workaround a while ago, I am not sure I ever applied
>>>>>>>> it to the ascii arcgrid reader.
>>>>>>>> Which version of geotools are you working with.
>>>>>>>
>>>>>>> Hello Simone,
>>>>>>> now I'm trying with geotools 2.5.7 and I get the same error.
>>>>>>> Any clue about this workaround?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> César
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>>>   César Martínez Izquierdo
>>>>>>>   GIS developer
>>>>>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>>>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>>>>>   Universitat Autònoma de Barcelona (SPAIN)
>>>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
>>>>>>> is the only developer event you need to attend this year. Jumpstart your
>>>>>>> developing skills, take BlackBerry mobile applications to market and stay
>>>>>>> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
>>>>>>> http://p.sf.net/sfu/devconf
>>>>>>> _______________________________________________
>>>>>>> Geotools-gt2-users mailing list
>>>>>>> Geo...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>   César Martínez Izquierdo
>>>>>   GIS developer
>>>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>>>   Universitat Autònoma de Barcelona (SPAIN)
>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>   César Martínez Izquierdo
>>>   GIS developer
>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>   Universitat Autònoma de Barcelona (SPAIN)
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>
>>>
>>
>
>
>
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   César Martínez Izquierdo
>   GIS developer
>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>   Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
> http://p.sf.net/sfu/devconf
> _______________________________________________
> Geotools-gt2-users mailing list
> Geo...@li...
> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

Hi again,
see my response bellow:

El día 5 de octubre de 2009 22:24, Simone Giannecchini
<sim...@ge...> escribió:
> Ciao Cesar,
[SNIP]
> This is probably the worst way to work with a PlanarImage. I dag once
> those bindings and I told Victor that he was loading everything into
> memory doing that. Actually not only that, that code cobbles the
> original raster into a single untiled raster in memory, performing a
> full copy! Rather inefficient I would say :-)
[SNIP]
> the goal is to use iterators to do the processing, or anyway to do it
> tile by tile in order to avoid copies and/or loading everything into
> memory.
>
> Simone.


Thanks a lot for the links and suggestions. I've read this
documentation, and if I've correctly understood there are two
different possible approaches:
- Using iterators
- Using tiles

I've tried both of them, but I've also found an error in the process.
As I'm a JAI and GeoTools beginner, I'm not sure if there is a problem
in the driver or I misunderstood documentation.

I paste relevant code bellow:

----------------- ITERATOR APPROACH -----
	ArcGridFormat agf = new ArcGridFormat();
	GridCoverageReader reader = agf.getReader(file);
	GridCoverage2D gc = (GridCoverage2D) reader.read(null);
	PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
	RectIter iter = RectIterFactory.create(img, null);
	iter.startBands();
    while (!iter.finishedBands()) {
        iter.startLines();
        while (!iter.finishedLines()) {
            iter.startPixels();
            while (!iter.finishedPixels()) {
            	int value = iter.getSample();
            	if (value!=9999 && value!=-9999) {
            		System.out.println(value);
            	}
                iter.nextPixel();
            }
            iter.nextLine();
        }
        iter.nextBand();
    }

-----------------

After some time running, I get the following exception:
java.lang.IllegalArgumentException: pos < flushedPos!
	at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
	at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
	at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
	at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
	at javax.media.jai.OpImage.getTile(OpImage.java:1129)
	at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
	at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:48)


I have also tried with tiles, but I get the same error.
I assume in this code that the tiles are automatically managed by the
PlanarImage and are not kept in memory (except the tiles that I ask
for and I keep referencing), but I'm not sure if this assumption is
correct...

---------------------- TILES APPROACH --------------------

	ArcGridFormat agf = new ArcGridFormat();
	GridCoverageReader reader = agf.getReader(file);
	GridCoverage2D gc = (GridCoverage2D) reader.read(null);
	PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
	
	for (int j=img.getMinTileY(), maxTileY = img.getMaxTileY();
j<=maxTileY ; j++) {
		for (int i=img.getMinTileX(), maxTileX = img.getMaxTileX();
i<=maxTileX; i++) {
			Raster tile = img.getTile(i, j);
			for (int y=tile.getMinY(), maxY = tile.getHeight()+y; y<maxY; y++) {
				for (int x=tile.getMinX(), maxX = tile.getWidth()+x; x<maxX; x++) {		
					int value = tile.getSample(x, y, 0); // assume band 0
						System.out.println("x: "+x+", y: "+y+", value: "+ value);

				}
			}		
		}
	}

-------------------------------

Again, after a while:

java.lang.IllegalArgumentException: pos < flushedPos!
	at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
	at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
	at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
	at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
	at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
	at javax.media.jai.OpImage.getTile(OpImage.java:1129)
	at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
	at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:70)



So I don't know if there is an error in my code, or it is a problem
with the driver.

By the way, I've also seen all these methods "getDataBlock"
(deprecated), "evaluate"... in the GridCoverage interface, and I
wonder which is the preferred/intended way to access the pixel values
in GeoTools.

Best regards,

César Martínez Izquierdo



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[Simone G. <sim...@ge...>]

Ciao Cesar,
is there any chance that you can make some test data available.

Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041  Camaiore (LU)
Italy

phone: +39 0584983027
fax:      +39 0584983027
mob:    +39 333 8128928


http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------



2009/10/7 César Martínez Izquierdo <ces...@gm...>:
> Hi again,
> see my response bellow:
>
> El día 5 de octubre de 2009 22:24, Simone Giannecchini
> <sim...@ge...> escribió:
>> Ciao Cesar,
> [SNIP]
>> This is probably the worst way to work with a PlanarImage. I dag once
>> those bindings and I told Victor that he was loading everything into
>> memory doing that. Actually not only that, that code cobbles the
>> original raster into a single untiled raster in memory, performing a
>> full copy! Rather inefficient I would say :-)
> [SNIP]
>> the goal is to use iterators to do the processing, or anyway to do it
>> tile by tile in order to avoid copies and/or loading everything into
>> memory.
>>
>> Simone.
>
>
> Thanks a lot for the links and suggestions. I've read this
> documentation, and if I've correctly understood there are two
> different possible approaches:
> - Using iterators
> - Using tiles
>
> I've tried both of them, but I've also found an error in the process.
> As I'm a JAI and GeoTools beginner, I'm not sure if there is a problem
> in the driver or I misunderstood documentation.
>
> I paste relevant code bellow:
>
> ----------------- ITERATOR APPROACH -----
>        ArcGridFormat agf = new ArcGridFormat();
>        GridCoverageReader reader = agf.getReader(file);
>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>        RectIter iter = RectIterFactory.create(img, null);
>        iter.startBands();
>    while (!iter.finishedBands()) {
>        iter.startLines();
>        while (!iter.finishedLines()) {
>            iter.startPixels();
>            while (!iter.finishedPixels()) {
>                int value = iter.getSample();
>                if (value!=9999 && value!=-9999) {
>                        System.out.println(value);
>                }
>                iter.nextPixel();
>            }
>            iter.nextLine();
>        }
>        iter.nextBand();
>    }
>
> -----------------
>
> After some time running, I get the following exception:
> java.lang.IllegalArgumentException: pos < flushedPos!
>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:48)
>
>
> I have also tried with tiles, but I get the same error.
> I assume in this code that the tiles are automatically managed by the
> PlanarImage and are not kept in memory (except the tiles that I ask
> for and I keep referencing), but I'm not sure if this assumption is
> correct...
>
> ---------------------- TILES APPROACH --------------------
>
>        ArcGridFormat agf = new ArcGridFormat();
>        GridCoverageReader reader = agf.getReader(file);
>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>
>        for (int j=img.getMinTileY(), maxTileY = img.getMaxTileY();
> j<=maxTileY ; j++) {
>                for (int i=img.getMinTileX(), maxTileX = img.getMaxTileX();
> i<=maxTileX; i++) {
>                        Raster tile = img.getTile(i, j);
>                        for (int y=tile.getMinY(), maxY = tile.getHeight()+y; y<maxY; y++) {
>                                for (int x=tile.getMinX(), maxX = tile.getWidth()+x; x<maxX; x++) {
>                                        int value = tile.getSample(x, y, 0); // assume band 0
>                                                System.out.println("x: "+x+", y: "+y+", value: "+ value);
>
>                                }
>                        }
>                }
>        }
>
> -------------------------------
>
> Again, after a while:
>
> java.lang.IllegalArgumentException: pos < flushedPos!
>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:70)
>
>
>
> So I don't know if there is an error in my code, or it is a problem
> with the driver.
>
> By the way, I've also seen all these methods "getDataBlock"
> (deprecated), "evaluate"... in the GridCoverage interface, and I
> wonder which is the preferred/intended way to access the pixel values
> in GeoTools.
>
> Best regards,
>
> César Martínez Izquierdo
>
>
>
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   César Martínez Izquierdo
>   GIS developer
>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>   Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Geotools-gt2-users mailing list
> Geo...@li...
> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

Hello Simone,
yes, I can provide test data. However, note that it is a 14,5 GB
raster, so we have to find the way to share it.
Any idea?

Regards,

César

El día 7 de octubre de 2009 16:36, Simone Giannecchini
<sim...@ge...> escribió:
> Ciao Cesar,
> is there any chance that you can make some test data available.
>
> Simone.
> -------------------------------------------------------
> Ing. Simone Giannecchini
> GeoSolutions S.A.S.
> Founder - Software Engineer
> Via Carignoni 51
> 55041  Camaiore (LU)
> Italy
>
> phone: +39 0584983027
> fax:      +39 0584983027
> mob:    +39 333 8128928
>
>
> http://www.geo-solutions.it
> http://geo-solutions.blogspot.com/
> http://simboss.blogspot.com/
> http://www.linkedin.com/in/simonegiannecchini
>
> -------------------------------------------------------
>
>
>
> 2009/10/7 César Martínez Izquierdo <ces...@gm...>:
>> Hi again,
>> see my response bellow:
>>
>> El día 5 de octubre de 2009 22:24, Simone Giannecchini
>> <sim...@ge...> escribió:
>>> Ciao Cesar,
>> [SNIP]
>>> This is probably the worst way to work with a PlanarImage. I dag once
>>> those bindings and I told Victor that he was loading everything into
>>> memory doing that. Actually not only that, that code cobbles the
>>> original raster into a single untiled raster in memory, performing a
>>> full copy! Rather inefficient I would say :-)
>> [SNIP]
>>> the goal is to use iterators to do the processing, or anyway to do it
>>> tile by tile in order to avoid copies and/or loading everything into
>>> memory.
>>>
>>> Simone.
>>
>>
>> Thanks a lot for the links and suggestions. I've read this
>> documentation, and if I've correctly understood there are two
>> different possible approaches:
>> - Using iterators
>> - Using tiles
>>
>> I've tried both of them, but I've also found an error in the process.
>> As I'm a JAI and GeoTools beginner, I'm not sure if there is a problem
>> in the driver or I misunderstood documentation.
>>
>> I paste relevant code bellow:
>>
>> ----------------- ITERATOR APPROACH -----
>>        ArcGridFormat agf = new ArcGridFormat();
>>        GridCoverageReader reader = agf.getReader(file);
>>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>>        RectIter iter = RectIterFactory.create(img, null);
>>        iter.startBands();
>>    while (!iter.finishedBands()) {
>>        iter.startLines();
>>        while (!iter.finishedLines()) {
>>            iter.startPixels();
>>            while (!iter.finishedPixels()) {
>>                int value = iter.getSample();
>>                if (value!=9999 && value!=-9999) {
>>                        System.out.println(value);
>>                }
>>                iter.nextPixel();
>>            }
>>            iter.nextLine();
>>        }
>>        iter.nextBand();
>>    }
>>
>> -----------------
>>
>> After some time running, I get the following exception:
>> java.lang.IllegalArgumentException: pos < flushedPos!
>>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:48)
>>
>>
>> I have also tried with tiles, but I get the same error.
>> I assume in this code that the tiles are automatically managed by the
>> PlanarImage and are not kept in memory (except the tiles that I ask
>> for and I keep referencing), but I'm not sure if this assumption is
>> correct...
>>
>> ---------------------- TILES APPROACH --------------------
>>
>>        ArcGridFormat agf = new ArcGridFormat();
>>        GridCoverageReader reader = agf.getReader(file);
>>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>>
>>        for (int j=img.getMinTileY(), maxTileY = img.getMaxTileY();
>> j<=maxTileY ; j++) {
>>                for (int i=img.getMinTileX(), maxTileX = img.getMaxTileX();
>> i<=maxTileX; i++) {
>>                        Raster tile = img.getTile(i, j);
>>                        for (int y=tile.getMinY(), maxY = tile.getHeight()+y; y<maxY; y++) {
>>                                for (int x=tile.getMinX(), maxX = tile.getWidth()+x; x<maxX; x++) {
>>                                        int value = tile.getSample(x, y, 0); // assume band 0
>>                                                System.out.println("x: "+x+", y: "+y+", value: "+ value);
>>
>>                                }
>>                        }
>>                }
>>        }
>>
>> -------------------------------
>>
>> Again, after a while:
>>
>> java.lang.IllegalArgumentException: pos < flushedPos!
>>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:70)
>>
>>
>>
>> So I don't know if there is an error in my code, or it is a problem
>> with the driver.
>>
>> By the way, I've also seen all these methods "getDataBlock"
>> (deprecated), "evaluate"... in the GridCoverage interface, and I
>> wonder which is the preferred/intended way to access the pixel values
>> in GeoTools.
>>
>> Best regards,
>>
>> César Martínez Izquierdo
>>
>>
>>
>> --
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>   César Martínez Izquierdo
>>   GIS developer
>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>   Universitat Autònoma de Barcelona (SPAIN)
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>
>> ------------------------------------------------------------------------------
>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
>> is the only developer event you need to attend this year. Jumpstart your
>> developing skills, take BlackBerry mobile applications to market and stay
>> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>> http://p.sf.net/sfu/devconference
>> _______________________________________________
>> Geotools-gt2-users mailing list
>> Geo...@li...
>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>
>



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[Simone G. <sim...@ge...>]

If it is an esri arcgrid, by targizipping it it should become much
smaller than that.
Do you have an FTP with good bandwidth at hand?

Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041  Camaiore (LU)
Italy

phone: +39 0584983027
fax:      +39 0584983027
mob:    +39 333 8128928


http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------



2009/10/7 César Martínez Izquierdo <ces...@gm...>:
> Hello Simone,
> yes, I can provide test data. However, note that it is a 14,5 GB
> raster, so we have to find the way to share it.
> Any idea?
>
> Regards,
>
> César
>
> El día 7 de octubre de 2009 16:36, Simone Giannecchini
> <sim...@ge...> escribió:
>> Ciao Cesar,
>> is there any chance that you can make some test data available.
>>
>> Simone.
>> -------------------------------------------------------
>> Ing. Simone Giannecchini
>> GeoSolutions S.A.S.
>> Founder - Software Engineer
>> Via Carignoni 51
>> 55041  Camaiore (LU)
>> Italy
>>
>> phone: +39 0584983027
>> fax:      +39 0584983027
>> mob:    +39 333 8128928
>>
>>
>> http://www.geo-solutions.it
>> http://geo-solutions.blogspot.com/
>> http://simboss.blogspot.com/
>> http://www.linkedin.com/in/simonegiannecchini
>>
>> -------------------------------------------------------
>>
>>
>>
>> 2009/10/7 César Martínez Izquierdo <ces...@gm...>:
>>> Hi again,
>>> see my response bellow:
>>>
>>> El día 5 de octubre de 2009 22:24, Simone Giannecchini
>>> <sim...@ge...> escribió:
>>>> Ciao Cesar,
>>> [SNIP]
>>>> This is probably the worst way to work with a PlanarImage. I dag once
>>>> those bindings and I told Victor that he was loading everything into
>>>> memory doing that. Actually not only that, that code cobbles the
>>>> original raster into a single untiled raster in memory, performing a
>>>> full copy! Rather inefficient I would say :-)
>>> [SNIP]
>>>> the goal is to use iterators to do the processing, or anyway to do it
>>>> tile by tile in order to avoid copies and/or loading everything into
>>>> memory.
>>>>
>>>> Simone.
>>>
>>>
>>> Thanks a lot for the links and suggestions. I've read this
>>> documentation, and if I've correctly understood there are two
>>> different possible approaches:
>>> - Using iterators
>>> - Using tiles
>>>
>>> I've tried both of them, but I've also found an error in the process.
>>> As I'm a JAI and GeoTools beginner, I'm not sure if there is a problem
>>> in the driver or I misunderstood documentation.
>>>
>>> I paste relevant code bellow:
>>>
>>> ----------------- ITERATOR APPROACH -----
>>>        ArcGridFormat agf = new ArcGridFormat();
>>>        GridCoverageReader reader = agf.getReader(file);
>>>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>>>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>>>        RectIter iter = RectIterFactory.create(img, null);
>>>        iter.startBands();
>>>    while (!iter.finishedBands()) {
>>>        iter.startLines();
>>>        while (!iter.finishedLines()) {
>>>            iter.startPixels();
>>>            while (!iter.finishedPixels()) {
>>>                int value = iter.getSample();
>>>                if (value!=9999 && value!=-9999) {
>>>                        System.out.println(value);
>>>                }
>>>                iter.nextPixel();
>>>            }
>>>            iter.nextLine();
>>>        }
>>>        iter.nextBand();
>>>    }
>>>
>>> -----------------
>>>
>>> After some time running, I get the following exception:
>>> java.lang.IllegalArgumentException: pos < flushedPos!
>>>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>>>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>>>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>>>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>>>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>>>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>>>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:48)
>>>
>>>
>>> I have also tried with tiles, but I get the same error.
>>> I assume in this code that the tiles are automatically managed by the
>>> PlanarImage and are not kept in memory (except the tiles that I ask
>>> for and I keep referencing), but I'm not sure if this assumption is
>>> correct...
>>>
>>> ---------------------- TILES APPROACH --------------------
>>>
>>>        ArcGridFormat agf = new ArcGridFormat();
>>>        GridCoverageReader reader = agf.getReader(file);
>>>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>>>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>>>
>>>        for (int j=img.getMinTileY(), maxTileY = img.getMaxTileY();
>>> j<=maxTileY ; j++) {
>>>                for (int i=img.getMinTileX(), maxTileX = img.getMaxTileX();
>>> i<=maxTileX; i++) {
>>>                        Raster tile = img.getTile(i, j);
>>>                        for (int y=tile.getMinY(), maxY = tile.getHeight()+y; y<maxY; y++) {
>>>                                for (int x=tile.getMinX(), maxX = tile.getWidth()+x; x<maxX; x++) {
>>>                                        int value = tile.getSample(x, y, 0); // assume band 0
>>>                                                System.out.println("x: "+x+", y: "+y+", value: "+ value);
>>>
>>>                                }
>>>                        }
>>>                }
>>>        }
>>>
>>> -------------------------------
>>>
>>> Again, after a while:
>>>
>>> java.lang.IllegalArgumentException: pos < flushedPos!
>>>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>>>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>>>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>>>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>>>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>>>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>>>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:70)
>>>
>>>
>>>
>>> So I don't know if there is an error in my code, or it is a problem
>>> with the driver.
>>>
>>> By the way, I've also seen all these methods "getDataBlock"
>>> (deprecated), "evaluate"... in the GridCoverage interface, and I
>>> wonder which is the preferred/intended way to access the pixel values
>>> in GeoTools.
>>>
>>> Best regards,
>>>
>>> César Martínez Izquierdo
>>>
>>>
>>>
>>> --
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>   César Martínez Izquierdo
>>>   GIS developer
>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>   Universitat Autònoma de Barcelona (SPAIN)
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>
>>> ------------------------------------------------------------------------------
>>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
>>> is the only developer event you need to attend this year. Jumpstart your
>>> developing skills, take BlackBerry mobile applications to market and stay
>>> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>>> http://p.sf.net/sfu/devconference
>>> _______________________________________________
>>> Geotools-gt2-users mailing list
>>> Geo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>>
>>
>
>
>
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   César Martínez Izquierdo
>   GIS developer
>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>   Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Geotools-gt2-users mailing list
> Geo...@li...
> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

Ciao Simone,
I send you the details in private mail.

Regards,

César

El día 7 de octubre de 2009 17:50, Simone Giannecchini
<sim...@ge...> escribió:
> If it is an esri arcgrid, by targizipping it it should become much
> smaller than that.
> Do you have an FTP with good bandwidth at hand?
>
> Simone.
> -------------------------------------------------------
> Ing. Simone Giannecchini
> GeoSolutions S.A.S.
> Founder - Software Engineer
> Via Carignoni 51
> 55041  Camaiore (LU)
> Italy
>
> phone: +39 0584983027
> fax:      +39 0584983027
> mob:    +39 333 8128928
>
>
> http://www.geo-solutions.it
> http://geo-solutions.blogspot.com/
> http://simboss.blogspot.com/
> http://www.linkedin.com/in/simonegiannecchini
>
> -------------------------------------------------------
>
>
>
> 2009/10/7 César Martínez Izquierdo <ces...@gm...>:
>> Hello Simone,
>> yes, I can provide test data. However, note that it is a 14,5 GB
>> raster, so we have to find the way to share it.
>> Any idea?
>>
>> Regards,
>>
>> César
>>
>> El día 7 de octubre de 2009 16:36, Simone Giannecchini
>> <sim...@ge...> escribió:
>>> Ciao Cesar,
>>> is there any chance that you can make some test data available.
>>>
>>> Simone.
>>> -------------------------------------------------------
>>> Ing. Simone Giannecchini
>>> GeoSolutions S.A.S.
>>> Founder - Software Engineer
>>> Via Carignoni 51
>>> 55041  Camaiore (LU)
>>> Italy
>>>
>>> phone: +39 0584983027
>>> fax:      +39 0584983027
>>> mob:    +39 333 8128928
>>>
>>>
>>> http://www.geo-solutions.it
>>> http://geo-solutions.blogspot.com/
>>> http://simboss.blogspot.com/
>>> http://www.linkedin.com/in/simonegiannecchini
>>>
>>> -------------------------------------------------------
>>>
>>>
>>>
>>> 2009/10/7 César Martínez Izquierdo <ces...@gm...>:
>>>> Hi again,
>>>> see my response bellow:
>>>>
>>>> El día 5 de octubre de 2009 22:24, Simone Giannecchini
>>>> <sim...@ge...> escribió:
>>>>> Ciao Cesar,
>>>> [SNIP]
>>>>> This is probably the worst way to work with a PlanarImage. I dag once
>>>>> those bindings and I told Victor that he was loading everything into
>>>>> memory doing that. Actually not only that, that code cobbles the
>>>>> original raster into a single untiled raster in memory, performing a
>>>>> full copy! Rather inefficient I would say :-)
>>>> [SNIP]
>>>>> the goal is to use iterators to do the processing, or anyway to do it
>>>>> tile by tile in order to avoid copies and/or loading everything into
>>>>> memory.
>>>>>
>>>>> Simone.
>>>>
>>>>
>>>> Thanks a lot for the links and suggestions. I've read this
>>>> documentation, and if I've correctly understood there are two
>>>> different possible approaches:
>>>> - Using iterators
>>>> - Using tiles
>>>>
>>>> I've tried both of them, but I've also found an error in the process.
>>>> As I'm a JAI and GeoTools beginner, I'm not sure if there is a problem
>>>> in the driver or I misunderstood documentation.
>>>>
>>>> I paste relevant code bellow:
>>>>
>>>> ----------------- ITERATOR APPROACH -----
>>>>        ArcGridFormat agf = new ArcGridFormat();
>>>>        GridCoverageReader reader = agf.getReader(file);
>>>>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>>>>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>>>>        RectIter iter = RectIterFactory.create(img, null);
>>>>        iter.startBands();
>>>>    while (!iter.finishedBands()) {
>>>>        iter.startLines();
>>>>        while (!iter.finishedLines()) {
>>>>            iter.startPixels();
>>>>            while (!iter.finishedPixels()) {
>>>>                int value = iter.getSample();
>>>>                if (value!=9999 && value!=-9999) {
>>>>                        System.out.println(value);
>>>>                }
>>>>                iter.nextPixel();
>>>>            }
>>>>            iter.nextLine();
>>>>        }
>>>>        iter.nextBand();
>>>>    }
>>>>
>>>> -----------------
>>>>
>>>> After some time running, I get the following exception:
>>>> java.lang.IllegalArgumentException: pos < flushedPos!
>>>>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>>>>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>>>>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>>>>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>>>>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>>>>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>>>>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:48)
>>>>
>>>>
>>>> I have also tried with tiles, but I get the same error.
>>>> I assume in this code that the tiles are automatically managed by the
>>>> PlanarImage and are not kept in memory (except the tiles that I ask
>>>> for and I keep referencing), but I'm not sure if this assumption is
>>>> correct...
>>>>
>>>> ---------------------- TILES APPROACH --------------------
>>>>
>>>>        ArcGridFormat agf = new ArcGridFormat();
>>>>        GridCoverageReader reader = agf.getReader(file);
>>>>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>>>>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>>>>
>>>>        for (int j=img.getMinTileY(), maxTileY = img.getMaxTileY();
>>>> j<=maxTileY ; j++) {
>>>>                for (int i=img.getMinTileX(), maxTileX = img.getMaxTileX();
>>>> i<=maxTileX; i++) {
>>>>                        Raster tile = img.getTile(i, j);
>>>>                        for (int y=tile.getMinY(), maxY = tile.getHeight()+y; y<maxY; y++) {
>>>>                                for (int x=tile.getMinX(), maxX = tile.getWidth()+x; x<maxX; x++) {
>>>>                                        int value = tile.getSample(x, y, 0); // assume band 0
>>>>                                                System.out.println("x: "+x+", y: "+y+", value: "+ value);
>>>>
>>>>                                }
>>>>                        }
>>>>                }
>>>>        }
>>>>
>>>> -------------------------------
>>>>
>>>> Again, after a while:
>>>>
>>>> java.lang.IllegalArgumentException: pos < flushedPos!
>>>>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>>>>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>>>>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>>>>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>>>>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>>>>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>>>>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>>>>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:70)
>>>>
>>>>
>>>>
>>>> So I don't know if there is an error in my code, or it is a problem
>>>> with the driver.
>>>>
>>>> By the way, I've also seen all these methods "getDataBlock"
>>>> (deprecated), "evaluate"... in the GridCoverage interface, and I
>>>> wonder which is the preferred/intended way to access the pixel values
>>>> in GeoTools.
>>>>
>>>> Best regards,
>>>>
>>>> César Martínez Izquierdo
>>>>
>>>>
>>>>
>>>> --
>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>   César Martínez Izquierdo
>>>>   GIS developer
>>>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>>>   Universitat Autònoma de Barcelona (SPAIN)
>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
>>>> is the only developer event you need to attend this year. Jumpstart your
>>>> developing skills, take BlackBerry mobile applications to market and stay
>>>> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>>>> http://p.sf.net/sfu/devconference
>>>> _______________________________________________
>>>> Geotools-gt2-users mailing list
>>>> Geo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>>>
>>>
>>
>>
>>
>> --
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>   César Martínez Izquierdo
>>   GIS developer
>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>   Universitat Autònoma de Barcelona (SPAIN)
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>
>> ------------------------------------------------------------------------------
>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
>> is the only developer event you need to attend this year. Jumpstart your
>> developing skills, take BlackBerry mobile applications to market and stay
>> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>> http://p.sf.net/sfu/devconference
>> _______________________________________________
>> Geotools-gt2-users mailing list
>> Geo...@li...
>> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>>
>



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[Simone G. <sim...@ge...>]

Dear Cesar,
please read below...
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041  Camaiore (LU)
Italy

phone: +39 0584983027
fax:      +39 0584983027
mob:    +39 333 8128928


http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------



2009/10/7 César Martínez Izquierdo <ces...@gm...>:
> Hi again,
> see my response bellow:
>
> El día 5 de octubre de 2009 22:24, Simone Giannecchini
> <sim...@ge...> escribió:
>> Ciao Cesar,
> [SNIP]
>> This is probably the worst way to work with a PlanarImage. I dag once
>> those bindings and I told Victor that he was loading everything into
>> memory doing that. Actually not only that, that code cobbles the
>> original raster into a single untiled raster in memory, performing a
>> full copy! Rather inefficient I would say :-)
> [SNIP]
>> the goal is to use iterators to do the processing, or anyway to do it
>> tile by tile in order to avoid copies and/or loading everything into
>> memory.
>>
>> Simone.
>
>
> Thanks a lot for the links and suggestions. I've read this
> documentation, and if I've correctly understood there are two
> different possible approaches:
> - Using iterators
> - Using tiles
>
> I've tried both of them, but I've also found an error in the process.
> As I'm a JAI and GeoTools beginner, I'm not sure if there is a problem
> in the driver or I misunderstood documentation.
>
> I paste relevant code bellow:
>
> ----------------- ITERATOR APPROACH -----
>        ArcGridFormat agf = new ArcGridFormat();
>        GridCoverageReader reader = agf.getReader(file);
>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>        RectIter iter = RectIterFactory.create(img, null);
>        iter.startBands();
>    while (!iter.finishedBands()) {
>        iter.startLines();
>        while (!iter.finishedLines()) {
>            iter.startPixels();
>            while (!iter.finishedPixels()) {
>                int value = iter.getSample();
>                if (value!=9999 && value!=-9999) {
>                        System.out.println(value);
>                }
>                iter.nextPixel();
>            }
>            iter.nextLine();
>        }
>        iter.nextBand();
>    }
>
> -----------------
>
> After some time running, I get the following exception:
> java.lang.IllegalArgumentException: pos < flushedPos!
>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:48)
>
>
> I have also tried with tiles, but I get the same error.
> I assume in this code that the tiles are automatically managed by the
> PlanarImage and are not kept in memory (except the tiles that I ask
> for and I keep referencing), but I'm not sure if this assumption is
> correct...
>
> ---------------------- TILES APPROACH --------------------
>
>        ArcGridFormat agf = new ArcGridFormat();
>        GridCoverageReader reader = agf.getReader(file);
>        GridCoverage2D gc = (GridCoverage2D) reader.read(null);
>        PlanarImage img = (PlanarImage) gc.geophysics(true).getRenderedImage();
>
>        for (int j=img.getMinTileY(), maxTileY = img.getMaxTileY();
> j<=maxTileY ; j++) {
>                for (int i=img.getMinTileX(), maxTileX = img.getMaxTileX();
> i<=maxTileX; i++) {
>                        Raster tile = img.getTile(i, j);
>                        for (int y=tile.getMinY(), maxY = tile.getHeight()+y; y<maxY; y++) {
>                                for (int x=tile.getMinX(), maxX = tile.getWidth()+x; x<maxX; x++) {
>                                        int value = tile.getSample(x, y, 0); // assume band 0
>                                                System.out.println("x: "+x+", y: "+y+", value: "+ value);
>
>                                }
>                        }
>                }
>        }
>
> -------------------------------
>
> Again, after a while:
>
> java.lang.IllegalArgumentException: pos < flushedPos!
>        at it.geosolutions.imageio.stream.input.FileImageInputStreamExtImpl.seek(FileImageInputStreamExtImpl.java:285)
>        at it.geosolutions.imageio.plugins.arcgrid.raster.AsciiGridRaster.readRaster(AsciiGridRaster.java:578)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.readRaster(AsciiGridsImageReader.java:659)
>        at it.geosolutions.imageio.plugins.arcgrid.AsciiGridsImageReader.read(AsciiGridsImageReader.java:520)
>        at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:697)
>        at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
>        at javax.media.jai.OpImage.getTile(OpImage.java:1129)
>        at javax.media.jai.RenderedOp.getTile(RenderedOp.java:2257)
>        at etclusi.tests.gt.TestArcGrid2.main(TestArcGrid2.java:70)
>
>
>
> So I don't know if there is an error in my code, or it is a problem
> with the driver.

Interesting, we have downloaded the test data you supplied, we'll play
with it and let you know...

>
> By the way, I've also seen all these methods "getDataBlock"
> (deprecated), "evaluate"... in the GridCoverage interface, and I
> wonder which is the preferred/intended way to access the pixel values
> in GeoTools.
>

The intended way is to use iterators and/or tile based access, the
getDataBlock will disappear as soon as we resolve the GeoApi
deprecation madness.

Simone.

> Best regards,
>
> César Martínez Izquierdo
>
>
>
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   César Martínez Izquierdo
>   GIS developer
>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>   Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Geotools-gt2-users mailing list
> Geo...@li...
> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
>

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

El día 9 de octubre de 2009 23:57, Simone Giannecchini
<sim...@ge...> escribió:
[SNIP]
>
> Interesting, we have downloaded the test data you supplied, we'll play
> with it and let you know...
[SNIP]

Hello Simone,

I've been playing with the code, and I've found the error: intValue()
is used instead of longValue() in ArcGridRaster, see attached patch.

After changing this line, I can correctly read all the file (either
using tiles or iterators).

Regards,

César



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Geotools-gt2-users] Problems reading huge rasters

[Simone G. <sim...@ge...>]

Thanks Cesar,

- bug report opened: https://imageio-ext.dev.java.net/issues/show_bug.cgi?id=17
- fix committed on 1.0.x an trunk of imageio-ext. Can you test as well?

Ciao,
Simone.

-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041  Camaiore (LU)
Italy

phone: +39 0584983027
fax:      +39 0584983027
mob:    +39 333 8128928


http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------



2009/10/13 César Martínez Izquierdo <ces...@gm...>:
> El día 9 de octubre de 2009 23:57, Simone Giannecchini
> <sim...@ge...> escribió:
> [SNIP]
>>
>> Interesting, we have downloaded the test data you supplied, we'll play
>> with it and let you know...
> [SNIP]
>
> Hello Simone,
>
> I've been playing with the code, and I've found the error: intValue()
> is used instead of longValue() in ArcGridRaster, see attached patch.
>
> After changing this line, I can correctly read all the file (either
> using tiles or iterators).
>
> Regards,
>
> César
>
>
>
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   César Martínez Izquierdo
>   GIS developer
>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>   Universitat Autònoma de Barcelona (SPAIN)
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>

Re: [Geotools-gt2-users] Problems reading huge rasters

[César M. I. <ces...@gm...>]

Hello Simone,
I've tried with trunk code and it works fine for me.

Regards,

César

El día 13 de octubre de 2009 16:41, Simone Giannecchini
<sim...@ge...> escribió:
> Thanks Cesar,
>
> - bug report opened: https://imageio-ext.dev.java.net/issues/show_bug.cgi?id=17
> - fix committed on 1.0.x an trunk of imageio-ext. Can you test as well?
>
> Ciao,
> Simone.
>
> -------------------------------------------------------
> Ing. Simone Giannecchini
> GeoSolutions S.A.S.
> Founder - Software Engineer
> Via Carignoni 51
> 55041  Camaiore (LU)
> Italy
>
> phone: +39 0584983027
> fax:      +39 0584983027
> mob:    +39 333 8128928
>
>
> http://www.geo-solutions.it
> http://geo-solutions.blogspot.com/
> http://simboss.blogspot.com/
> http://www.linkedin.com/in/simonegiannecchini
>
> -------------------------------------------------------
>
>
>
> 2009/10/13 César Martínez Izquierdo <ces...@gm...>:
>> El día 9 de octubre de 2009 23:57, Simone Giannecchini
>> <sim...@ge...> escribió:
>> [SNIP]
>>>
>>> Interesting, we have downloaded the test data you supplied, we'll play
>>> with it and let you know...
>> [SNIP]
>>
>> Hello Simone,
>>
>> I've been playing with the code, and I've found the error: intValue()
>> is used instead of longValue() in ArcGridRaster, see attached patch.
>>
>> After changing this line, I can correctly read all the file (either
>> using tiles or iterators).
>>
>> Regards,
>>
>> César
>>
>>
>>
>> --
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>   César Martínez Izquierdo
>>   GIS developer
>>   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
>>   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
>>   Universitat Autònoma de Barcelona (SPAIN)
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>
>



-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-LUSI: http://etc-lusi.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -