Menu
▾
▴
Re: [Geotools-gt2-users] VERY big GeoTIFF drawing - what's the error ?
|
From: Simone G. <sim...@gm...> - 2008-04-03 19:41:48
|
Ciao Jan,
please, read below...
On Thu, Apr 3, 2008 at 9:24 PM, Jan Goyvaerts <goy...@te...> wrote:
> Hello Simone,
>
> Thanks for your reply. I'm aware geotiff can contain multiple multi-levelled&tiled images, but I didn't know geotools was able to cope with this directly.
Yes it is able :-).
> I assumed some pre-processing was required: splitting into separate files and indexing them using one shp file. Just out of curiosity, in what situations is this technique useful ? When the data format doesn't allow for multiple levels and tiles ?
>
Well the image mosaic plugin was made for the opposite reason, I had
20.000 tiles from a side scan sonar acquisition (postprocessed) and I
was asked to show them in geoserver in a few days. Merging them was
not an option since the size was too big.
In general the best way to use the mosaic is with a few very big
geotiff with internal pyramids (yeah the imagemosaic try to use them).
In case you have several thousands of files I'd build a pyramid, i.e.,
multiple mosaics at dcreasing resolution ( I really need to write some
docs about this I know).
> I'll put gdal to work tomorrow to prepare a geotiff file. I assume with 2/4/8/16/... you mean the various levels of the geotiff file ? The second level half the size, the third a fourth of the size, etc... ?
>
Yeah, exact.
1> tile original image
2> add overvies to it
The file size will be a bit bigger but it should work out fine. As an
lternative you can split the original image in let's say 4 big tiles,
do the same things on each of then and then mosaic.
> Concerning the third point - feeding the reader instead of the converage - I assume one uses the coverage when the raster needs to reside completely into memory and the reader for lazy loading of parts of the raster. Correct ?
>
If you create the covreage like you were doing the highest available
resolutions is chosen, and no overviews are used.
Ciao,
Simone.
> Many thanks for helping me !!
>
> Jan
>
>
>
> On Thu, 3 Apr 2008 18:43:06 +0200
> "Simone Giannecchini" <sim...@gm...> wrote:
>
> > Ciao Jan,
> > I am chiming in a bit late but I'll try to be helpful anyway.
> > I scanned quickly this email so I might have not understood everything
> > correclty, let me try to summarize (and ask some questions along the
> > line):
> >
> > 1> you had a big geotiff (2GB (30000x20000px) GeoTIFF) that you wanted to show
> > 2> you tiled it small chunks of 512,512
> > 3> you build an index for it
> > 4> you are expeting to render it on a 640x480 window.
> >
> > Let's assume that so far, I am right.
> > Well, I think you went down the wrong path, since you are trying to
> > compose too many files at the same time and also you are subsampling
> > them on the fly to get the final small output raster.
> >
> > Here is what i would do instead.
> > 1>if the original geotiff is not tiled, tile it using gdal_translate
> > and a tile size of 512,512. Note that I am not talking about splitting
> > the geotiff in N smaller geotiff files, I am talking about ineer
> > tiling (check the ggdal tiff driver options or I'll tell you how to
> > get there)
> > 2> use gdal_addo and add overviews to the tiled geotiff. I usually use
> > 2 4 8 16 32 64 128 level with average resampling for better visual
> > quality )no aliasing)
> > 3> ffed the mapcontext with the reader not with the coverage itself,
> > so that the renderer will choose the best resolution at rendering time
> >
> >
> > Ciao,
> > Simone.
> >
> > On Thu, Apr 3, 2008 at 6:24 PM, Jan Goyvaerts <goy...@te...> wrote:
> > >
> > > Okay, I wasn't aware Linux has a hard coded limit of 1024 open files. I guess
> > > a proxy pattern on the ImageInputStream would solve this problem. Anyway,
> > > using bigger and less tiles solves this issue.
> > >
> > > I don't have any exceptions now, but I can't get the image displayed. The
> > > application eats away any resource I'm throwing to it. It is stuck at
> > > GridCoverageRenderer.paint():754 (graphics.drawRenderedImage()). As my
> > > viewport is 640x480 and the 882 tiles are 512x512 I'm probably doing some
> > > beginners mistake when the program starts easting away all memory it gets.
> > >
> > > I'm putting the code sample I'm using in this message. If somebody sees my
> > > mistake, please tell me.
> > >
> > > Many thanks for help me !!!
> > >
> > > Jan
> > >
> > > // (1) initialize the reader
> > > File indexFile = new File( "/u01/data/BlueMarble/split2/tiles_index.shp"
> > > );
> > > ImageMosaicReader reader = new ImageMosaicReader( indexFile );
> > >
> > > // (2) create the coverage
> > > GridCoverage coverage = reader.read( null );
> > >
> > > // (3) create the style - it's a raster so we need a raster symbolizer
> > > StyleBuilder styleBuilder = new StyleBuilder();
> > > RasterSymbolizer rasterSymbolizer =
> > > styleBuilder.createRasterSymbolizer();
> > > rasterSymbolizer.setGeometryPropertyName( "geom" );
> > > Style style = styleBuilder.createStyle( rasterSymbolizer );
> > >
> > > // (4) create the layer;
> > > DefaultMapLayer layer = new DefaultMapLayer( coverage, style );
> > > layer.setTitle( "Test" );
> > > layer.setVisible( true );
> > >
> > > // (5) create the map context
> > > MapContext map = new DefaultMapContext( WGS84 );
> > > map.addLayer( layer );
> > >
> > > // (6) create the renderer
> > > GTRenderer renderer = new StreamingRenderer();
> > > RenderingHints hints = new RenderingHints(
> > > RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON );
> > > Map<String, Object> rendererParams = new HashMap<String, Object>();
> > > rendererParams.put( "optimizedDataLoadingEnabled", new Boolean( true )
> > > );
> > > renderer.setJava2DHints( hints );
> > > renderer.setRendererHints( rendererParams );
> > >
> > > // (7) assemble the created components into a regular Swing frame
> > > JMapPane panel = new JMapPane( renderer, map );
> > > panel.setMapArea( map.getAreaOfInterest() );
> > >
> > > JFrame frame = new JFrame( "TrueMarble.2km.21600x10800.tif" );
> > > frame.setSize( 640, 480 );
> > > frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );
> > > frame.getContentPane().add( panel );
> > > frame.setVisible( true );
> > >
> > >
> > > Jan Goyvaerts wrote:
> > > >
> > > > Hello Diego,
> > > >
> > > > I'm trying to do roughly the same with the Bleumarble data set. I tiled a
> > > > geotiff file using your script - very useful ! - and now I'm trying to use
> > > > the ImageMosaic plugin to display it in a layer of a JMapPanel. I was
> > > > wondering whether you managed to make it work because I'm having a weird
> > > > problem: When the tiles are read by ImageIO from some point the
> > > > createImageInputStream() method only returns nulls. (After having read
> > > > successfully 903 tiles.) The very same file can be displayed individually
> > > > though.
> > > >
> > > > Wondering this might be soft/weak reference problem I tried with increased
> > > > heap sizes. Alas, with the same result. As far as I can see the .prj and
> > > > .properties files are ok.
> > > >
> > > > By any chance, did you ran into a similar issue ? I'm rather new to
> > > > GeoTools, so if you have a code sample of some kind to help me out that
> > > > would be great.
> > > >
> > > > Many thanks !
> > > >
> > > > Jan
> > > >
> > > >
> > > > Diego Fdez. Durán wrote:
> > > >>
> > > >> Step by step.
> > > >>
> > > >> I've done a little script[1] that chops a big GeoTIFF in tiles using
> > > >> gdal[2], and generates the .shp index file. Feel free to put it in the
> > > >> GeoTools doc if you found it interesting.
> > > >>
> > > >> Tomorrow I'll try using ImageMosaic to load the image (I suppose that I
> > > >> only have to load the .shp file and the plugin do the .tif loading
> > > >> itself).
> > > >>
> > > >> :)
> > > >>
> > > >> [1]
> > > >> -- BEGIN SCRIPT
> > > >> #!/bin/bash
> > > >> #
> > > >> # This script divides a GeoTIFF file in tiles and generates a .shp index
> > > >> file.
> > > >> #
> > > >> # Diego Fdez. Durán <di...@go...>
> > > >> # AAC IAF - Universidad de León - 2007
> > > >>
> > > >>
> > > >> TILESIZE=256
> > > >> STORE_DIR="tiles"
> > > >> INDEX_FILE="index_tiles.shp"
> > > >>
> > > >> usage() {
> > > >> echo "Usage: $0 file.tif [tilesize]"
> > > >> }
> > > >>
> > > >> if [ -n "$1" ] && [ -e $1 ]; then
> > > >> MAP_FILE=$1
> > > >> else
> > > >> usage
> > > >> exit -1;
> > > >> fi
> > > >>
> > > >> STR=( `gdalinfo $MAP_FILE | grep ^Size | cut -d' ' -f3,4 | sed s/','/'
> > > >> '/` )
> > > >> MAP_H_RES=${STR[0]}
> > > >> MAP_V_RES=${STR[1]}
> > > >>
> > > >>
> > > >> if [ -n "$2" ]; then
> > > >> TILESIZE=$2
> > > >> fi
> > > >>
> > > >> if [ ! -d "$STORE_DIR" ]; then
> > > >> mkdir $STORE_DIR
> > > >> fi
> > > >>
> > > >> let "N_H_TILES=$MAP_H_RES / $TILESIZE"
> > > >> let "N_V_TILES=$MAP_V_RES / $TILESIZE"
> > > >> let "N_TILES=$N_H_TILES * $N_V_TILES"
> > > >>
> > > >>
> > > >> echo "File: $MAP_FILE of ${MAP_H_RES}x${MAP_V_RES}px"
> > > >> echo "Generating $N_TILES (${N_H_TILES}*${N_V_TILES}) ${TILESIZE}x
> > > >> ${TILESIZE}px tiles"
> > > >>
> > > >> COUNT=1
> > > >> for ((i=0; i < N_H_TILES; i++))
> > > >> do
> > > >> let "H_OFFSET=$i * $TILESIZE"
> > > >> for ((j=0; j < N_V_TILES; j++))
> > > >> do
> > > >> let "V_OFFSET=$j * $TILESIZE"
> > > >> let "PERCENT=(($COUNT / $N_TILES) * 100)"
> > > >> TILE_FILE="tile-${i}_${j}.tif"
> > > >> echo -ne "Working: $COUNT/$N_TILES ($PERCENT%) ::
> > > >> Creating tile $TILE_FILE\r"
> > > >> gdal_translate -of GTiff -co "TILED=YES" -srcwin
> > > >> $H_OFFSET $V_OFFSET $TILESIZE $TILESIZE $MAP_FILE $STORE_DIR/$TILE_FILE
> > > >>> /dev/null
> > > >> let "COUNT=$COUNT + 1"
> > > >> done
> > > >> done
> > > >>
> > > >>
> > > >> echo "Generating tile index $INDEX_FILE"
> > > >> gdaltindex tiles_index.shp tiles/*.tif
> > > >> -- END SCRIPT
> > > >>
> > > >> [2] http://www.gdal.org/
> > > >>
> > > >>
> > > >> El sáb, 01-12-2007 a las 14:21 -0800, Jody Garnett escribió:
> > > >>> They made a BigTIFF "fake specification" recently for which you may want
> > > >>> to look around. Creatively named BIGTIFF I think.
> > > >>>
> > > >>> Two things:
> > > >>> 1) just try it; grab the uDig tech preview
> > > >>> - http://udig.refractions.net/downloads/ECWDemo.win32.win32.x86.zip
> > > >>> - http://udig.refractions.net/downloads/jre/jre1.6.0_03.win32.zip (in
> > > >>> zip this JRE into the same folder; it contains imageio-ext for geotiff)
> > > >>> (ImageIO is good about leaving the image on disk so it may be slow but
> > > >>> you should see something)
> > > >>>
> > > >>> 2) image moasic (break the image up using gdal or some imageio tools
> > > >>> and use a shapefile to keep track of this parts)
> > > >>> - http://docs.codehaus.org/display/GEOTDOC/Image+Mosaic+Plugin
> > > >>> The above page links to most of the information I could find (including
> > > >>> a tutorial of breaking up a large bluemarble image); I have been using
> > > >>> the result (amazing!) but have not tried breaking up an image yet.
> > > >>>
> > > >>> Cheers,
> > > >>> Jody
> > > >>> > Hi all:
> > > >>> >
> > > >>> > I'm trying visualize a 2GB (30000x20000px) GeoTIFF. I need to show a
> > > >>> > 1280x800px viewport of the image and be able to navigate through the
> > > >>> > full GeoTIFF.
> > > >>> >
> > > >>> > Is there any way to do this?
> > > >>> >
> > > >>> > - Directly working with the GeoTIFF.
> > > >>> > - Dividing the GeoTIFF in small tiles of 256x256px and doing on
> > > >>> demand
> > > >>> > loading? (How can I divide the GeoTiFF?)
> > > >>> >
> > > >>> > Can you point me in the right direction?
> > > >>> >
> > > >>> > Thanks in advance.
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> ------------------------------------------------------------------------
> > > >>> >
> > > >>> >
> > > >>> -------------------------------------------------------------------------
> > > >>> > SF.Net email is sponsored by: The Future of Linux Business White Paper
> > > >>> > from Novell. From the desktop to the data center, Linux is going
> > > >>> > mainstream. Let it simplify your IT future.
> > > >>> > http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
> > > >>> >
> > > >>> ------------------------------------------------------------------------
> > > >>> >
> > > >>> > _______________________________________________
> > > >>> > Geotools-gt2-users mailing list
> > > >>> > Geo...@li...
> > > >>> > https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
> > > >>> >
> > > >>>
> > > >> --
> > > >> Diego Fdez. Durán <di...@go...> | http://www.goedi.net
> > > >> GPG : 925C 9A21 7A11 3B13 6E43 50DB F579 D119 90D2 66BB
> > > >>
> > > >>
> > > >>
> > > >> -------------------------------------------------------------------------
> > > >> SF.Net email is sponsored by: The Future of Linux Business White Paper
> > > >> from Novell. From the desktop to the data center, Linux is going
> > > >> mainstream. Let it simplify your IT future.
> > > >> http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
> > > >> _______________________________________________
> > > >> Geotools-gt2-users mailing list
> > > >> Geo...@li...
> > > >> https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
> > > >>
> > > >>
> > > >
> > > >
> > >
> > > --
> > > View this message in context: http://www.nabble.com/VERY-big-GeoTIFF-drawing-tp14109985p16467800.html
> > > Sent from the geotools-gt2-users mailing list archive at Nabble.com.
> > >
> > >
> > > -------------------------------------------------------------------------
> > > Check out the new SourceForge.net Marketplace.
> > > It's the best place to buy or sell services for
> > > just about anything Open Source.
> > > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> > > _______________________________________________
> > > Geotools-gt2-users mailing list
> > > Geo...@li...
> > > https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
> > >
> >
> >
> >
> > --
> > -------------------------------------------------------
> > Eng. Simone Giannecchini
> > President /CEO GeoSolutions S.A.S.
> > Via Carignoni 51
> > 55041 Camaiore (LU)
> > Italy
> >
> > phone: +39 0584983027
> > fax: +39 0584983027
> > mob: +39 333 8128928
> >
> >
> > http://www.geo-solutions.it
> >
> > -------------------------------------------------------
> >
>
>
> --
>
>
> Jan Goyvaerts <goy...@te...>
>
--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy
phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928
http://www.geo-solutions.it
-------------------------------------------------------
|
