Menu
▾
▴
Re: [geowebcache-users] GeoWebCache directory/file name structure
|
From: TMartin <Tim...@or...> - 2010-06-14 14:28:50
|
Hi Arne In our OpenSpace API we have 10 zoom levels and their resolutions. 10 Streetview 1m 9 Streetview Resampled 2m 8 1:50 000 5m 7 1:50 000 Resampled 10m 6 1:250 000 25m 5 1:250 000 Resampled 50m 4 MiniScale 100m 3 MiniScale Resampled 200m 2 Overview 500m 1 Overview Resampled 1000m 0 Overview 2000m These are already raster tiles and are chopped into either 200x200 pixel tiles or 250x250 pixel tiles depending on their original tile size. When OpenSpace was built we used our own tile naming convention and directory structure because there were no others in existence. We are now wanting to create tiles from a vector dataset to replace raster dataset at zoom levels 10 and 9 (resolution 1m and 2m) So we have PostGIS with the data in, Geoserver 2.0.2, style sheets (SLDs) and want to use Geowebcache to create the tiles in EPSG 27700 (obviously british national grid ;) After spending sometime with a calculator i have worked out that we need 14,560,000 250x250 pixel tiles to cover GB, with coordinates 0, 0, 700000, 1300000 at 1m resolution. This is the configuration file I have been using, and it works and creates tiles. <?xml version="1.0" encoding="utf-8"?> <gwcConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://geowebcache.org/schema/1.2.0/geowebcache.xsd" xmlns="http://geowebcache.org/schema/1.2.0"> <version>1.2.0</version> <backendTimeout>300</backendTimeout> <gridSets> <!-- String name; SRS srs; BoundingBox extent; double[] resolutions; double[] scales; String[] scaleNames; Integer levels; Integer tileHeight; Integer tileWidth; --> <gridSet> <name>osgb_vml</name> <srs><number>27700</number></srs> <extent> <coords> <double>0</double> <double>0</double> <double>700000</double> <double>1300000</double> </coords> </extent> <alignTopLeft>false</alignTopLeft> <resolutions> <double>1</double> </resolutions> <tileHeight>250</tileHeight> <tileWidth>250</tileWidth> </gridSet> </gridSets> <layers> <wmsLayer> <name>osgb:vml/name> <gridSubsets> <gridSubset> <gridSetName>osgb_vml</gridSetName> </gridSubset> </gridSubsets> <wmsUrl><string>http://localhost:8080/geoserver/wms</string></wmsUrl> <wmsLayers>osgb:areas,osgb:lines,osgb:roadclines,osgb:text</wmsLayers> <!-- OPTIONAL The metatiling factors used for this layer If not specified, 3x3 metatiling is used for image formats --> <metaWidthHeight><int>1</int><int>1</int></metaWidthHeight> <bgColor>0xFFFFFF</bgColor> </wmsLayer> </layers> </gwcConfiguration> So when i seed this layer and go to the setup page i only the option of seeding between zoom level 00 and finish zoom level 00. Which is fine. The problem as Pete mentioned above is the shear number of directories that only have 4 tiles in each folder. I hope that gives a better picture of what we are up do. I appreciate its an extreme case but when we get it sorted we will post up the solution and the timeframes it took. cheers Tim Arne Kepp-2 wrote: > > So the plot thickens :) > > You mentioned in another email that you were just seeding one zoomlevel. > So I guess you also made a grid set configuration with very few levels, > perhaps just 2? > > This isn't useful for most GWC users, you need a tile pyramid with many > levels sot hat people can gradually zoom in. I am guessing that for > OpenSpace you'll be subsampling these tiles to make up intermediate > resolutions. > > So this becomes a very extreme case that the default directory structure > was definitely not designed to handle, normally you only have 1 to 16 > tiles at zoomlevel 0 , so by the time you get down to 16 million tiles > (z=12 or so) they should be spread across 4000 directories with 4000 > files each. > > If you have a thought-out directory structure for your application, I > would implement that as a new BlobStore, because your needs will be > different from the normal GWC user. Alternatively, you could use the > existing tile structure and just define a few resolutions you will not > use, so that the level with 16 million tiles is z=12 or so. > > Note that you need to clear the cache (delete the directory) if you > change the grid set definition. > > -Arne > > > On 06/10/2010 10:05 PM, TMartin wrote: >> Hi Arne >> >> Tim here. Pete and I are both working on the same issue here at OS. >> >> In each directory there are only 4 tiles of 250x250pixels and 1metre >> resolution as specified in our geowebcache configuration xml document. >> >> If thats the case we may end up with over 3 million directories for the >> 14,560,000 tiles we are creating. >> >> I will try and set it up so you can take a look at if you are interested? >> >> Tim >> >> >> >> Arne Kepp-2 wrote: >> >>> It's a compromise, and this part of the code is definitely made easily >>> pluggable so that one can use a different structures for special needs. >>> I was expecting that we would have a few different implementations by >>> now, but apparently not. >>> >>> Every directory level requires a node to be looked up (move the disk >>> head), so too many directories is not good either. Within a directory >>> there is a proper index. Listing the content of the cache is generally >>> not recommended, I don't know of a single filesystem that is optimized >>> for that and most utilities can't handle something of this size. "find" >>> is usually ok. >>> >>> But 400 000 directories.... Unless you've discovered a bug, you should >>> have at least 80 billion tiles? I wouldn't use a conventional >>> filesystem for that, and on a fast machine (100 tiles per second) it >>> would take 25 years to build a cache of that size. >>> >>> -Arne >>> >>> >>> On 06/10/2010 11:22 AM, samclemmens wrote: >>> >>>> Cheers, Arne! >>>> >>>> I'm curious; what was the rationale for creating a directory structure >>>> with >>>> just one level? We processed a data set earlier this week, which >>>> resulted >>>> in 400,000 directories. When I tried to an "ls", my PuTTY console >>>> froze... >>>> >>>> Peter >>>> >>>> >>>> >>>> Arne Kepp-2 wrote: >>>> >>>> >>>>> 1) "zoom level". For the default EPSG:900913 grid z=0 means the whole >>>>> world in 4 tiles >>>>> >>>>> 2) Hm,,, good point. I dont think I went back and updated that wiki >>>>> page, it was not more than a scratchpad for sharing ideas. The actual >>>>> code is here[1], the difference is that the bit shifting starts with >>>>> 2, >>>>> effectively making it 2^( 1 + ( z / 2 )). And there's some >>>>> zero-padding. >>>>> >>>>> 3) The catch is that you need to know a little bit of Java, but if you >>>>> do then it's potentially straightforward. It depends what the >>>>> OpenSpace >>>>> structure is based on, and whether we have similar tokens in GWC. >>>>> >>>>> blobstore/file contains an implementation of BlobStore. You can copy >>>>> the >>>>> whole thing to a new package, and then make the FilePathGenerator >>>>> output >>>>> the structure you want. It is linked into the application using >>>>> Spring, >>>>> WEB-INF/geowebcache-core-context.xml , if you are working off trunk. I >>>>> would guess that you also want a particular GridSet to go with that, >>>>> to >>>>> match the resolutions and origin used by OpenSpace. >>>>> >>>>> It's possible someone has already done it, but so far I have not heard >>>>> about it. >>>>> >>>>> -Arne >>>>> >>>>> 1: >>>>> http://geowebcache.org/trac/browser/trunk/geowebcache/core/src/main/java/org/geowebcache/storage/blobstore/file/FilePathGenerator.java >>>>> >>>>> >>>>> On 06/08/2010 05:33 PM, samclemmens wrote: >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm relatively new to GeoWebCache...and I'm a bit puzzled by the >>>>>> directory/file name structure that is produced. I have a few >>>>>> questions, >>>>>> namely: >>>>>> >>>>>> 1. The documentation states that the naming convention is: >>>>>> layername/projection_z/[x/(2^(z/2))]_[y/(2^(z/2))]/x_y.extension >>>>>> What >>>>>> is >>>>>> the z value? >>>>>> >>>>>> 2. Given the sample output >>>>>> .\gwc\roads\EPSG_900913_12\006_019\000848_002540.png, I assume that >>>>>> z=12, >>>>>> x=848, and y=2540. Obviously the above formula won't produce 006_019 >>>>>> based >>>>>> on these numbers. What conversion is required? >>>>>> >>>>>> 3. Is it possible to change the directory/file name format? I'd >>>>>> like >>>>>> to >>>>>> create a data set that can be used by Ordnance Survey OpenSpace, >>>>>> which >>>>>> uses >>>>>> a different structure based on the British National Grid. >>>>>> >>>>>> Cheers, >>>>>> >>>>>> Peter >>>>>> >>>>>> >>>>>> >>>>> ------------------------------------------------------------------------------ >>>>> ThinkGeek and WIRED's GeekDad team up for the Ultimate >>>>> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the >>>>> lucky parental unit. See the prize list and enter to win: >>>>> http://p.sf.net/sfu/thinkgeek-promo >>>>> _______________________________________________ >>>>> Geowebcache-users mailing list >>>>> Geo...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/geowebcache-users >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >>> ------------------------------------------------------------------------------ >>> ThinkGeek and WIRED's GeekDad team up for the Ultimate >>> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the >>> lucky parental unit. See the prize list and enter to win: >>> http://p.sf.net/sfu/thinkgeek-promo >>> _______________________________________________ >>> Geowebcache-users mailing list >>> Geo...@li... >>> https://lists.sourceforge.net/lists/listinfo/geowebcache-users >>> >>> >>> >> > > > ------------------------------------------------------------------------------ > ThinkGeek and WIRED's GeekDad team up for the Ultimate > GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the > lucky parental unit. See the prize list and enter to win: > http://p.sf.net/sfu/thinkgeek-promo > _______________________________________________ > Geowebcache-users mailing list > Geo...@li... > https://lists.sourceforge.net/lists/listinfo/geowebcache-users > > -- View this message in context: http://old.nabble.com/GeoWebCache-directory-file-name-structure-tp28819540p28880229.html Sent from the GeoWebCache Users mailing list archive at Nabble.com. |
