Menu
▾
▴
Re: [geowebcache-users] GeoWebCache directory/file name structure
|
From: Arne K. <ar...@ti...> - 2010-06-11 12:03:15
|
So the plot thickens :) You mentioned in another email that you were just seeding one zoomlevel. So I guess you also made a grid set configuration with very few levels, perhaps just 2? This isn't useful for most GWC users, you need a tile pyramid with many levels sot hat people can gradually zoom in. I am guessing that for OpenSpace you'll be subsampling these tiles to make up intermediate resolutions. So this becomes a very extreme case that the default directory structure was definitely not designed to handle, normally you only have 1 to 16 tiles at zoomlevel 0 , so by the time you get down to 16 million tiles (z=12 or so) they should be spread across 4000 directories with 4000 files each. If you have a thought-out directory structure for your application, I would implement that as a new BlobStore, because your needs will be different from the normal GWC user. Alternatively, you could use the existing tile structure and just define a few resolutions you will not use, so that the level with 16 million tiles is z=12 or so. Note that you need to clear the cache (delete the directory) if you change the grid set definition. -Arne On 06/10/2010 10:05 PM, TMartin wrote: > Hi Arne > > Tim here. Pete and I are both working on the same issue here at OS. > > In each directory there are only 4 tiles of 250x250pixels and 1metre > resolution as specified in our geowebcache configuration xml document. > > If thats the case we may end up with over 3 million directories for the > 14,560,000 tiles we are creating. > > I will try and set it up so you can take a look at if you are interested? > > Tim > > > > Arne Kepp-2 wrote: > >> It's a compromise, and this part of the code is definitely made easily >> pluggable so that one can use a different structures for special needs. >> I was expecting that we would have a few different implementations by >> now, but apparently not. >> >> Every directory level requires a node to be looked up (move the disk >> head), so too many directories is not good either. Within a directory >> there is a proper index. Listing the content of the cache is generally >> not recommended, I don't know of a single filesystem that is optimized >> for that and most utilities can't handle something of this size. "find" >> is usually ok. >> >> But 400 000 directories.... Unless you've discovered a bug, you should >> have at least 80 billion tiles? I wouldn't use a conventional >> filesystem for that, and on a fast machine (100 tiles per second) it >> would take 25 years to build a cache of that size. >> >> -Arne >> >> >> On 06/10/2010 11:22 AM, samclemmens wrote: >> >>> Cheers, Arne! >>> >>> I'm curious; what was the rationale for creating a directory structure >>> with >>> just one level? We processed a data set earlier this week, which >>> resulted >>> in 400,000 directories. When I tried to an "ls", my PuTTY console >>> froze... >>> >>> Peter >>> >>> >>> >>> Arne Kepp-2 wrote: >>> >>> >>>> 1) "zoom level". For the default EPSG:900913 grid z=0 means the whole >>>> world in 4 tiles >>>> >>>> 2) Hm,,, good point. I dont think I went back and updated that wiki >>>> page, it was not more than a scratchpad for sharing ideas. The actual >>>> code is here[1], the difference is that the bit shifting starts with 2, >>>> effectively making it 2^( 1 + ( z / 2 )). And there's some >>>> zero-padding. >>>> >>>> 3) The catch is that you need to know a little bit of Java, but if you >>>> do then it's potentially straightforward. It depends what the OpenSpace >>>> structure is based on, and whether we have similar tokens in GWC. >>>> >>>> blobstore/file contains an implementation of BlobStore. You can copy the >>>> whole thing to a new package, and then make the FilePathGenerator output >>>> the structure you want. It is linked into the application using Spring, >>>> WEB-INF/geowebcache-core-context.xml , if you are working off trunk. I >>>> would guess that you also want a particular GridSet to go with that, to >>>> match the resolutions and origin used by OpenSpace. >>>> >>>> It's possible someone has already done it, but so far I have not heard >>>> about it. >>>> >>>> -Arne >>>> >>>> 1: >>>> http://geowebcache.org/trac/browser/trunk/geowebcache/core/src/main/java/org/geowebcache/storage/blobstore/file/FilePathGenerator.java >>>> >>>> >>>> On 06/08/2010 05:33 PM, samclemmens wrote: >>>> >>>> >>>>> Hi, >>>>> >>>>> I'm relatively new to GeoWebCache...and I'm a bit puzzled by the >>>>> directory/file name structure that is produced. I have a few >>>>> questions, >>>>> namely: >>>>> >>>>> 1. The documentation states that the naming convention is: >>>>> layername/projection_z/[x/(2^(z/2))]_[y/(2^(z/2))]/x_y.extension >>>>> What >>>>> is >>>>> the z value? >>>>> >>>>> 2. Given the sample output >>>>> .\gwc\roads\EPSG_900913_12\006_019\000848_002540.png, I assume that >>>>> z=12, >>>>> x=848, and y=2540. Obviously the above formula won't produce 006_019 >>>>> based >>>>> on these numbers. What conversion is required? >>>>> >>>>> 3. Is it possible to change the directory/file name format? I'd like >>>>> to >>>>> create a data set that can be used by Ordnance Survey OpenSpace, which >>>>> uses >>>>> a different structure based on the British National Grid. >>>>> >>>>> Cheers, >>>>> >>>>> Peter >>>>> >>>>> >>>>> >>>> ------------------------------------------------------------------------------ >>>> ThinkGeek and WIRED's GeekDad team up for the Ultimate >>>> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the >>>> lucky parental unit. See the prize list and enter to win: >>>> http://p.sf.net/sfu/thinkgeek-promo >>>> _______________________________________________ >>>> Geowebcache-users mailing list >>>> Geo...@li... >>>> https://lists.sourceforge.net/lists/listinfo/geowebcache-users >>>> >>>> >>>> >>>> >>> >>> >> >> ------------------------------------------------------------------------------ >> ThinkGeek and WIRED's GeekDad team up for the Ultimate >> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the >> lucky parental unit. See the prize list and enter to win: >> http://p.sf.net/sfu/thinkgeek-promo >> _______________________________________________ >> Geowebcache-users mailing list >> Geo...@li... >> https://lists.sourceforge.net/lists/listinfo/geowebcache-users >> >> >> > |
