Re: [geowebcache-users] GeoWebCache directory/file name structure

[TMartin <Tim...@or...>]

Hi Arne

Tim here. Pete and I are both working on the same issue here at OS.

In each directory there are only 4 tiles of 250x250pixels and 1metre
resolution as specified in our geowebcache configuration xml document.

If thats the case we may end up with over 3 million directories for the
14,560,000 tiles we are creating.

I will try and set it up so you can take a look at if you are interested?

Tim



Arne Kepp-2 wrote:
> 
> It's a compromise, and this part of the code is definitely made easily 
> pluggable so that one can use a different structures for special needs. 
> I was expecting that we would have a few different implementations by 
> now, but apparently not.
> 
> Every directory level requires a node to be looked up (move the disk 
> head), so too many directories is not good either. Within a directory 
> there is a proper index. Listing the content of the cache is generally 
> not recommended, I don't know of a single filesystem that is optimized 
> for that and most utilities can't handle something of this size. "find" 
> is usually ok.
> 
> But 400 000 directories....  Unless you've discovered a bug, you should 
> have at least  80 billion tiles? I wouldn't use a conventional 
> filesystem for that, and on a fast machine (100 tiles per second) it 
> would take  25 years to build a cache of that size.
> 
> -Arne
> 
> 
> On 06/10/2010 11:22 AM, samclemmens wrote:
>> Cheers, Arne!
>>
>> I'm curious; what was the rationale for creating a directory structure
>> with
>> just one level?  We processed a data set earlier this week, which
>> resulted
>> in 400,000 directories.  When I tried to an "ls", my PuTTY console
>> froze...
>>
>> Peter
>>
>>
>>
>> Arne Kepp-2 wrote:
>>    
>>> 1) "zoom level". For the default EPSG:900913 grid z=0 means the whole
>>> world in 4 tiles
>>>
>>> 2) Hm,,, good point. I dont think I went back and updated that wiki
>>> page, it was not more than a scratchpad for sharing ideas. The actual
>>> code is here[1], the difference is that the bit shifting starts with 2,
>>> effectively making it  2^( 1 +  ( z / 2 )). And there's some
>>> zero-padding.
>>>
>>> 3) The catch is that you need to know a little bit of Java, but if you
>>> do then it's potentially straightforward. It depends what the OpenSpace
>>> structure is based on, and whether we have similar tokens in GWC.
>>>
>>> blobstore/file contains an implementation of BlobStore. You can copy the
>>> whole thing to a new package, and then make the FilePathGenerator output
>>> the structure you want. It is linked into the application using Spring,
>>> WEB-INF/geowebcache-core-context.xml , if you are working off trunk. I
>>> would guess that you also want a particular GridSet to go with that, to
>>> match the resolutions and origin used by OpenSpace.
>>>
>>> It's possible someone has already done it, but so far I have not heard
>>> about it.
>>>
>>> -Arne
>>>
>>> 1:
>>> http://geowebcache.org/trac/browser/trunk/geowebcache/core/src/main/java/org/geowebcache/storage/blobstore/file/FilePathGenerator.java
>>>
>>>
>>> On 06/08/2010 05:33 PM, samclemmens wrote:
>>>      
>>>> Hi,
>>>>
>>>> I'm relatively new to GeoWebCache...and I'm a bit puzzled by the
>>>> directory/file name structure that is produced.  I have a few
>>>> questions,
>>>> namely:
>>>>
>>>> 1.  The documentation states that the naming convention is:
>>>> layername/projection_z/[x/(2^(z/2))]_[y/(2^(z/2))]/x_y.extension   
>>>> What
>>>> is
>>>> the z value?
>>>>
>>>> 2.  Given the sample output
>>>> .\gwc\roads\EPSG_900913_12\006_019\000848_002540.png, I assume that
>>>> z=12,
>>>> x=848, and y=2540.  Obviously the above formula won't produce 006_019
>>>> based
>>>> on these numbers.  What conversion is required?
>>>>
>>>> 3.  Is it possible to change the directory/file name format?  I'd like
>>>> to
>>>> create a data set that can be used by Ordnance Survey OpenSpace, which
>>>> uses
>>>> a different structure based on the British National Grid.
>>>>
>>>> Cheers,
>>>>
>>>> Peter
>>>>
>>>>        
>>>
>>> ------------------------------------------------------------------------------
>>> ThinkGeek and WIRED's GeekDad team up for the Ultimate
>>> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
>>> lucky parental unit.  See the prize list and enter to win:
>>> http://p.sf.net/sfu/thinkgeek-promo
>>> _______________________________________________
>>> Geowebcache-users mailing list
>>> Geo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/geowebcache-users
>>>
>>>
>>>      
>>    
> 
> 
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate 
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the 
> lucky parental unit.  See the prize list and enter to win: 
> http://p.sf.net/sfu/thinkgeek-promo
> _______________________________________________
> Geowebcache-users mailing list
> Geo...@li...
> https://lists.sourceforge.net/lists/listinfo/geowebcache-users
> 
> 

-- 
View this message in context: http://old.nabble.com/GeoWebCache-directory-file-name-structure-tp28819540p28847840.html
Sent from the GeoWebCache Users mailing list archive at Nabble.com.