Re: [geowebcache-users] GeoWebCache directory/file name structure

[Arne K. <ar...@ti...>]

It's a compromise, and this part of the code is definitely made easily 
pluggable so that one can use a different structures for special needs. 
I was expecting that we would have a few different implementations by 
now, but apparently not.

Every directory level requires a node to be looked up (move the disk 
head), so too many directories is not good either. Within a directory 
there is a proper index. Listing the content of the cache is generally 
not recommended, I don't know of a single filesystem that is optimized 
for that and most utilities can't handle something of this size. "find" 
is usually ok.

But 400 000 directories....  Unless you've discovered a bug, you should 
have at least  80 billion tiles? I wouldn't use a conventional 
filesystem for that, and on a fast machine (100 tiles per second) it 
would take  25 years to build a cache of that size.

-Arne


On 06/10/2010 11:22 AM, samclemmens wrote:
> Cheers, Arne!
>
> I'm curious; what was the rationale for creating a directory structure with
> just one level?  We processed a data set earlier this week, which resulted
> in 400,000 directories.  When I tried to an "ls", my PuTTY console froze...
>
> Peter
>
>
>
> Arne Kepp-2 wrote:
>    
>> 1) "zoom level". For the default EPSG:900913 grid z=0 means the whole
>> world in 4 tiles
>>
>> 2) Hm,,, good point. I dont think I went back and updated that wiki
>> page, it was not more than a scratchpad for sharing ideas. The actual
>> code is here[1], the difference is that the bit shifting starts with 2,
>> effectively making it  2^( 1 +  ( z / 2 )). And there's some zero-padding.
>>
>> 3) The catch is that you need to know a little bit of Java, but if you
>> do then it's potentially straightforward. It depends what the OpenSpace
>> structure is based on, and whether we have similar tokens in GWC.
>>
>> blobstore/file contains an implementation of BlobStore. You can copy the
>> whole thing to a new package, and then make the FilePathGenerator output
>> the structure you want. It is linked into the application using Spring,
>> WEB-INF/geowebcache-core-context.xml , if you are working off trunk. I
>> would guess that you also want a particular GridSet to go with that, to
>> match the resolutions and origin used by OpenSpace.
>>
>> It's possible someone has already done it, but so far I have not heard
>> about it.
>>
>> -Arne
>>
>> 1:
>> http://geowebcache.org/trac/browser/trunk/geowebcache/core/src/main/java/org/geowebcache/storage/blobstore/file/FilePathGenerator.java
>>
>>
>> On 06/08/2010 05:33 PM, samclemmens wrote:
>>      
>>> Hi,
>>>
>>> I'm relatively new to GeoWebCache...and I'm a bit puzzled by the
>>> directory/file name structure that is produced.  I have a few questions,
>>> namely:
>>>
>>> 1.  The documentation states that the naming convention is:
>>> layername/projection_z/[x/(2^(z/2))]_[y/(2^(z/2))]/x_y.extension    What
>>> is
>>> the z value?
>>>
>>> 2.  Given the sample output
>>> .\gwc\roads\EPSG_900913_12\006_019\000848_002540.png, I assume that z=12,
>>> x=848, and y=2540.  Obviously the above formula won't produce 006_019
>>> based
>>> on these numbers.  What conversion is required?
>>>
>>> 3.  Is it possible to change the directory/file name format?  I'd like to
>>> create a data set that can be used by Ordnance Survey OpenSpace, which
>>> uses
>>> a different structure based on the British National Grid.
>>>
>>> Cheers,
>>>
>>> Peter
>>>
>>>        
>>
>> ------------------------------------------------------------------------------
>> ThinkGeek and WIRED's GeekDad team up for the Ultimate
>> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
>> lucky parental unit.  See the prize list and enter to win:
>> http://p.sf.net/sfu/thinkgeek-promo
>> _______________________________________________
>> Geowebcache-users mailing list
>> Geo...@li...
>> https://lists.sourceforge.net/lists/listinfo/geowebcache-users
>>
>>
>>      
>