Re: [Opengrads-devel] Dat File re-reads

[Arlindo da S. <da...@al...>]

On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code 7542 <
gar...@nr...> wrote:

> Hi Brian, Jennifer,
>
> We use GrADS extensively in a production mode and have found when profiling
> GrADS runs that 2/3 rds of the data reads are the coastlines and font files.
>  This may not sound like much, but at FNMOC when running 100's of charts for
> each of 32 multi-grid projects, the coastline and font reads amount to
> terabytes for each 12 hour set of runs.
>
> The re-reads occur for each and every display of a variable from the same
> file after it is opened.  We are going to try to store these files in memory
> after the first read.
>
> My first question is whether anyone else has looked at this and has a
> solution?  My next questions, do you have any ideas or guidance to help us
> do this?
>
> I know that GrADS was designed to operate well on small platforms with
> little memory, so our goal is to implement a memory caching capability that
> would be controlled by a 'set' command and would be moderated by the amount
> of memory available.
>

As Brian mentions, the map and font database are relatively small. So, it
would be very feasible to setup a ramdisk to hold these files which would
probably give you a nice performance boost with virtually no programming
involved. Depending on your system architecture, this ramdisk could
effectively function as "shared memory" used by multiple cores. (Because
grads is not thread safe, having an explicit memory buffer to hold this
would require that each grads process duplicate the memory necessary to hold
the maps/font databases.)  One hardware solution is to place grads and its
data on a solid state disk, these are quite affordable these days.

  Just my 2 lazy cents.

       Arlindo

-- 
Arlindo da Silva
da...@al...