opengrads-devel — OpenGrADS development issues, with focus on interfaces and extensions

Re: [Opengrads-devel] diff in 'q shades'

[Jennifer A. <jm...@CO...>]

The change from sprintf to snprintf was implemented throughout the  
source code -- I guess it is less prone to memory overrides this way.  
The other change, from '<' to '<=' is deliberate and meaningful -- the  
'q shades' output did not reflect the algorithm for doing the shaded  
contouring. This was a mod checked in on June 10, 2010, so it was in  
2.0.a9. The ChangeLog says, "Fixed color selection for shaded plots:  
lev1 < val <= lev2".
--Jennifer

On Oct 14, 2011, at 2:04 PM, Michael Fiorino wrote:

> hi Brian and Jennifer,
>
> i noticed a diff in 'q shades' in gauser -- now using snprintf...   
> this
> tripped up my python the parses the output, just curious if this was
> intended?  not clear it really in is "<=" vice "<".  the right hand  
> side is the new gauser.c
> the left older.
>
> have a good weekend.
>
> best /r mike

[Opengrads-devel] diff in 'q shades'

[Michael F. <mic...@no...>]

hi Brian and Jennifer,

i noticed a diff in 'q shades' in gauser -- now using snprintf...  this
tripped up my python the parses the output, just curious if this was
intended?  not clear it really in is "<=" vice "<".  the right hand side is the new gauser.c
the left older.

have a good weekend.

best /r mike


  else if (cmpwrd(arg,"shades")) {                                                       |  else if (cmpwrd(arg,"shades")) {

    if (pcm->shdcnt<1) {                                                                 |    if (pcm->shdcnt<1) {

      gaprnt(2,"None\n");                                                                |      gaprnt(2,"None\n");

    } else {                                                                             |    } else {

      sprintf(pout,"Number of levels = %i\n",pcm->shdcnt);                               |      snprintf(pout,255,"Number of levels = %i\n",pcm->shdcnt);

      gaprnt(2,pout);                                                                    |      gaprnt(2,pout);

      for (i=0; i<pcm->shdcnt; i++) {                                                    |      for (i=0; i<pcm->shdcnt; i++) {

        if (i==0)                                                                        |        if (i==0)

          sprintf (pout,"%i < %g\n",pcm->shdcls[i],pcm->shdlvs[1]);                      |          snprintf(pout,255,"%i <= %g\n",pcm->shdcls[i],pcm->shdlvs[1]);

        else if (i==pcm->shdcnt-1)                                                       |        else if (i==pcm->shdcnt-1)

          sprintf (pout,"%i %g >\n",pcm->shdcls[i],pcm->shdlvs[i]);                      |          snprintf(pout,255,"%i %g >\n",pcm->shdcls[i],pcm->shdlvs[i]);

        else                                                                             |        else

          sprintf (pout,"%i %g %g\n",pcm->shdcls[i],pcm->shdlvs[i],pcm->shdlvs[i+1]);    |          snprintf(pout,255,"%i %g %g\n",pcm->shdcls[i],pcm->shdlvs[i],pcm->shdlvs[i+1]);

        gaprnt(2,pout);                                                                  |        gaprnt(2,pout);

      }                                                                                  |      }

    }                                                                                    |    }

  }                                                                                      |  }




i've added your gxwmap.c to our opengrads/src (the only diff was your
caching code) and set

#define CACHEMAX 12000000            

to handle the hires map as Gary suggested

i also added a tweak to 'set grid' to allow setting line thickness,
e.g.,

'set grid on 5 15 5'

to make the default grid a 5 thickness vice the default of 1

the mods are in grads.h, grads.c, gauser.c and gagx.c (attached).

that's the only tweaking i've done with the code the last couple of
years.  i had added 'set z 1 last' but agree that it's not really proper
to support that option since 'z' is not the same kind of coordinate as
't'

best /r mike



On 10/13/11 7:23 PM, Brian Doty wrote:
> Hi Gary, I checked the font behavior, and it seems to be working as
> intended. I put a debug write statement in the routine that reads the
> font file, and it indicates it only gets called once, when a
> particular font is first used. I was testing with the COLA 2.0.0 code
> base. If you want to test with your version, you can put a debug
> write statement at the beginning of the function gxchrd in the
> gxchpl.c source file; gxchrd is the routine that actually reads in a
> font file... Brian
>
> On Oct 13, 2011, at 2:58 PM, Love, Mr. Gary, Contractor, Code 7542
> wrote:
>
>> Hi Brian,
>>
>> I think this solution is exactly on point and I will test it for
>> number of file reads and bytes read. This will be especially useful
>> when I convert to creating a series of plots within the same grads
>> sessions. I will be increasing CACHEMAX because we frequently use
>> the worldmap file for the ultra-high resolution coastlines. Since
>> it is 9671652 bytes, is 10MB big enough for CACHEMAX?
>>
>> When we profiled grads, an equal amount of bytes were read from the
>> font files as were read from the coastline files. I guess every
>> draw string must invoke a font read. Do you have a similar fix for
>> the font problem?
>>
>> Thanks,
>> Gary
>>
>> From: Brian Doty [mailto:do...@co...]
>> Sent: Thursday, October 13, 2011 7:56 AM
>> To: Love, Mr. Gary, Contractor, Code 7542
>> Cc: ope...@li...; Frost, Mr. Michael
>> Subject: Re: [Opengrads-devel] Dat File re-reads
>>
>> Ok, not too hard, I've added some basic caching of the map file to
>> gxwmap.c. The changes are all isolated to gxwmap.c for now. I kinda
>> did this quick so I haven't done a whole lot of testing on it. The
>> cache lasts for the entire grads session. I don't think this hurts
>> performance any for those cases when grads is just being run to do one
>> plot. This mod is applied to the gxwmap.c from the 2.0.0 code base --
>> but we will not include it in 2.0; it will be included in 2.1. Right
>> now the CACHEMAX is set to 2MB which will cache lowres, mres, and
>> hires. If you have your own map files that are larger, you may want
>> to increase the CACHEMAX value. This will have no affect on shapefile
>> usage.
>>
>> Gary, I'm not really sure this will help your situation; it will
>> depend on your system environment. Arlindo's suggestion to set up
>> some system-wide solution like a ramdisk may be more along the lines
>> of what you need... Brian
>>
>>
>>
>>
>> On Oct 12, 2011, at 8:38 PM, Love, Mr. Gary, Contractor, Code 7542
>> wrote:
>>
>>> Arlindo,
>>>
>>> We have some solid state disks that function as local disks because
>>> the global file systems running gpfs are too slow. So our challenge
>>> is how to accomplish how to do the shared memory which we also
>>> thought of using.
>>>
>>> Do you know if anyone else has this problem and who could benefit
>>> from the solution?
>>>
>>> Thanks for your input. Ill ask for more when we get into the
>>> project.
>>>
>>> Gary
>>>
>>> From: Arlindo da Silva [mailto:da...@al...]
>>> Sent: Wednesday, October 12, 2011 4:41 PM
>>> To: ope...@li...
>>> Cc: Brian Doty; Frost, Mr. Michael
>>> Subject: Re: [Opengrads-devel] Dat File re-reads
>>>
>>> On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code
>>> 7542 <gar...@nr...> wrote:
>>> Hi Brian, Jennifer,
>>>
>>> We use GrADS extensively in a production mode and have found when
>>> profiling GrADS runs that 2/3 rds of the data reads are the
>>> coastlines and font files. This may not sound like much, but at
>>> FNMOC when running 100's of charts for each of 32 multi-grid
>>> projects, the coastline and font reads amount to terabytes for each
>>> 12 hour set of runs.
>>>
>>> The re-reads occur for each and every display of a variable from the
>>> same file after it is opened. We are going to try to store these
>>> files in memory after the first read.
>>>
>>> My first question is whether anyone else has looked at this and has
>>> a solution? My next questions, do you have any ideas or guidance to
>>> help us do this?
>>>
>>> I know that GrADS was designed to operate well on small platforms
>>> with little memory, so our goal is to implement a memory caching
>>> capability that would be controlled by a 'set' command and would be
>>> moderated by the amount of memory available.
>>>
>>> As Brian mentions, the map and font database are relatively small.
>>> So, it would be very feasible to setup a ramdisk to hold these files
>>> which would probably give you a nice performance boost with
>>> virtually no programming involved. Depending on your system
>>> architecture, this ramdisk could effectively function as "shared
>>> memory" used by multiple cores. (Because grads is not thread safe,
>>> having an explicit memory buffer to hold this would require that
>>> each grads process duplicate the memory necessary to hold the maps/
>>> font databases.) One hardware solution is to place grads and its
>>> data on a solid state disk, these are quite affordable these days.
>>>
>>> Just my 2 lazy cents.
>>>
>>> Arlindo
>>>
>>> --
>>> Arlindo da Silva
>>> da...@al...
>>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2d-oct
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

Re: [Opengrads-devel] Dat File re-reads -- a small tweak

[Love, M. G. C. C. 7. <gar...@nr...>]

Thanks guys, the response has been tremendous!  Gary

From: Michael Fiorino [mailto:Mic...@no...]
Sent: Thursday, October 13, 2011 2:13 PM
To: ope...@li...; Jennifer Adams
Cc: Brian Doty; Love, Mr. Gary, Contractor, Code 7542; Frost, Mr. Michael
Subject: Re: [Opengrads-devel] Dat File re-reads -- a small tweak

hi Brian,

i've added your gxwmap.c to our opengrads/src (the only diff was your
caching code) and set

#define CACHEMAX 12000000

to handle the hires map as Gary suggested

i also added a tweak to 'set grid' to allow setting line thickness,
e.g.,

'set grid on 5 15 5'

to make the default grid a 5 thickness vice the default of 1

the mods are in grads.h, grads.c, gauser.c and gagx.c (attached).

that's the only tweaking i've done with the code the last couple of
years.  i had added 'set z 1 last' but agree that it's not really proper
to support that option since 'z' is not the same kind of coordinate as
't'

best /r mike



On 10/13/11 7:23 PM, Brian Doty wrote:
> Hi Gary, I checked the font
      behavior, and it seems to be working as

      > intended. I put a debug write statement in the routine that
      reads the

      > font file, and it indicates it only gets called once, when a


      > particular font is first used. I was testing with the COLA
      2.0.0 code

      > base. If you want to test with your version, you can put a
      debug

      > write statement at the beginning of the function gxchrd in
      the

      > gxchpl.c source file; gxchrd is the routine that actually
      reads in a

      > font file... Brian

      >

      > On Oct 13, 2011, at 2:58 PM, Love, Mr. Gary, Contractor, Code
      7542

      > wrote:

      >

      >> Hi Brian,

      >>

      >> I think this solution is exactly on point and I will test
      it for

      >> number of file reads and bytes read. This will be
      especially useful

      >> when I convert to creating a series of plots within the
      same grads

      >> sessions. I will be increasing CACHEMAX because we
      frequently use

      >> the worldmap file for the ultra-high resolution
      coastlines. Since

      >> it is 9671652 bytes, is 10MB big enough for CACHEMAX?

      >>

      >> When we profiled grads, an equal amount of bytes were
      read from the

      >> font files as were read from the coastline files. I guess
      every

      >> draw string must invoke a font read. Do you have a
      similar fix for

      >> the font problem?

      >>

      >> Thanks,

      >> Gary

      >>

      >> From: Brian Doty [mailto:do...@co...]

      >> Sent: Thursday, October 13, 2011 7:56 AM

      >> To: Love, Mr. Gary, Contractor, Code 7542

      >> Cc: ope...@li...<mailto:ope...@li...>; Frost, Mr.
     Michael

      >> Subject: Re: [Opengrads-devel] Dat File re-reads

      >>

      >> Ok, not too hard, I've added some basic caching of the
      map file to

      >> gxwmap.c. The changes are all isolated to gxwmap.c for
      now. I kinda

      >> did this quick so I haven't done a whole lot of testing
      on it. The

      >> cache lasts for the entire grads session. I don't think
      this hurts

      >> performance any for those cases when grads is just being
      run to do one

      >> plot. This mod is applied to the gxwmap.c from the 2.0.0
      code base --

      >> but we will not include it in 2.0; it will be included in
      2.1. Right

      >> now the CACHEMAX is set to 2MB which will cache lowres,
      mres, and

      >> hires. If you have your own map files that are larger,
      you may want

      >> to increase the CACHEMAX value. This will have no affect
      on shapefile

      >> usage.

      >>

      >> Gary, I'm not really sure this will help your situation;
      it will

      >> depend on your system environment. Arlindo's suggestion
      to set up

      >> some system-wide solution like a ramdisk may be more
      along the lines

      >> of what you need... Brian

      >>

      >>

      >>

      >>

      >> On Oct 12, 2011, at 8:38 PM, Love, Mr. Gary, Contractor,
      Code 7542

      >> wrote:

      >>

      >>> Arlindo,

      >>>

      >>> We have some solid state disks that function as local
      disks because

      >>> the global file systems running gpfs are too slow. So
      our challenge

      >>> is how to accomplish how to do the shared memory
      which we also

      >>> thought of using.

      >>>

      >>> Do you know if anyone else has this problem and who
      could benefit

      >>> from the solution?

      >>>

      >>> Thanks for your input. Ill ask for more when we get
      into the

      >>> project.

      >>>

      >>> Gary

      >>>

      >>> From: Arlindo da Silva [mailto:da...@al...]

      >>> Sent: Wednesday, October 12, 2011 4:41 PM

      >>> To: ope...@li...<mailto:ope...@li...>

      >>> Cc: Brian Doty; Frost, Mr. Michael

      >>> Subject: Re: [Opengrads-devel] Dat File re-reads

      >>>

      >>> On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary,
      Contractor, Code

      >>> 7542 <gar...@nr...><mailto:gar...@nr...> wrote:

      >>> Hi Brian, Jennifer,

      >>>

      >>> We use GrADS extensively in a production mode and
      have found when

      >>> profiling GrADS runs that 2/3 rds of the data reads
      are the

      >>> coastlines and font files. This may not sound like
      much, but at

      >>> FNMOC when running 100's of charts for each of 32
      multi-grid

      >>> projects, the coastline and font reads amount to
      terabytes for each

      >>> 12 hour set of runs.

      >>>

      >>> The re-reads occur for each and every display of a
      variable from the

      >>> same file after it is opened. We are going to try to
      store these

      >>> files in memory after the first read.

      >>>

      >>> My first question is whether anyone else has looked
      at this and has

      >>> a solution? My next questions, do you have any ideas
      or guidance to

      >>> help us do this?

      >>>

      >>> I know that GrADS was designed to operate well on
      small platforms

      >>> with little memory, so our goal is to implement a
      memory caching

      >>> capability that would be controlled by a 'set'
      command and would be

      >>> moderated by the amount of memory available.

      >>>

      >>> As Brian mentions, the map and font database are
      relatively small.

      >>> So, it would be very feasible to setup a ramdisk to
      hold these files

      >>> which would probably give you a nice performance
      boost with

      >>> virtually no programming involved. Depending on your
      system

      >>> architecture, this ramdisk could effectively function
      as "shared

      >>> memory" used by multiple cores. (Because grads is not
      thread safe,

      >>> having an explicit memory buffer to hold this would
      require that

      >>> each grads process duplicate the memory necessary to
      hold the maps/

      >>> font databases.) One hardware solution is to place
      grads and its

      >>> data on a solid state disk, these are quite
      affordable these days.

      >>>

      >>> Just my 2 lazy cents.

      >>>

      >>> Arlindo

      >>>

      >>> --

      >>> Arlindo da Silva

      >>> da...@al...<mailto:da...@al...>

      >>

      >

      >

      >
------------------------------------------------------------------------------

      > All the data continuously generated in your IT infrastructure
      contains a

      > definitive record of customers, application performance,
      security

      > threats, fraudulent activity and more. Splunk takes this data
      and makes

      > sense of it. Business sense. IT sense. Common sense.

      > http://p.sf.net/sfu/splunk-d2d-oct

      > _______________________________________________

      > Opengrads-devel mailing list

      > Ope...@li...<mailto:Ope...@li...>

      > https://lists.sourceforge.net/lists/listinfo/opengrads-devel

Re: [Opengrads-devel] Dat File re-reads

[Brian D. <do...@co...>]

Hi Gary, I checked the font behavior, and it seems to be working as  
intended.  I put a debug write statement in the routine that reads the  
font file, and it indicates it only gets called once, when a  
particular font is first used.  I was testing with the COLA 2.0.0 code  
base.  If you want to test with your version, you can put a debug  
write statement at the beginning of the function gxchrd in the  
gxchpl.c source file; gxchrd is the routine that actually reads in a  
font file... Brian

On Oct 13, 2011, at 2:58 PM, Love, Mr. Gary, Contractor, Code 7542  
wrote:

> Hi Brian,
>
> I think this solution is exactly on point and I will test it for  
> number of file reads and bytes read.  This will be especially useful  
> when I convert to creating a series of plots within the same grads  
> sessions.  I will be increasing CACHEMAX because we frequently use  
> the worldmap file for the ultra-high resolution coastlines.  Since  
> it is  9671652 bytes, is 10MB big enough for CACHEMAX?
>
> When we profiled grads, an equal amount of bytes were read from the  
> font files as were read from the coastline files.  I guess every  
> draw string must invoke a font read. Do you have a similar fix for  
> the font problem?
>
> Thanks,
> Gary
>
> From: Brian Doty [mailto:do...@co...]
> Sent: Thursday, October 13, 2011 7:56 AM
> To: Love, Mr. Gary, Contractor, Code 7542
> Cc: ope...@li...; Frost, Mr. Michael
> Subject: Re: [Opengrads-devel] Dat File re-reads
>
> Ok, not too hard, I've added some basic caching of the map file to
> gxwmap.c.  The changes are all isolated to gxwmap.c for now.  I kinda
> did this quick so I haven't done a whole lot of testing on it.  The
> cache lasts for the entire grads session.  I don't think this hurts
> performance any for those cases when grads is just being run to do one
> plot.  This mod is applied to the gxwmap.c from the 2.0.0 code base --
> but we will not include it in 2.0; it will be included in 2.1.  Right
> now the CACHEMAX is set to 2MB which will cache lowres, mres, and
> hires.  If you have your own map files that are larger, you may want
> to increase the CACHEMAX value.  This will have no affect on shapefile
> usage.
>
> Gary, I'm not really sure this will help your situation; it will
> depend on your system environment.  Arlindo's suggestion to set up
> some system-wide solution like a ramdisk may be more along the lines
> of what you need... Brian
>
>
>
>
> On Oct 12, 2011, at 8:38 PM, Love, Mr. Gary, Contractor, Code 7542
> wrote:
>
> > Arlindo,
> >
> > We have some solid state disks that function as local disks because
> > the global file systems running gpfs are too slow.  So our challenge
> > is how to accomplish how to do the “shared memory” which we also
> > thought of using.
> >
> > Do you know if anyone else has this problem and who could benefit
> > from the solution?
> >
> > Thanks for your input.  I’ll ask for more when we get into the
> > project.
> >
> > Gary
> >
> > From: Arlindo da Silva [mailto:da...@al...]
> > Sent: Wednesday, October 12, 2011 4:41 PM
> > To: ope...@li...
> > Cc: Brian Doty; Frost, Mr. Michael
> > Subject: Re: [Opengrads-devel] Dat File re-reads
> >
> > On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code
> > 7542 <gar...@nr...> wrote:
> > Hi Brian, Jennifer,
> >
> > We use GrADS extensively in a production mode and have found when
> > profiling GrADS runs that 2/3 rds of the data reads are the
> > coastlines and font files.  This may not sound like much, but at
> > FNMOC when running 100's of charts for each of 32 multi-grid
> > projects, the coastline and font reads amount to terabytes for each
> > 12 hour set of runs.
> >
> > The re-reads occur for each and every display of a variable from the
> > same file after it is opened.  We are going to try to store these
> > files in memory after the first read.
> >
> > My first question is whether anyone else has looked at this and has
> > a solution?  My next questions, do you have any ideas or guidance to
> > help us do this?
> >
> > I know that GrADS was designed to operate well on small platforms
> > with little memory, so our goal is to implement a memory caching
> > capability that would be controlled by a 'set' command and would be
> > moderated by the amount of memory available.
> >
> > As Brian mentions, the map and font database are relatively small.
> > So, it would be very feasible to setup a ramdisk to hold these files
> > which would probably give you a nice performance boost with
> > virtually no programming involved. Depending on your system
> > architecture, this ramdisk could effectively function as "shared
> > memory" used by multiple cores. (Because grads is not thread safe,
> > having an explicit memory buffer to hold this would require that
> > each grads process duplicate the memory necessary to hold the maps/
> > font databases.)  One hardware solution is to place grads and its
> > data on a solid state disk, these are quite affordable these days.
> >
> >   Just my 2 lazy cents.
> >
> >        Arlindo
> >
> > --
> > Arlindo da Silva
> > da...@al...
>

Re: [Opengrads-devel] Dat File re-reads

[Brian D. <do...@co...>]

Dear Gary, yes if you make CACHEMAX larger than 9671652, the larger  
map file will be cached.  I am not sure about the font files.  The  
intent of the old code is that they should only be getting read  
once.   I will take a look again at the code -- there may be a bug  
there... Brian

On Oct 13, 2011, at 2:58 PM, Love, Mr. Gary, Contractor, Code 7542  
wrote:

> Hi Brian,
>
> I think this solution is exactly on point and I will test it for  
> number of file reads and bytes read.  This will be especially useful  
> when I convert to creating a series of plots within the same grads  
> sessions.  I will be increasing CACHEMAX because we frequently use  
> the worldmap file for the ultra-high resolution coastlines.  Since  
> it is  9671652 bytes, is 10MB big enough for CACHEMAX?
>
> When we profiled grads, an equal amount of bytes were read from the  
> font files as were read from the coastline files.  I guess every  
> draw string must invoke a font read. Do you have a similar fix for  
> the font problem?
>
> Thanks,
> Gary
>
> From: Brian Doty [mailto:do...@co...]
> Sent: Thursday, October 13, 2011 7:56 AM
> To: Love, Mr. Gary, Contractor, Code 7542
> Cc: ope...@li...; Frost, Mr. Michael
> Subject: Re: [Opengrads-devel] Dat File re-reads
>
> Ok, not too hard, I've added some basic caching of the map file to
> gxwmap.c.  The changes are all isolated to gxwmap.c for now.  I kinda
> did this quick so I haven't done a whole lot of testing on it.  The
> cache lasts for the entire grads session.  I don't think this hurts
> performance any for those cases when grads is just being run to do one
> plot.  This mod is applied to the gxwmap.c from the 2.0.0 code base --
> but we will not include it in 2.0; it will be included in 2.1.  Right
> now the CACHEMAX is set to 2MB which will cache lowres, mres, and
> hires.  If you have your own map files that are larger, you may want
> to increase the CACHEMAX value.  This will have no affect on shapefile
> usage.
>
> Gary, I'm not really sure this will help your situation; it will
> depend on your system environment.  Arlindo's suggestion to set up
> some system-wide solution like a ramdisk may be more along the lines
> of what you need... Brian
>
>
>
>
> On Oct 12, 2011, at 8:38 PM, Love, Mr. Gary, Contractor, Code 7542
> wrote:
>
> > Arlindo,
> >
> > We have some solid state disks that function as local disks because
> > the global file systems running gpfs are too slow.  So our challenge
> > is how to accomplish how to do the “shared memory” which we also
> > thought of using.
> >
> > Do you know if anyone else has this problem and who could benefit
> > from the solution?
> >
> > Thanks for your input.  I’ll ask for more when we get into the
> > project.
> >
> > Gary
> >
> > From: Arlindo da Silva [mailto:da...@al...]
> > Sent: Wednesday, October 12, 2011 4:41 PM
> > To: ope...@li...
> > Cc: Brian Doty; Frost, Mr. Michael
> > Subject: Re: [Opengrads-devel] Dat File re-reads
> >
> > On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code
> > 7542 <gar...@nr...> wrote:
> > Hi Brian, Jennifer,
> >
> > We use GrADS extensively in a production mode and have found when
> > profiling GrADS runs that 2/3 rds of the data reads are the
> > coastlines and font files.  This may not sound like much, but at
> > FNMOC when running 100's of charts for each of 32 multi-grid
> > projects, the coastline and font reads amount to terabytes for each
> > 12 hour set of runs.
> >
> > The re-reads occur for each and every display of a variable from the
> > same file after it is opened.  We are going to try to store these
> > files in memory after the first read.
> >
> > My first question is whether anyone else has looked at this and has
> > a solution?  My next questions, do you have any ideas or guidance to
> > help us do this?
> >
> > I know that GrADS was designed to operate well on small platforms
> > with little memory, so our goal is to implement a memory caching
> > capability that would be controlled by a 'set' command and would be
> > moderated by the amount of memory available.
> >
> > As Brian mentions, the map and font database are relatively small.
> > So, it would be very feasible to setup a ramdisk to hold these files
> > which would probably give you a nice performance boost with
> > virtually no programming involved. Depending on your system
> > architecture, this ramdisk could effectively function as "shared
> > memory" used by multiple cores. (Because grads is not thread safe,
> > having an explicit memory buffer to hold this would require that
> > each grads process duplicate the memory necessary to hold the maps/
> > font databases.)  One hardware solution is to place grads and its
> > data on a solid state disk, these are quite affordable these days.
> >
> >   Just my 2 lazy cents.
> >
> >        Arlindo
> >
> > --
> > Arlindo da Silva
> > da...@al...
>

Re: [Opengrads-devel] Dat File re-reads

[Love, M. G. C. C. 7. <gar...@nr...>]

Hi Brian,

I think this solution is exactly on point and I will test it for number of file reads and bytes read.  This will be especially useful when I convert to creating a series of plots within the same grads sessions.  I will be increasing CACHEMAX because we frequently use the worldmap file for the ultra-high resolution coastlines.  Since it is  9671652 bytes, is 10MB big enough for CACHEMAX?

When we profiled grads, an equal amount of bytes were read from the font files as were read from the coastline files.  I guess every draw string must invoke a font read. Do you have a similar fix for the font problem?

Thanks,
Gary

From: Brian Doty [mailto:do...@co...]
Sent: Thursday, October 13, 2011 7:56 AM
To: Love, Mr. Gary, Contractor, Code 7542
Cc: ope...@li...; Frost, Mr. Michael
Subject: Re: [Opengrads-devel] Dat File re-reads

Ok, not too hard, I've added some basic caching of the map file to
gxwmap.c.  The changes are all isolated to gxwmap.c for now.  I kinda
did this quick so I haven't done a whole lot of testing on it.  The
cache lasts for the entire grads session.  I don't think this hurts
performance any for those cases when grads is just being run to do one
plot.  This mod is applied to the gxwmap.c from the 2.0.0 code base --
but we will not include it in 2.0; it will be included in 2.1.  Right
now the CACHEMAX is set to 2MB which will cache lowres, mres, and
hires.  If you have your own map files that are larger, you may want
to increase the CACHEMAX value.  This will have no affect on shapefile
usage.

Gary, I'm not really sure this will help your situation; it will
depend on your system environment.  Arlindo's suggestion to set up
some system-wide solution like a ramdisk may be more along the lines
of what you need... Brian



On Oct 12, 2011, at 8:38 PM, Love, Mr. Gary, Contractor, Code 7542
wrote:

> Arlindo,
>
> We have some solid state disks that function as local disks because
> the global file systems running gpfs are too slow.  So our challenge
> is how to accomplish how to do the “shared memory” which we also
> thought of using.
>
> Do you know if anyone else has this problem and who could benefit
> from the solution?
>
> Thanks for your input.  I’ll ask for more when we get into the
> project.
>
> Gary
>
> From: Arlindo da Silva [mailto:da...@al...]
> Sent: Wednesday, October 12, 2011 4:41 PM
> To: ope...@li...<mailto:ope...@li...>
> Cc: Brian Doty; Frost, Mr. Michael
> Subject: Re: [Opengrads-devel] Dat File re-reads
>
> On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code
> 7542 <gar...@nr...<mailto:gar...@nr...>> wrote:
> Hi Brian, Jennifer,
>
> We use GrADS extensively in a production mode and have found when
> profiling GrADS runs that 2/3 rds of the data reads are the
> coastlines and font files.  This may not sound like much, but at
> FNMOC when running 100's of charts for each of 32 multi-grid
> projects, the coastline and font reads amount to terabytes for each
> 12 hour set of runs.
>
> The re-reads occur for each and every display of a variable from the
> same file after it is opened.  We are going to try to store these
> files in memory after the first read.
>
> My first question is whether anyone else has looked at this and has
> a solution?  My next questions, do you have any ideas or guidance to
> help us do this?
>
> I know that GrADS was designed to operate well on small platforms
> with little memory, so our goal is to implement a memory caching
> capability that would be controlled by a 'set' command and would be
> moderated by the amount of memory available.
>
> As Brian mentions, the map and font database are relatively small.
> So, it would be very feasible to setup a ramdisk to hold these files
> which would probably give you a nice performance boost with
> virtually no programming involved. Depending on your system
> architecture, this ramdisk could effectively function as "shared
> memory" used by multiple cores. (Because grads is not thread safe,
> having an explicit memory buffer to hold this would require that
> each grads process duplicate the memory necessary to hold the maps/
> font databases.)  One hardware solution is to place grads and its
> data on a solid state disk, these are quite affordable these days.
>
>   Just my 2 lazy cents.
>
>        Arlindo
>
> --
> Arlindo da Silva
> da...@al...<mailto:da...@al...>

Re: [Opengrads-devel] Dat File re-reads

[Mike F. <mfi...@gm...>]

thanks Brian and all for the good discussion regarding performance.  i too generate order 
10,000 images / day and would hopefully benefit from the cache.  i'll build and see what 
happens.  best /r mike

On 10/13/11 2:56 PM, Brian Doty wrote:
> Ok, not too hard, I've added some basic caching of the map file to gxwmap.c.  The changes 
> are all isolated to gxwmap.c for now.  I kinda did this quick so I haven't done a whole 
> lot of testing on it.  The cache lasts for the entire grads session.  I don't think this 
> hurts performance any for those cases when grads is just being run to do one plot.  This 
> mod is applied to the gxwmap.c from the 2.0.0 code base -- but we will not include it in 
> 2.0; it will be included in 2.1.  Right now the CACHEMAX is set to 2MB which will cache 
> lowres, mres, and hires.  If you have your own map files that are larger, you may want to 
> increase the CACHEMAX value.  This will have no affect on shapefile usage.
>
> Gary, I'm not really sure this will help your situation; it will depend on your system 
> environment.  Arlindo's suggestion to set up some system-wide solution like a ramdisk may 
> be more along the lines of what you need... Brian
>
>
>
>
> On Oct 12, 2011, at 8:38 PM, Love, Mr. Gary, Contractor, Code 7542 wrote:
>
>> Arlindo,
>>
>> We have some solid state disks that function as local disks because the global file 
>> systems running gpfs are too slow.  So our challenge is how to accomplish how to do the 
>> "shared memory" which we also thought of using.
>>
>> Do you know if anyone else has this problem and who could benefit from the solution?
>>
>> Thanks for your input.  I'll ask for more when we get into the project.
>>
>> Gary
>>
>> From: Arlindo da Silva [mailto:da...@al...]
>> Sent: Wednesday, October 12, 2011 4:41 PM
>> To: ope...@li...
>> Cc: Brian Doty; Frost, Mr. Michael
>> Subject: Re: [Opengrads-devel] Dat File re-reads
>>
>> On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code 7542 
>> <gar...@nr...> wrote:
>> Hi Brian, Jennifer,
>>
>> We use GrADS extensively in a production mode and have found when profiling GrADS runs 
>> that 2/3 rds of the data reads are the coastlines and font files.  This may not sound 
>> like much, but at FNMOC when running 100's of charts for each of 32 multi-grid projects, 
>> the coastline and font reads amount to terabytes for each 12 hour set of runs.
>>
>> The re-reads occur for each and every display of a variable from the same file after it 
>> is opened.  We are going to try to store these files in memory after the first read.
>>
>> My first question is whether anyone else has looked at this and has a solution?  My next 
>> questions, do you have any ideas or guidance to help us do this?
>>
>> I know that GrADS was designed to operate well on small platforms with little memory, so 
>> our goal is to implement a memory caching capability that would be controlled by a 'set' 
>> command and would be moderated by the amount of memory available.
>>
>> As Brian mentions, the map and font database are relatively small. So, it would be very 
>> feasible to setup a ramdisk to hold these files which would probably give you a nice 
>> performance boost with virtually no programming involved. Depending on your system 
>> architecture, this ramdisk could effectively function as "shared memory" used by multiple 
>> cores. (Because grads is not thread safe, having an explicit memory buffer to hold this 
>> would require that each grads process duplicate the memory necessary to hold the 
>> maps/font databases.)  One hardware solution is to place grads and its data on a solid 
>> state disk, these are quite affordable these days.
>>
>>   Just my 2 lazy cents.
>>
>>        Arlindo
>>
>> -- 
>> Arlindo da Silva
>> da...@al...
>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2d-oct
>
>
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

Re: [Opengrads-devel] Dat File re-reads

[Brian D. <do...@co...>]

Ok, not too hard, I've added some basic caching of the map file to  
gxwmap.c.  The changes are all isolated to gxwmap.c for now.  I kinda  
did this quick so I haven't done a whole lot of testing on it.  The  
cache lasts for the entire grads session.  I don't think this hurts  
performance any for those cases when grads is just being run to do one  
plot.  This mod is applied to the gxwmap.c from the 2.0.0 code base --  
but we will not include it in 2.0; it will be included in 2.1.  Right  
now the CACHEMAX is set to 2MB which will cache lowres, mres, and  
hires.  If you have your own map files that are larger, you may want  
to increase the CACHEMAX value.  This will have no affect on shapefile  
usage.

Gary, I'm not really sure this will help your situation; it will  
depend on your system environment.  Arlindo's suggestion to set up  
some system-wide solution like a ramdisk may be more along the lines  
of what you need... Brian

Re: [Opengrads-devel] Dat File re-reads

[Brian D. <do...@co...>]

Hi Gary, these days the map files should be cached, at least the the  
smaller ones.  I will put this code in to gxwmap.c. This will benefit  
COLA and others.  With the new distributed parallel file systems, the  
old assumptions about system file i/o caching are no longer valid in  
many cases... Brian

On Oct 12, 2011, at 8:38 PM, Love, Mr. Gary, Contractor, Code 7542  
wrote:

> Arlindo,
>
> We have some solid state disks that function as local disks because  
> the global file systems running gpfs are too slow.  So our challenge  
> is how to accomplish how to do the “shared memory” which we also  
> thought of using.
>
> Do you know if anyone else has this problem and who could benefit  
> from the solution?
>
> Thanks for your input.  I’ll ask for more when we get into the  
> project.
>
> Gary
>
> From: Arlindo da Silva [mailto:da...@al...]
> Sent: Wednesday, October 12, 2011 4:41 PM
> To: ope...@li...
> Cc: Brian Doty; Frost, Mr. Michael
> Subject: Re: [Opengrads-devel] Dat File re-reads
>
> On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code  
> 7542 <gar...@nr...> wrote:
> Hi Brian, Jennifer,
>
> We use GrADS extensively in a production mode and have found when  
> profiling GrADS runs that 2/3 rds of the data reads are the  
> coastlines and font files.  This may not sound like much, but at  
> FNMOC when running 100's of charts for each of 32 multi-grid  
> projects, the coastline and font reads amount to terabytes for each  
> 12 hour set of runs.
>
> The re-reads occur for each and every display of a variable from the  
> same file after it is opened.  We are going to try to store these  
> files in memory after the first read.
>
> My first question is whether anyone else has looked at this and has  
> a solution?  My next questions, do you have any ideas or guidance to  
> help us do this?
>
> I know that GrADS was designed to operate well on small platforms  
> with little memory, so our goal is to implement a memory caching  
> capability that would be controlled by a 'set' command and would be  
> moderated by the amount of memory available.
>
> As Brian mentions, the map and font database are relatively small.  
> So, it would be very feasible to setup a ramdisk to hold these files  
> which would probably give you a nice performance boost with  
> virtually no programming involved. Depending on your system  
> architecture, this ramdisk could effectively function as "shared  
> memory" used by multiple cores. (Because grads is not thread safe,  
> having an explicit memory buffer to hold this would require that  
> each grads process duplicate the memory necessary to hold the maps/ 
> font databases.)  One hardware solution is to place grads and its  
> data on a solid state disk, these are quite affordable these days.
>
>   Just my 2 lazy cents.
>
>        Arlindo
>
> -- 
> Arlindo da Silva
> da...@al...

Re: [Opengrads-devel] Dat File re-reads

[Arlindo da S. <da...@al...>]

On Wed, Oct 12, 2011 at 8:38 PM, Love, Mr. Gary, Contractor, Code 7542 <
gar...@nr...> wrote:

> Arlindo,****
>
> ** **
>
> We have some solid state disks that function as local disks because the
> global file systems running gpfs are too slow.
>

gpfs has been a big disappointment...



> So our challenge is how to accomplish how to do the “shared memory” which
> we also thought of using.****
>
> **
>

Do you mean standard "Unix shared memory" or "openMP" type of shared memory
parallel programing? I have looked into Unix Shared Memory as an approach
for UDF data exchange but found it a bit too clumsy to program into. (I take
it back: it is much simpler if you only need to read from it.) But, really,
I'd try the ramdisk approach first because it is so simple. I am assuming
you are on a linux platform, correct?


> **
>
> Do you know if anyone else has this problem and who could benefit from the
> solution?****
>
> **
>


I used a similar approach recently. As you know, gxyat() is both an off-line
translator or grads metafiles into png, pdf, etc, as well as a replacement
for printim. I use absolutely the same code for both. In the on-line
(printim replacement) I have

#define fread udx_mread

where udx_mread() is a function that works pretty much like fread() but
instead of reading from a file it reads from the grads graphics (memory)
buffer.

Now, this makes wonder. Do you really want to cache the map/font database or
the rendering "strokes" associated with it? For example, we could have this
functionality:

ga-> enable gx_cache joseph
ga-> draw map
ga-> disable gx_cache

where "joseph" is the name of the gx cache. Next time you wanted that map
drawn you could simply draw it

ga-> draw gx_cache joseph

and the map would be redrawn, possibly even with clipping, without reading
any file. This would be trivial to implement; all you would need to do is to
make a copy of the graphics buffer between  enable/disable gx_cache.

  Arlindo





> **
>
> Thanks for your input.  I’ll ask for more when we get into the project.***
> *
>
> ** **
>
> Gary****
>
> ** **
>
> *From:* Arlindo da Silva [mailto:da...@al...]
> *Sent:* Wednesday, October 12, 2011 4:41 PM
> *To:* ope...@li...
> *Cc:* Brian Doty; Frost, Mr. Michael
> *Subject:* Re: [Opengrads-devel] Dat File re-reads****
>
> ** **
>
> On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code 7542 <
> gar...@nr...> wrote:****
>
> Hi Brian, Jennifer,
>
> We use GrADS extensively in a production mode and have found when profiling
> GrADS runs that 2/3 rds of the data reads are the coastlines and font files.
>  This may not sound like much, but at FNMOC when running 100's of charts for
> each of 32 multi-grid projects, the coastline and font reads amount to
> terabytes for each 12 hour set of runs.
>
> The re-reads occur for each and every display of a variable from the same
> file after it is opened.  We are going to try to store these files in memory
> after the first read.
>
> My first question is whether anyone else has looked at this and has a
> solution?  My next questions, do you have any ideas or guidance to help us
> do this?
>
> I know that GrADS was designed to operate well on small platforms with
> little memory, so our goal is to implement a memory caching capability that
> would be controlled by a 'set' command and would be moderated by the amount
> of memory available.****
>
> ** **
>
> As Brian mentions, the map and font database are relatively small. So, it
> would be very feasible to setup a ramdisk to hold these files which would
> probably give you a nice performance boost with virtually no programming
> involved. Depending on your system architecture, this ramdisk could
> effectively function as "shared memory" used by multiple cores. (Because
> grads is not thread safe, having an explicit memory buffer to hold this
> would require that each grads process duplicate the memory necessary to hold
> the maps/font databases.)  One hardware solution is to place grads and its
> data on a solid state disk, these are quite affordable these days.****
>
> ** **
>
>   Just my 2 lazy cents.****
>
> ** **
>
>        Arlindo****
>
>  ****
>
> --
> Arlindo da Silva
> da...@al...****
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2d-oct
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>
>


-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] Dat File re-reads

[Love, M. G. C. C. 7. <gar...@nr...>]

Arlindo,

We have some solid state disks that function as local disks because the global file systems running gpfs are too slow.  So our challenge is how to accomplish how to do the "shared memory" which we also thought of using.

Do you know if anyone else has this problem and who could benefit from the solution?

Thanks for your input.  I'll ask for more when we get into the project.

Gary

From: Arlindo da Silva [mailto:da...@al...]
Sent: Wednesday, October 12, 2011 4:41 PM
To: ope...@li...
Cc: Brian Doty; Frost, Mr. Michael
Subject: Re: [Opengrads-devel] Dat File re-reads

On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code 7542 <gar...@nr...<mailto:gar...@nr...>> wrote:
Hi Brian, Jennifer,

We use GrADS extensively in a production mode and have found when profiling GrADS runs that 2/3 rds of the data reads are the coastlines and font files.  This may not sound like much, but at FNMOC when running 100's of charts for each of 32 multi-grid projects, the coastline and font reads amount to terabytes for each 12 hour set of runs.

The re-reads occur for each and every display of a variable from the same file after it is opened.  We are going to try to store these files in memory after the first read.

My first question is whether anyone else has looked at this and has a solution?  My next questions, do you have any ideas or guidance to help us do this?

I know that GrADS was designed to operate well on small platforms with little memory, so our goal is to implement a memory caching capability that would be controlled by a 'set' command and would be moderated by the amount of memory available.

As Brian mentions, the map and font database are relatively small. So, it would be very feasible to setup a ramdisk to hold these files which would probably give you a nice performance boost with virtually no programming involved. Depending on your system architecture, this ramdisk could effectively function as "shared memory" used by multiple cores. (Because grads is not thread safe, having an explicit memory buffer to hold this would require that each grads process duplicate the memory necessary to hold the maps/font databases.)  One hardware solution is to place grads and its data on a solid state disk, these are quite affordable these days.

  Just my 2 lazy cents.

       Arlindo

--
Arlindo da Silva
da...@al...<mailto:da...@al...>

Re: [Opengrads-devel] Dat File re-reads

[Arlindo da S. <da...@al...>]

On Wed, Oct 12, 2011 at 3:16 PM, Love, Mr. Gary, Contractor, Code 7542 <
gar...@nr...> wrote:

> Hi Brian, Jennifer,
>
> We use GrADS extensively in a production mode and have found when profiling
> GrADS runs that 2/3 rds of the data reads are the coastlines and font files.
>  This may not sound like much, but at FNMOC when running 100's of charts for
> each of 32 multi-grid projects, the coastline and font reads amount to
> terabytes for each 12 hour set of runs.
>
> The re-reads occur for each and every display of a variable from the same
> file after it is opened.  We are going to try to store these files in memory
> after the first read.
>
> My first question is whether anyone else has looked at this and has a
> solution?  My next questions, do you have any ideas or guidance to help us
> do this?
>
> I know that GrADS was designed to operate well on small platforms with
> little memory, so our goal is to implement a memory caching capability that
> would be controlled by a 'set' command and would be moderated by the amount
> of memory available.
>

As Brian mentions, the map and font database are relatively small. So, it
would be very feasible to setup a ramdisk to hold these files which would
probably give you a nice performance boost with virtually no programming
involved. Depending on your system architecture, this ramdisk could
effectively function as "shared memory" used by multiple cores. (Because
grads is not thread safe, having an explicit memory buffer to hold this
would require that each grads process duplicate the memory necessary to hold
the maps/font databases.)  One hardware solution is to place grads and its
data on a solid state disk, these are quite affordable these days.

  Just my 2 lazy cents.

       Arlindo

-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] Dat File re-reads

[Brian D. <do...@co...>]

Hi Gary, what you report is of concern and should be fixed.  You are  
correct that the old rationale for the way it works no longer applies.

The map and font files that come with grads are quite small by today's  
standards.  Even the hires data is only 1.7MB.  If you have your own  
map files they may be bigger.

Looking at the map code in gxwmap.c, the entry point is function  
gxdmap.  My approach to caching the map files would be to replace the  
calls to fopen, fclose, fread, and fseek that appear in gxdmap with  
calls to some new routines that would be put in gxwmap.c.  The new  
open routine would read the file in its entirety and put it in some  
dynamically-allocated memory buffer -- if it is smaller than some  
threshold, say.  Then the read and seek routines would just pull data  
out of the memory buffer, or do the I/O if the memory buffer wasn't  
allocated.  The open routine would also need to detect if the file  
that is being opened is already in memory.  So the memory buffer  
should be a link list of structures where each structure contains the  
file name and the pointer to the data cache.  It is probably  
sufficient to just test on the full path name; it seems unlikely  
people would point to different locations of the same file within one  
grads session.  There probably needs to also be a routine to free up  
all the memory and reset; perhaps on "reinit" this should be done?

For the fonts -- don't these get read in once?  Well it's been 20  
years so I'll take a look... in gxchpl.c the string plotting all goes  
through the gxchpl entry point.  That calls an internal routine  
gxchgc, which checks some pointer and reads in the font data for that  
font if the pointer is NULL.  Looks like the font data is read once,  
the first time the font is used.   Your stats are showing a ton of  
reads to these files?  That is odd and may be worth looking into  
further... Brian


On Oct 12, 2011, at 3:16 PM, Love, Mr. Gary, Contractor, Code 7542  
wrote:

> Hi Brian, Jennifer,
>
> We use GrADS extensively in a production mode and have found when  
> profiling GrADS runs that 2/3 rds of the data reads are the  
> coastlines and font files.  This may not sound like much, but at  
> FNMOC when running 100's of charts for each of 32 multi-grid  
> projects, the coastline and font reads amount to terabytes for each  
> 12 hour set of runs.
>
> The re-reads occur for each and every display of a variable from the  
> same file after it is opened.  We are going to try to store these  
> files in memory after the first read.
>
> My first question is whether anyone else has looked at this and has  
> a solution?  My next questions, do you have any ideas or guidance to  
> help us do this?
>
> I know that GrADS was designed to operate well on small platforms  
> with little memory, so our goal is to implement a memory caching  
> capability that would be controlled by a 'set' command and would be  
> moderated by the amount of memory available.
>
> Thanks,
> Gary

[Opengrads-devel] Dat File re-reads

[Love, M. G. C. C. 7. <gar...@nr...>]

Hi Brian, Jennifer,

We use GrADS extensively in a production mode and have found when profiling GrADS runs that 2/3 rds of the data reads are the coastlines and font files.  This may not sound like much, but at FNMOC when running 100's of charts for each of 32 multi-grid projects, the coastline and font reads amount to terabytes for each 12 hour set of runs.

The re-reads occur for each and every display of a variable from the same file after it is opened.  We are going to try to store these files in memory after the first read.  

My first question is whether anyone else has looked at this and has a solution?  My next questions, do you have any ideas or guidance to help us do this?

I know that GrADS was designed to operate well on small platforms with little memory, so our goal is to implement a memory caching capability that would be controlled by a 'set' command and would be moderated by the amount of memory available.

Thanks,
Gary

Re: [Opengrads-devel] Gxshad error

[Brian D. <do...@co...>]

I should also mention "set gxout shade2b", available in v2.0.0.  This  
does the shading grid-box by grid-box, so it gives a large number of  
small polygons, which is usually very slow to render.  But the  
algorithm is very simple (relatively) and it is the version most  
likely to be correct, should some question arise about the other  
versions of shaded contouring.  I wrote this code as my first attempt  
of a rewrite of the shaded contouring algorithm, but what I had  
intended didn't work out.  But I left the code there since it is  
useful for testing and it may have uses in the future since the  
polygons are "pre-clipped"... Brian

On Oct 6, 2011, at 7:36 PM, Jennifer Adams wrote:

> Hi, Gary --
>
> Here is the relevant code in gxshad.c:
>
> #define XYBMAX 5000
> ...
>     if (stkcnt>=XYBMAX) {
>       printf ("Buffer stack limit exceeded in gxshad\n");
>       return(1);
>     }
>
> Looks like you've bumped up against a hard-coded limit. You can  
> change XYBMAX and recompile, or try 'set gxout shade2' and see if  
> that changes anything.
> --Jennifer
>
>
> On Oct 6, 2011, at 7:07 PM, Love, Mr. Gary, Contractor, Code 7542  
> wrote:
>
>> Hi Jennifer,
>>
>> I just encountered any error I have never seen
>>
>> Buffer stack limit exceeded in gxshad
>> Error in gxshad
>>
>> I'm only shading about a dozen colors, but I get this when the plot  
>> is very dense.
>>
>> Are there any quick solutions? Reduce colors, etc?
>>
>> Gary
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jm...@co...
>
>
>

Re: [Opengrads-devel] Cygwin's gcc version

[Arlindo da S. <da...@al...>]

On Tue, Oct 11, 2011 at 6:04 PM, Jennifer Adams <jm...@co...> wrote:

>
> On Oct 11, 2011, at 5:49 PM, Arlindo da Silva wrote:
>
> On Tue, Oct 11, 2011 at 5:37 PM, Huddleston, John <
> Hud...@ci...> wrote:
>
>>  Hi Arlindo
>>
>> Sorry for the delayed responses this week. I'm taking leave to take care
>> of issues around the place before winter sets in. Today I put up a retaining
>> wall with 21 10' 4x4 timber. We live on 35 acres and the four horses are
>> a continual challenge.
>>
>
>
> Horses? We have a cat and he is already trouble enough...
>
>
>> Tomorrow I have to cut down several trees killed by the pine beetle. We
>> have 35 acres and live up at 6850' elevation in the foothills of the rockies.
>> It's about 40 miles from Fort Collins or 40 miles from Laramie.
>>
>>
>
> It sounds very cool, the kind of life style my wife would not mind...
>
>
>>  I received an email from Barbara Jones of the hdfgroup. She is going to
>> ask her developers the best way to find the bug.
>>
>>
>
> I am pretty sure running hrepack on a sample file would show the
> performance issue. No need to bring grads into the picture.
>
> Barbara (the brain behind the help desk at HDF) is extremely capable. But I
> agree it would simplify things if GrADS could be factored out.
>
>
>
>> Yes, it is HDF4.2r3.  The hdfgroup has gone to a new naming convention so
>> I did not want to confuse you. The GrADS version supported by
>> the SUPPLIBS works. All newer hdfgroup HDF4 versions have a
>> serious performance issue.  Do not use HDF4.2r5 (aka hdf-4.2.5) it has
>> serious performance issues as does HDF4.2r6 (aka hdf-4.2.6).
>>
>> Both current HDF5 and NetCDF4 libraries are fine.
>>
>>
>
> Is this windows specific or on all platforms? I know for a fact that HDF
> 4.2.3 does not build on mac os x 10.6. Perhaps I should hack it until it
> builds.
>
> I have never even tried a more recent version than 4.2r3, but I'd be
> willing to do some testing on our CentOS boxes if that would be useful.
>
>
> Jennifer: Have you attempted building HDF 4.2.3 on Snow Leopard?
>
> I never had a Snow Leopard laptop, I am leapfrogging from 10.5 to 10.7, but
> Brian does, and I'm pretty sure I got everything to build for his
> environment. The trick I needed was to set CFLAGS to "-arch i386".
>

But that forces 32 bit only binaries. By default you get universal 32/64 bit
binaries. It would be nice to 64 bit binaries going forward, don't you
think? This is what we are doing on Linux (although the 32-bit linux builds
work everywhere).


> I will check on it when I'm in the office tomorrow (the OSX 10.6 server has
> to be turned on -- it gets very little use).
>

Lion is very similar to snow leopard, provided you have the same Xcode
version (I have the latest 4.1.1). Here is the error I get compiling 4.2.3
on Snow Leopard (without -arch i386):

Making all in hdf
Making all in src
make  all-am
gcc -DHAVE_CONFIG_H -I.   -I/usr/include/rpc
 -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/
 -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/ncurses
   -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/curl
   -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/xml2
   -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/zlib
   -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/szip

 -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/libpng
   -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/jpeg

 -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/jasper
   -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/gd
 -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/freetype2

 -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/includepixman-1
   -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/hdf5
  -I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/zlib
-I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/jpeg
-I/Users/adasilva/src/supplibs-2.2.1/i686-apple-darwin10.8.0/include/szlib
-DBIG_LONGS -DSWAP   -O2  -MT atom.o -MD -MP -MF .deps/atom.Tpo -c -o atom.o
atom.c
In file included from hdf.h:20,
                 from atom.c:64:
hdfi.h:1352: error: expected '=', ',', ';', 'asm' or '__attribute__' before
'you'
In file included from hdf.h:20,
                 from atom.c:64:
hdfi.h:1355:1: warning: "GOT_MACHINE" redefined
hdfi.h:760:1: warning: this is the location of the previous definition
hdfi.h:1359:1: warning: "DF_MT" redefined
hdfi.h:775:1: warning: this is the location of the previous definition
hdfi.h:1361: error: redefinition of typedef 'VOIDP'
hdfi.h:779: error: previous declaration of 'VOIDP' was here
hdfi.h:1362: error: redefinition of typedef '_fcd'
hdfi.h:780: error: previous declaration of '_fcd' was here
hdfi.h:1363: error: redefinition of typedef 'char8'
hdfi.h:781: error: previous declaration of 'char8' was here
hdfi.h:1364: error: redefinition of typedef 'uchar8'
hdfi.h:782: error: previous declaration of 'uchar8' was here
hdfi.h:1365: error: redefinition of typedef 'int8'
hdfi.h:783: error: previous declaration of 'int8' was here
hdfi.h:1366: error: redefinition of typedef 'uint8'
hdfi.h:784: error: previous declaration of 'uint8' was here
hdfi.h:1367: error: redefinition of typedef 'int16'
hdfi.h:785: error: previous declaration of 'int16' was here
hdfi.h:1368: error: redefinition of typedef 'uint16'
hdfi.h:786: error: previous declaration of 'uint16' was here
hdfi.h:1369: error: redefinition of typedef 'int32'
hdfi.h:787: error: previous declaration of 'int32' was here
hdfi.h:1370: error: redefinition of typedef 'uint32'
hdfi.h:788: error: previous declaration of 'uint32' was here
hdfi.h:1371: error: redefinition of typedef 'intn'
hdfi.h:789: error: previous declaration of 'intn' was here
hdfi.h:1372: error: redefinition of typedef 'uintn'
hdfi.h:790: error: previous declaration of 'uintn' was here
hdfi.h:1373: error: redefinition of typedef 'intf'
hdfi.h:793: error: previous declaration of 'intf' was here
hdfi.h:1374: error: redefinition of typedef 'float32'
hdfi.h:791: error: previous declaration of 'float32' was here
hdfi.h:1375: error: redefinition of typedef 'float64'
hdfi.h:792: error: previous declaration of 'float64' was here
hdfi.h:1376: error: conflicting types for 'hdf_pint_t'
hdfi.h:794: error: previous declaration of 'hdf_pint_t' was here
make[3]: *** [atom.o] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1

This problem is fixed in 4.2.5. Replacing hdfi.h with the newest version
gets it to finish the build, but then it fails the hdf-4 unit tests (make
check).

    Arlindo



-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] 4.1.1 vs. 4.1.2

[Arlindo da S. <da...@al...>]

On Mon, Oct 10, 2011 at 7:47 AM, Huddleston, John <
Hud...@ci...> wrote:

>  Hi Arlindo
>
> The AMSR data was stored in HDF4 format; and, the HDF4r3 version works
> best. I sent a report to the hdfgroup and they have started a help
> ticket.  It took 1.2 seconds to process the AMSR HDF4 file with HDF4r3 and
> it took nearly two minutes using later HDF4 libraries.
>
> The GrADS-2.0.0 was faster than GrADS-2.0a9 and a8 on a Linux Ubuntuserver. It was about 2Xfaster than Windows 7. The Ubuntuserver has single processor 2.8GHzwith 3GBof RAM. The Windows 7 has two 3.6GHzprocessors with 8 GB RAM. You'd expect the Windows version to be faster. I'm
> working on a MinGW native windows build. All the HDF4, HDF5, and NetCDFapplications under MingWrun 10xfaster than equivalent Ubuntuversions (which is what you would expect from the hardware difference.)
>
>

What are your plans for X11? It is possible to have stable X11 builds on
mingwin as demonstrated by the Xming guy. However, I was not able to
convince him to share his libraries with me. Having built X11 on Windows in
the past (very early on I maintained X11 cygwin builds), I don't look
forward to the task... Time would be better spent replacing gxX.c with a
native gxCairo.c. (This is different from what Jennifer is doing for 2.1 as
she is using the X11 surface with cairo.) One should be able to erase any
trace of X from the code, but it is not as easy because X11 is not
restricted to gxX.c. The problem would be more doable if one would drop the
older X11 widgets and the Athena GUI features in GrADS. I'd start adapting
the gxJava.c which part of GrADSVM...


>  e.g. See http://vista.cira.colostate.edu/nco/Grib2/ WGRIB2 executables I
> build for Wesley.
>
>

A couple of years back I also had a mini-supplibs built with mingwin that I
used for wgrib2, there is a binary on sf.net:

http://sourceforge.net/projects/opengrads/files/wgrib2/0.1.7.8e/

The only think I found annoying with the mingwin ports is the handling of
the "\" vs "/" in directory names...

 But I agreee, It would be nice to convert grads to mingwin.

  Cheers!

     Arlindo




>  John
>
>  ------------------------------
> *From:* mor...@gm... [mor...@gm...] On Behalf Of
> Arlindo da Silva [arl...@na...]
> *Sent:* Sunday, October 09, 2011 4:41 PM
> *To:* Huddleston, John
> *Cc:* ope...@li...
>
> *Subject:* Re: 4.1.1 vs. 4.1.2
>
>   On Fri, Oct 7, 2011 at 12:27 PM, Huddleston, John <
> Hud...@ci...> wrote:
>
>>  Jennifer****
>>
>> ****
>>
>> While creating the AMSR CTL file I did some testing on the Ubuntu Linux
>> build of GrADS and found that version 2.0.0 was faster than both 2.0a8 and
>> 2.0a9 by some 20-30%.****
>>
>> ****
>>
>> So, I did the same run on Windows and it was horribly slow. I re-compiled
>> GrADS with the GNU profiler and found that the HDF5 HAPatom_object was the
>> culprit.****
>>
>> ****
>>
>>   %   cumulative   self              self     total****
>>
>> time   seconds   seconds    calls  ms/call  ms/call  name****
>>
>> 99.21     85.51    85.51                             HAPatom_object****
>>
>>   0.15     85.64     0.13   475492     0.00     0.00  gahrow****
>>
>>   0.07     85.70     0.06   475544     0.00     0.00  gree****
>>
>>   0.06     85.75     0.05                             HTPinquire****
>>
>>   0.05     85.79     0.04    76373     0.00     0.00  gxfill****
>>
>>   0.03     85.82     0.03   475559     0.00     0.00  galloc****
>>
>>   0.03     85.85     0.03                             DFKconvert****
>>
>> ****
>>
>> I checked and I was using HDF5-1.8.6; so, I’m building HDF5-1.8.7 for
>> Cygwin and will rebuild GrADS 2.0.0.****
>>
>> **
>>
>
>
> Please let me know that you find. I have tried my windows build of 2.0.a9
> on this large netcdf-4 file (with compression)
>
>
> ftp://gma...@ft.../fp/das/Y2011/M10/D01/DAS.ops.asm.inst3_3d_asm_Np.GEOS572.20111001_0000.V01.nc4
>
> and found that
>
> lats4d -i DAS.ops.asm.inst3_3d_asm_Np.GEOS572.20111001_0000.V01.nc4 -format
> stats
>
> ran slightly faster on Windows 7 than Mac OS X 10.6 (48.6s vs 51.6s) on the
> same hardware (different boots).  This was based on HDF5 1.8.4 and NetCDF
> 4.1.1 that I used for the 2.0.a9 builds.
>
>   I will now build 2.0.0 with new libraries and rerun the benchmark.
>
>       Arlindo
>
>
>
>
>
>>  **
>>
>> John****
>>
>> ****
>>
>> ****
>>
>> ****
>>
>> *From:* Jennifer Adams [mailto:jm...@co...]
>> *Sent:* Wednesday, October 05, 2011 3:39 PM
>> *To:* Arlindo da Silva
>> *Cc:* Brian Doty; Huddleston, John
>> *Subject:* Re: 4.1.1 vs. 4.1.2****
>>
>> ****
>>
>> I think netcdf-4.1.2 (with zlib-1.2.5 and hdf5-1.8.7) is what we should
>> use for GrADS 2.0.0. We've been using 4.1.2 at COLA for just about a year. I
>> have made new builds for darwin and CentOS and posted them to our FTP
>> server, and I changed my supplibs.html page too. Arlindo, thanks for
>> bringing this bug to my attention. John, I guess you have no extra work
>> since your builds are already linked with 4.1.2. ****
>>
>>  --Jennifer****
>>
>> ****
>>
>> p.s. Here is what Dennis Heimbigner wrote about the performance bug: ****
>>
>> "The performance bug has been in the code since almost****
>>
>> the beginning of opendap support.  It is possible
>> (even probable) that you will not encounter it.
>> It occurs when you read a large variable and specify
>> it as a constraint in the url. E.g. http://x.com/y.nc?var[0:1]
>> The bug is that the code ignores the [0:1] and reads
>> the whole variable.  The result is correct, but unnecessary
>> data is read." ****
>>
>> ****
>>
>> I don't exactly know whether that syntax he shows is different from the
>> requests in the gds log that look like this:****
>>
>> "...GET /gfs.dods?ps.ps[0][0:360][0:719] ..."****
>>
>> Maybe they ARE different and that's why we don't encounter a performance
>> change between old versions and the new snapshot. In any event, I think
>> 4.1.2 is the way to go for now. ****
>>
>> ****
>>
>> ****
>>
>> On Oct 5, 2011, at 1:20 PM, Arlindo da Silva wrote:****
>>
>>
>>
>> ****
>>
>> ****
>>
>> On Wed, Oct 5, 2011 at 11:51 AM, Jennifer Adams <jm...@co...>
>> wrote:****
>>
>> Hi, Arlindo -- ****
>>
>> The release notes for 4.1.2 (
>> http://www.unidata.ucar.edu/software/netcdf/release-notes-4.1.2.html )
>> mention a lot of items that are relevant for GrADS, one especially that was
>> pointed out by me (speedup in opening of large files). I will definitely not
>> be going to back to 4.1.1. We've been using 4.1.2 at COLA for a long time.
>> ****
>>
>> ****
>>
>> ****
>>
>> This is good to know. If you have been using 4.1.2 for sometime than this
>> is proof of stability. I'll follow your lead.****
>>
>> ****
>>
>>   Here are the 7 tests I ran with the two builds -- I could not detect
>> any difference in the time it took for these tasks to run when comparing
>> 4.1.2 and 4.2.snapshot.****
>>
>>  ****
>>
>> I share your concern about 4.2. First, it is very new, and second, a large
>> portion of the opendap codebase was apparently rewritten. Let me know when
>> you make the final decision about 4.1.2 and I'll update my supplibs
>> accordingly.****
>>
>> ****
>>
>>   Cheers!****
>>
>> ****
>>
>>      Arlindo****
>>
>> ****
>>
>> ****
>>
>> ****
>>
>> ****
>>
>> ****
>>
>>   ****
>>
>> --Jennifer****
>>
>> ****
>>
>> 'reinit'****
>>
>> '!date'****
>>
>> ****
>>
>> if (0)****
>>
>> * The original bug****
>>
>>  'sdfopen
>> http://opendap.gsfc.nasa.gov:9090/dods/GEOS-5/fp/0.25_deg/assim/inst1_2d_hwl_Nx'
>> ****
>>
>>  'set x 1 1152'****
>>
>>  'd slp'****
>>
>>  say result****
>>
>> endif****
>>
>> ****
>>
>> if (0)****
>>
>> * NOMADS GDS****
>>
>>  'sdfopen http://nomads.ncep.noaa.gov:9090/dods/gfs/gfs20111003/gfs_00z'*
>> ***
>>
>>  'set x 1 360'****
>>
>>  'set y 1 181'****
>>
>>  'set z 1 26'****
>>
>>  'set t 1 65'****
>>
>>  'define foo = hgtprs'****
>>
>> endif****
>>
>> ****
>>
>> if (0)****
>>
>> * COLA GDS, Ensembles****
>>
>>  'sdfopen http://monsoondata.org:9090/dods/gfsens/gfsens.2011100400'****
>>
>>  'set x 1 360'****
>>
>>  'set y 1 181'****
>>
>>  'set z 1 7'****
>>
>>  'set e 1 22'****
>>
>>  'define foo = z'****
>>
>> endif****
>>
>> ****
>>
>> if (0)****
>>
>> * Netcdf4 high res data behind GDS****
>>
>>  'sdfopen http://monsoondata.org:9090/dods/nicam/sst'****
>>
>>  'set x 1 5120'****
>>
>>  'set y 1 2556'****
>>
>>  'd ave(sst,t=1,t=8)'****
>>
>> endif****
>>
>> ****
>>
>> if (0)****
>>
>> * Netcdf4 high res data on local disk****
>>
>>  'open /data/hdf5/grids/sst.ctl'****
>>
>>  'set x 1 5120'****
>>
>>  'set y 1 2556'****
>>
>>  'd ave(sst,t=1,t=8)'****
>>
>> endif****
>>
>> ****
>>
>> if (0)****
>>
>> * Local file, PDEF'd****
>>
>>  'open /data/netcdf/gswp/Albedo_mean_csu.ctl'****
>>
>>  'set x 1 360'****
>>
>>  'set y 1 150'****
>>
>>  'set z 1'****
>>
>>  'set t 1 12'****
>>
>>  'define foo = albedo'****
>>
>> endif****
>>
>> ****
>>
>> if (1)****
>>
>> * Classic netcdf****
>>
>>  'sdfopen /data/netcdf/rean/air.mon.mean.nc'****
>>
>>  'set x 1 144'****
>>
>>  'set y 1 73'****
>>
>>  'set z 1 17'****
>>
>>  'set t 1 349'****
>>
>>  'define foo = air'****
>>
>> endif****
>>
>> ****
>>
>> '!date'****
>>
>> ****
>>
>> ****
>>
>> On Oct 5, 2011, at 10:26 AM, Arlindo da Silva wrote:****
>>
>>
>>
>> ****
>>
>> On Tue, Oct 4, 2011 at 5:06 PM, Jennifer Adams <jm...@co...> wrote:
>> ****
>>
>>  Hi, Dennis -- ****
>>
>> Using the same GrADS source code, I have created two builds, one linked
>> with netcdf-4.1.2 and the other netcdf-4.2-snapshot2011100320. Both use
>> hdf5-1.8.7. The bug that showed up with 4.1.3 does not reproduce with either
>> build. Furthermore, I am unable to detect any performance difference with
>> these two builds, having tested some bulky I/O requests via OPeNDAP from
>> several different GDS servers, and also from local classic netcdf files and
>> very large compressed netcdf4 files. I am inclined to use 4.1.2 for my GrADS
>> release. Can you give me an example for how to test the performance
>> enhancements in the 4.2 snapshot? ****
>>
>>  ****
>>
>> I have a similar experience. Comparing builds with older 4.1.1 and most
>> recent 4.2 snapshot (with subsetting) gives essentially the same timing (if
>> anything, 4.1.1 is a tiny bit faster: 49.69 vs 45.63, but this difference
>> could be "natural variability").  ****
>>
>> ****
>>
>> Jennifer:  BTW, here is my grads benchmark:****
>>
>> ****
>>
>>      lats4d -i
>> http://opendap.gsfc.nasa.gov:9090/dods/GEOS-5/fp/0.25_deg/assim/inst1_2d_hwl_Nx-format stats -lat 0 30 -ntimes 10
>> ****
>>
>> ****
>>
>> Since 4.1.1 has been working for us very reliably for over a year, I am
>> very tempted to stick with it in name of stability. Are there any major bug
>> fixes going from 4.1.1 to 4.1.2?****
>>
>> ****
>>
>>    Arlindo****
>>
>> ****
>>
>>   ****
>>
>> ****
>>
>> ****
>>
>> ****
>>
>> ****
>>
>> --****
>>
>> Jennifer M. Adams****
>>
>> IGES/COLA****
>>
>> 4041 Powder Mill Road, Suite 302****
>>
>> Calverton, MD 20705****
>>
>> jm...@co...****
>>
>> ****
>>
>>
>>
>> ****
>>
>> ****
>>
>> ****
>>
>> ****
>>
>> --****
>>
>> Jennifer M. Adams****
>>
>> IGES/COLA****
>>
>> 4041 Powder Mill Road, Suite 302****
>>
>> Calverton, MD 20705****
>>
>> jm...@co...****
>>
>> ****
>>
>>
>>
>> ****
>>
>> ****
>>
>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2d-oct
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>
>


-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] OpenGrADS Bundle 2.0.0 almost ready

[Arlindo da S. <da...@al...>]

On Mon, Oct 10, 2011 at 12:21 PM, Michael Fiorino
<mic...@no...>wrote:

> hi Arlindo, i tried building on my snow leopard machine (did the gacvs
> co -P supplibs and Grads);
>
> sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 233 > uname -a
> Darwin Tenki.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36
> PDT 2011;
> root:xnu-1504.15.3~1/RELEASE_I386 i386
>
> supplibs built ok,


Did you move the supplibs after you built it? (If you move it you must run
relocate.sh). Next, how did you setup the symlink? Here is how I do it:

% gacvs -P Grads
% ln -s /path/to/supplibs-2.2.1/i686.... supplibs

So that GrADS/ and supplibs/ are at the same level; also notice that the
symlink source go down to the level of i686...whatever...

   Arlindo


   Arlindo







> but the build of grads failed again, in the usual
> place? because i'm at 10.8?  i just did the usual updates...
>
> details below; best /r mike
>
> sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 233 > uname -a
> Darwin Tenki.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36
> PDT 2011;
> root:xnu-1504.15.3~1/RELEASE_I386 i386
>
> -------+---------+--------------
> Config | Install |   Package
> -------+---------+--------------
>  ok   |   ok    | pkg-config
>  ok   |   ok    | udunits
>  ok   |   ok    | zlib
>  ok   |   ok    | szlib
>  ok   |   ok    | libpng
>  ok   |   ok    | jpeg
>  ok   |   ok    | jasper
>  ok   |   ok    | tiff
>  ok   |   ok    | geotiff
>  ok   |   ok    | hdf4
>  ok   |   ok    | grib2c
>  ok   |   ok    | xml2
>  ok   |   ok    | curl
>  ok   |   ok    | dap
>  ok   |   ok    | gadap
>  ok   |   ok    | hdf5
>  ok   |   ok    | netcdf
>  ok   |   ok    | ncurses
>  ok   |   ok    | readline
>  ok   |   ok    | libsx
>  ok   |   ok    | freetype
>  ok   |   ok    | fontconfig
>  ok   |   ok    | pixman
>  ok   |   ok    | cairo
>  ok   |   ok    | shp
>  ok   |   ok    | gd
> -------+---------+--------------
>
>
>
>  +=========================================================================+
>   GrADS 2.0.0.oga.1 : Built Mon Oct 10 16:12:06 GMT 2011 for
> i386-apple-darwin10.8.0
>
>  +=========================================================================+
>
>  +-------------------------------+
>  |                               |
>  |  GrADS will be built with:    |
>  |                               |
>  |  + GUI enabled                |
>  |  + readline enabled           |
>  |  + geotiff enabled            |
>  |  + shapefile enabled          |
>  |  + printim enabled            |
>  |  + grib2 enabled              |
>  |  + hdf4 enabled               |
>  |  + hdf5 enabled               |
>  |  + netcdf-4 enabled           |
>  |  + OPeNDAP enabled            |
>  |     for grids and stn data    |
>  |                               |
>  +-------------------------------+
> configure: creating src/VERSION
> configure: creating src/buildinfo.h
> configure: creating ./config.status
> config.status: creating Makefile
> config.status: creating src/Makefile
> config.status: creating pytests/Makefile
> config.status: creating src/config.h
>
> Run 'make && make install' to build GrADS.
> Executables will be installed to bin/ in current dir, unless prefix was
> specified.
>
> sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 230 > pg Bu
> Bu: No such file or directory
> sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 231 > pg BUILD
>                    Building GrADS from Sources
>                    ===========================
>
> VERY IMPORTANT: You need the supplibs v2.1.0 for building
>                GrADS v2.0.a5.oga.1 and later.
>
> Quick Instructions
> ------------------
>
> Here is the quickest way to build GrADS from sources:
>
>    * Choose a working directory and make sure you have GNU make.
>    * Download the Supplemental Libraries (Supplibs)
>      preferably pre-compiled, and untar them in this working directory
>    * If you download the /supplibs/ sources instead, untar them in the
>      same location and build:
>
>  % ln -s supplibs-2.1.0 supplibs <==== Notice version!!!
>  % cd supplibs/src
>  % gmake install     (do not configure or issue a simple make)
>
>    * Download the grads sources, untar them in this same working
>      directory, configure and make
>
>  % cd grads-2.0.x
>  % ./configure
>  % gmake
>  % gmake check
>  % gmake bundle-dist   (to create a bundle tarball)
>
> See file INSTALL for installing this bundle.
>
> Read on if you need more information.
>
> Introduction
> ------------
>
> There are three different methods for having GrADS installed on your
> computer:
>
>   1. If it is included in your Linux distribution, for example, recent
>      versions of Red Hat and Fedora distributions, install it directly
>      from your CD or from the net with update tools such as yum, apt or
>      urpmi.
>   2. Install one of the binaries distributions from the GrADS download
>      page <http://grads.iges.org/grads/download> at COLA. This usually
>      works for a large number of platforms and gives you binaries
> sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 232 > gmake
> Making all in src
> gmake[1]: Entering directory
> `/Volumes/dat1/w21/app/src/opengrads2.0/Grads/src'
> gmake  all-am
> gmake[2]: Entering directory
> `/Volumes/dat1/w21/app/src/opengrads2.0/Grads/src'
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c main.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c grads.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxsubs.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxmeta.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxchpl.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxcntr.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxstrm.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxwmap.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxshad.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxshad2.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gaexpr.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gafunc.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gautil.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gagx.c
> gagx.c: In function ÔgashpwrtÕ:
> gagx.c:3594: warning: format not a string literal and no format arguments
> gagx.c: In function ÔgascatÕ:
> gagx.c:3330: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagx.c: In function ÔgashpwrtÕ:
> gagx.c:3906: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagx.c:3931: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagx.c: In function ÔgakmlÕ:
> gagx.c:4143: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagx.c:4145: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagx.c:4189: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagx.c:4191: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gscrpt.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gamach.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c bufrstn.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gabufr.c
> gabufr.c: In function Ôgabufr_load2memÕ:
> gabufr.c:96: warning: format Ô%ldÕ expects type Ôlong intÕ, but argument 2
> has type Ôoff_tÕ
> gabufr.c:106: warning: format Ô%ldÕ expects type Ôlong intÕ, but argument 2
> has type Ôoff_tÕ
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gabufrtbl.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxX.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gxdxwd.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c galloc.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gagui.c
> gagui.c: In function ÔCB_OpenÕ:
> gagui.c:347: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 2 has
> type Ôvoid *Õ
> gagui.c: In function ÔCB_LoadÕ:
> gagui.c:402: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 6 has
> type Ôvoid *Õ
> gagui.c:418: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 2 has
> type Ôvoid *Õ
> gagui.c: In function ÔCB_CmdStrOldÕ:
> gagui.c:955: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 6 has
> type Ôvoid *Õ
> gagui.c: In function ÔCB_TextCancelÕ:
> gagui.c:985: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 2 has
> type Ôvoid *Õ
> gagui.c: In function ÔCB_DisplayÕ:
> gagui.c:801: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagui.c:844: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagui.c:849: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> gagui.c:855: warning: call to __builtin___snprintf_chk will always overflow
> destination buffer
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gsgui.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c dodstn.c
> dodstn.c: In function ÔdappfiÕ:
> dodstn.c:350: warning: passing argument 2 of ÔstrtodÕ from incompatible
> pointer type
> dodstn.c: In function ÔdapgetÕ:
> dodstn.c:525: warning: passing argument 1 of ÔfreeÕ discards qualifiers
> from pointer target type
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gaddes.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gaio.c
> gaio.c: In function ÔgairowÕ:
> gaio.c:1571: warning: format Ô%jdÕ expects type Ôintmax_tÕ, but argument 6
> has type Ôoff_tÕ
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gacfg.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gauser.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gasdf.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gatxt.c
> cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
> -I.././supplibs/include/grib2c -I.././supplibs/include/libsx
> -I.././supplibs/include/udunits
> -I.././supplibs/include/netcdf -I.././supplibs/include/hdf
> -I.././supplibs/include/udunits
> -I.././supplibs/include/hdf5 -I.././supplibs/include/tiff
> -I.././supplibs/include/geotiff
> -I.././supplibs/include/shapelib -I.././supplibs/include/shp
> -I.././supplibs/include/gadap
> -I/opt/local/include       -g -O2 -c gaudx.c
> g++  -g -O2  -lSystemStubs -o grads main.o grads.o gxsubs.o gxmeta.o
> gxchpl.o gxcntr.o
> gxstrm.o gxwmap.o gxshad.o gxshad2.o gaexpr.o gafunc.o gautil.o gagx.o
> gscrpt.o gamach.o
> bufrstn.o gabufr.o gabufrtbl.o gxX.o gxdxwd.o galloc.o gagui.o gsgui.o
> dodstn.o gaddes.o
> gaio.o gacfg.o gauser.o gasdf.o gatxt.o gaudx.o -L.././supplibs/lib
>  -L/opt/local/lib
> -R/opt/local/lib  -lX11 .././supplibs/lib/libreadline.a
> .././supplibs/lib/libncurses.a
> .././supplibs/lib/libgd.a .././supplibs/lib/libpng12.a
> .././supplibs/lib/libz.a
> .././supplibs/lib/libjpeg.a .././supplibs/lib/libgrib2c.a
> .././supplibs/lib/libjasper.a
> .././supplibs/lib/libpng12.a .././supplibs/lib/libz.a
> .././supplibs/lib/libmfhdf.a
> .././supplibs/lib/libdf.a .././supplibs/lib/libudunits.a
> .././supplibs/lib/libsz.a
> .././supplibs/lib/libjpeg.a .././supplibs/lib/libz.a
> .././supplibs/lib/libhdf5.a
> .././supplibs/lib/libsz.a .././supplibs/lib/libjpeg.a
> .././supplibs/lib/libz.a
> .././supplibs/lib/libudunits.a .././supplibs/lib/libnetcdf.a
> .././supplibs/lib/libhdf5_hl.a
> .././supplibs/lib/libhdf5.a .././supplibs/lib/libz.a
> .././supplibs/lib/libsz.a
> .././supplibs/lib/libcurl.a -lsx -lXaw3d -lXmu -lXt -lXext -lXpm
> .././supplibs/lib/libtiff.a
> .././supplibs/lib/libgeotiff.a .././supplibs/lib/libshp.a
> .././supplibs/lib/libgadap.a
> .././supplibs/lib/libdapclient.a .././supplibs/lib/libdap.a
> .././supplibs/lib/libcurl.a
> .././supplibs/lib/libxml2.a .././supplibs/lib/libz.a -lpthread -lm -liconv
> -lm
> Undefined symbols:
>  "_iconv_close", referenced from:
>      _xmlFindCharEncodingHandler in libxml2.a(encoding.o)
>      _xmlFindCharEncodingHandler in libxml2.a(encoding.o)
>      _xmlCharEncCloseFunc in libxml2.a(encoding.o)
>      _xmlCharEncCloseFunc in libxml2.a(encoding.o)
>  "_iconv", referenced from:
>      _xmlIconvWrapper in libxml2.a(encoding.o)
>  "_iconv_open", referenced from:
>      _xmlFindCharEncodingHandler in libxml2.a(encoding.o)
>      _xmlFindCharEncodingHandler in libxml2.a(encoding.o)
> ld: symbol(s) not found
> collect2: ld returned 1 exit status
> gmake[2]: *** [grads] Error 1
> gmake[2]: Leaving directory
> `/Volumes/dat1/w21/app/src/opengrads2.0/Grads/src'
> gmake[1]: *** [all] Error 2
> gmake[1]: Leaving directory
> `/Volumes/dat1/w21/app/src/opengrads2.0/Grads/src'
> gmake: *** [all-recursive] Error 1
> sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 233 >
>
>
>
> On 2011-10-09 15:38, Arlindo da Silva wrote:
> > All,
> >
> > I have uploaded sources and binaries (Linux and Mac OS X for now) to
> sf.net
> >
> > https://sourceforge.net/projects/opengrads/files/grads2/2.0.0.oga.1/
> >
> >
> > Of course, this requires the new supplibs-2.2.1:
> >
> > https://sourceforge.net/projects/opengrads/files/supplibs/2.2.1/
> >
> > Notice that for the first time I am including 2 set of binaries for the
> Mac: one for
> Leopard (macosx_10.5, 32-bit only) and another for Snow Leopard/Lion
> (macosx_10.6, both 32
> and 64-bits). The Snow Leopard binaries are universal, although the file is
> branded "x86_64"
> they include both 32 and 64 bits. The 10.5 binaries work for Mac OS X 10.5
> thru 10.7; the
> 10.6 binaries work for Mac OS X 10.6-10.7.
> >
> > Another addition that I'll not call attention for in the user list is a
> new binary called
> "sgrads" for "Secure GrADS". This is a special build that disables X11,
> system() and other
> such features to make easier the deployment of grads web servers (as
> Daniel's
> http://webgrads.me).
> >
> > Before I announce these binaries to gradsusr, I'd appreciate if some of
> you could try out
> these binaries on your specific platforms and send me feedback.
> >
> > I am off to prepare the windows superpack. I dread the prospect of having
> to boot into
> windows...
> >
> > Cheers!
> >
> > Arlindo
> >
> > --
> > Arlindo da Silva
> > da...@al...
> >
> >
> >
> ------------------------------------------------------------------------------
> > All of the data generated in your IT infrastructure is seriously
> valuable.
> > Why? It contains a definitive record of application performance, security
> > threats, fraudulent activity, and more. Splunk takes this data and makes
> > sense of it. IT sense. And common sense.
> > http://p.sf.net/sfu/splunk-d2dcopy2
> >
> >
> > _______________________________________________
> > Opengrads-devel mailing list
> > Ope...@li...
> > https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2dcopy1
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>



-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] OpenGrADS Bundle 2.0.0 almost ready

[Michael F. <mic...@no...>]

hi Arlindo, i tried building on my snow leopard machine (did the gacvs
co -P supplibs and Grads);

sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 233 > uname -a
Darwin Tenki.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011;
root:xnu-1504.15.3~1/RELEASE_I386 i386

supplibs built ok, but the build of grads failed again, in the usual
place? because i'm at 10.8?  i just did the usual updates...

details below; best /r mike

sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 233 > uname -a
Darwin Tenki.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011;
root:xnu-1504.15.3~1/RELEASE_I386 i386

-------+---------+--------------
Config | Install |   Package
-------+---------+--------------
  ok   |   ok    | pkg-config
  ok   |   ok    | udunits
  ok   |   ok    | zlib
  ok   |   ok    | szlib
  ok   |   ok    | libpng
  ok   |   ok    | jpeg
  ok   |   ok    | jasper
  ok   |   ok    | tiff
  ok   |   ok    | geotiff
  ok   |   ok    | hdf4
  ok   |   ok    | grib2c
  ok   |   ok    | xml2
  ok   |   ok    | curl
  ok   |   ok    | dap
  ok   |   ok    | gadap
  ok   |   ok    | hdf5
  ok   |   ok    | netcdf
  ok   |   ok    | ncurses
  ok   |   ok    | readline
  ok   |   ok    | libsx
  ok   |   ok    | freetype
  ok   |   ok    | fontconfig
  ok   |   ok    | pixman
  ok   |   ok    | cairo
  ok   |   ok    | shp
  ok   |   ok    | gd
-------+---------+--------------


  +=========================================================================+
   GrADS 2.0.0.oga.1 : Built Mon Oct 10 16:12:06 GMT 2011 for i386-apple-darwin10.8.0
  +=========================================================================+

  +-------------------------------+
  |                               |
  |  GrADS will be built with:    |
  |                               |
  |  + GUI enabled                |
  |  + readline enabled           |
  |  + geotiff enabled            |
  |  + shapefile enabled          |
  |  + printim enabled            |
  |  + grib2 enabled              |
  |  + hdf4 enabled               |
  |  + hdf5 enabled               |
  |  + netcdf-4 enabled           |
  |  + OPeNDAP enabled            |
  |     for grids and stn data    |
  |                               |
  +-------------------------------+
configure: creating src/VERSION
configure: creating src/buildinfo.h
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating pytests/Makefile
config.status: creating src/config.h

Run 'make && make install' to build GrADS.
Executables will be installed to bin/ in current dir, unless prefix was specified.

sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 230 > pg Bu
Bu: No such file or directory
sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 231 > pg BUILD
                    Building GrADS from Sources
                    ===========================

VERY IMPORTANT: You need the supplibs v2.1.0 for building
                GrADS v2.0.a5.oga.1 and later.

Quick Instructions
------------------

Here is the quickest way to build GrADS from sources:

    * Choose a working directory and make sure you have GNU make.
    * Download the Supplemental Libraries (Supplibs)
      preferably pre-compiled, and untar them in this working directory
    * If you download the /supplibs/ sources instead, untar them in the
      same location and build:

 % ln -s supplibs-2.1.0 supplibs <==== Notice version!!!
 % cd supplibs/src
 % gmake install     (do not configure or issue a simple make)

    * Download the grads sources, untar them in this same working
      directory, configure and make

 % cd grads-2.0.x
 % ./configure
 % gmake
 % gmake check
 % gmake bundle-dist   (to create a bundle tarball)

See file INSTALL for installing this bundle.

Read on if you need more information.

Introduction
------------

There are three different methods for having GrADS installed on your
computer:

   1. If it is included in your Linux distribution, for example, recent
      versions of Red Hat and Fedora distributions, install it directly
      from your CD or from the net with update tools such as yum, apt or
      urpmi.
   2. Install one of the binaries distributions from the GrADS download
      page <http://grads.iges.org/grads/download> at COLA. This usually
      works for a large number of platforms and gives you binaries
sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 232 > gmake
Making all in src
gmake[1]: Entering directory `/Volumes/dat1/w21/app/src/opengrads2.0/Grads/src'
gmake  all-am
gmake[2]: Entering directory `/Volumes/dat1/w21/app/src/opengrads2.0/Grads/src'
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c main.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c grads.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxsubs.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxmeta.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxchpl.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxcntr.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxstrm.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxwmap.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxshad.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxshad2.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gaexpr.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gafunc.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gautil.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gagx.c
gagx.c: In function ÔgashpwrtÕ:
gagx.c:3594: warning: format not a string literal and no format arguments
gagx.c: In function ÔgascatÕ:
gagx.c:3330: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagx.c: In function ÔgashpwrtÕ:
gagx.c:3906: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagx.c:3931: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagx.c: In function ÔgakmlÕ:
gagx.c:4143: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagx.c:4145: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagx.c:4189: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagx.c:4191: warning: call to __builtin___snprintf_chk will always overflow destination buffer
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gscrpt.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gamach.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c bufrstn.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gabufr.c
gabufr.c: In function Ôgabufr_load2memÕ:
gabufr.c:96: warning: format Ô%ldÕ expects type Ôlong intÕ, but argument 2 has type Ôoff_tÕ
gabufr.c:106: warning: format Ô%ldÕ expects type Ôlong intÕ, but argument 2 has type Ôoff_tÕ
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gabufrtbl.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxX.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gxdxwd.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c galloc.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gagui.c
gagui.c: In function ÔCB_OpenÕ:
gagui.c:347: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 2 has type Ôvoid *Õ
gagui.c: In function ÔCB_LoadÕ:
gagui.c:402: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 6 has type Ôvoid *Õ
gagui.c:418: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 2 has type Ôvoid *Õ
gagui.c: In function ÔCB_CmdStrOldÕ:
gagui.c:955: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 6 has type Ôvoid *Õ
gagui.c: In function ÔCB_TextCancelÕ:
gagui.c:985: warning: format Ô%sÕ expects type Ôchar *Õ, but argument 2 has type Ôvoid *Õ
gagui.c: In function ÔCB_DisplayÕ:
gagui.c:801: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagui.c:844: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagui.c:849: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gagui.c:855: warning: call to __builtin___snprintf_chk will always overflow destination buffer
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gsgui.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c dodstn.c
dodstn.c: In function ÔdappfiÕ:
dodstn.c:350: warning: passing argument 2 of ÔstrtodÕ from incompatible pointer type
dodstn.c: In function ÔdapgetÕ:
dodstn.c:525: warning: passing argument 1 of ÔfreeÕ discards qualifiers from pointer target type
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gaddes.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gaio.c
gaio.c: In function ÔgairowÕ:
gaio.c:1571: warning: format Ô%jdÕ expects type Ôintmax_tÕ, but argument 6 has type Ôoff_tÕ
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gacfg.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gauser.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gasdf.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gatxt.c
cc -DHAVE_CONFIG_H -I. -I.././supplibs/include/ -I.././supplibs/include/gd
-I.././supplibs/include/grib2c -I.././supplibs/include/libsx -I.././supplibs/include/udunits
-I.././supplibs/include/netcdf -I.././supplibs/include/hdf -I.././supplibs/include/udunits
-I.././supplibs/include/hdf5 -I.././supplibs/include/tiff -I.././supplibs/include/geotiff
-I.././supplibs/include/shapelib -I.././supplibs/include/shp -I.././supplibs/include/gadap
-I/opt/local/include       -g -O2 -c gaudx.c
g++  -g -O2  -lSystemStubs -o grads main.o grads.o gxsubs.o gxmeta.o gxchpl.o gxcntr.o
gxstrm.o gxwmap.o gxshad.o gxshad2.o gaexpr.o gafunc.o gautil.o gagx.o gscrpt.o gamach.o
bufrstn.o gabufr.o gabufrtbl.o gxX.o gxdxwd.o galloc.o gagui.o gsgui.o dodstn.o gaddes.o
gaio.o gacfg.o gauser.o gasdf.o gatxt.o gaudx.o -L.././supplibs/lib  -L/opt/local/lib
-R/opt/local/lib  -lX11 .././supplibs/lib/libreadline.a .././supplibs/lib/libncurses.a 
.././supplibs/lib/libgd.a .././supplibs/lib/libpng12.a .././supplibs/lib/libz.a
.././supplibs/lib/libjpeg.a .././supplibs/lib/libgrib2c.a .././supplibs/lib/libjasper.a
.././supplibs/lib/libpng12.a .././supplibs/lib/libz.a .././supplibs/lib/libmfhdf.a
.././supplibs/lib/libdf.a .././supplibs/lib/libudunits.a .././supplibs/lib/libsz.a
.././supplibs/lib/libjpeg.a .././supplibs/lib/libz.a .././supplibs/lib/libhdf5.a
.././supplibs/lib/libsz.a .././supplibs/lib/libjpeg.a .././supplibs/lib/libz.a
.././supplibs/lib/libudunits.a .././supplibs/lib/libnetcdf.a .././supplibs/lib/libhdf5_hl.a
.././supplibs/lib/libhdf5.a .././supplibs/lib/libz.a .././supplibs/lib/libsz.a
.././supplibs/lib/libcurl.a -lsx -lXaw3d -lXmu -lXt -lXext -lXpm .././supplibs/lib/libtiff.a
.././supplibs/lib/libgeotiff.a .././supplibs/lib/libshp.a .././supplibs/lib/libgadap.a
.././supplibs/lib/libdapclient.a .././supplibs/lib/libdap.a .././supplibs/lib/libcurl.a
.././supplibs/lib/libxml2.a .././supplibs/lib/libz.a -lpthread -lm -liconv -lm
Undefined symbols:
  "_iconv_close", referenced from:
      _xmlFindCharEncodingHandler in libxml2.a(encoding.o)
      _xmlFindCharEncodingHandler in libxml2.a(encoding.o)
      _xmlCharEncCloseFunc in libxml2.a(encoding.o)
      _xmlCharEncCloseFunc in libxml2.a(encoding.o)
  "_iconv", referenced from:
      _xmlIconvWrapper in libxml2.a(encoding.o)
  "_iconv_open", referenced from:
      _xmlFindCharEncodingHandler in libxml2.a(encoding.o)
      _xmlFindCharEncodingHandler in libxml2.a(encoding.o)
ld: symbol(s) not found
collect2: ld returned 1 exit status
gmake[2]: *** [grads] Error 1
gmake[2]: Leaving directory `/Volumes/dat1/w21/app/src/opengrads2.0/Grads/src'
gmake[1]: *** [all] Error 2
gmake[1]: Leaving directory `/Volumes/dat1/w21/app/src/opengrads2.0/Grads/src'
gmake: *** [all-recursive] Error 1
sslvpn-132(DARWIN)[W2:ESRL]: /w21/app/src/opengrads2.0/Grads 233 >



On 2011-10-09 15:38, Arlindo da Silva wrote:
> All,
>
> I have uploaded sources and binaries (Linux and Mac OS X for now) to sf.net
>
> https://sourceforge.net/projects/opengrads/files/grads2/2.0.0.oga.1/
>
>
> Of course, this requires the new supplibs-2.2.1:
>
> https://sourceforge.net/projects/opengrads/files/supplibs/2.2.1/
>
> Notice that for the first time I am including 2 set of binaries for the Mac: one for
Leopard (macosx_10.5, 32-bit only) and another for Snow Leopard/Lion (macosx_10.6, both 32
and 64-bits). The Snow Leopard binaries are universal, although the file is branded "x86_64"
they include both 32 and 64 bits. The 10.5 binaries work for Mac OS X 10.5 thru 10.7; the
10.6 binaries work for Mac OS X 10.6-10.7.
>
> Another addition that I'll not call attention for in the user list is a new binary called
"sgrads" for "Secure GrADS". This is a special build that disables X11, system() and other
such features to make easier the deployment of grads web servers (as Daniel's
http://webgrads.me).
>
> Before I announce these binaries to gradsusr, I'd appreciate if some of you could try out
these binaries on your specific platforms and send me feedback.
>
> I am off to prepare the windows superpack. I dread the prospect of having to boot into
windows...
>
> Cheers!
>
> Arlindo
>
> --
> Arlindo da Silva
> da...@al...
>
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2dcopy2
>
>
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel

Re: [Opengrads-devel] 4.1.1 vs. 4.1.2

[Huddleston, J. <Hud...@ci...>]

Hi Arlindo

The AMSR data was stored in HDF4 format; and, the HDF4r3 version works best. I sent a report to the hdfgroup and they have started a help ticket.  It took 1.2 seconds to process the AMSR HDF4 file with HDF4r3 and it took nearly two minutes using later HDF4 libraries.

The GrADS-2.0.0 was faster than GrADS-2.0a9 and a8 on a Linux Ubuntu server. It was about 2X faster than Windows 7. The Ubuntu server has single processor 2.8GHz with 3GB of RAM. The Windows 7 has two 3.6GHz processors with 8 GB RAM. You'd expect the Windows version to be faster. I'm working on a MinGW native windows build. All the HDF4, HDF5, and NetCDF applications under MingW run 10x faster than equivalent Ubuntu versions (which is what you would expect from the hardware difference.)

e.g. See http://vista.cira.colostate.edu/nco/Grib2/ WGRIB2 executables I build for Wesley.

John

________________________________
From: mor...@gm... [mor...@gm...] On Behalf Of Arlindo da Silva [arl...@na...]
Sent: Sunday, October 09, 2011 4:41 PM
To: Huddleston, John
Cc: ope...@li...
Subject: Re: 4.1.1 vs. 4.1.2

On Fri, Oct 7, 2011 at 12:27 PM, Huddleston, John <Hud...@ci...<mailto:Hud...@ci...>> wrote:
Jennifer

While creating the AMSR CTL file I did some testing on the Ubuntu Linux build of GrADS and found that version 2.0.0 was faster than both 2.0a8 and 2.0a9 by some 20-30%.

So, I did the same run on Windows and it was horribly slow. I re-compiled GrADS with the GNU profiler and found that the HDF5 HAPatom_object was the culprit.

  %   cumulative   self              self     total
time   seconds   seconds    calls  ms/call  ms/call  name
99.21     85.51    85.51                             HAPatom_object
  0.15     85.64     0.13   475492     0.00     0.00  gahrow
  0.07     85.70     0.06   475544     0.00     0.00  gree
  0.06     85.75     0.05                             HTPinquire
  0.05     85.79     0.04    76373     0.00     0.00  gxfill
  0.03     85.82     0.03   475559     0.00     0.00  galloc
  0.03     85.85     0.03                             DFKconvert

I checked and I was using HDF5-1.8.6; so, I’m building HDF5-1.8.7 for Cygwin and will rebuild GrADS 2.0.0.



Please let me know that you find. I have tried my windows build of 2.0.a9 on this large netcdf-4 file (with compression)

ftp://gma...@ft.../fp/das/Y2011/M10/D01/DAS.ops.asm.inst3_3d_asm_Np.GEOS572.20111001_0000.V01.nc4

and found that

lats4d -i DAS.ops.asm.inst3_3d_asm_Np.GEOS572.20111001_0000.V01.nc4 -format stats

ran slightly faster on Windows 7 than Mac OS X 10.6 (48.6s vs 51.6s) on the same hardware (different boots).  This was based on HDF5 1.8.4 and NetCDF 4.1.1 that I used for the 2.0.a9 builds.

  I will now build 2.0.0 with new libraries and rerun the benchmark.

      Arlindo




John



From: Jennifer Adams [mailto:jm...@co...<mailto:jm...@co...>]
Sent: Wednesday, October 05, 2011 3:39 PM
To: Arlindo da Silva
Cc: Brian Doty; Huddleston, John
Subject: Re: 4.1.1 vs. 4.1.2

I think netcdf-4.1.2 (with zlib-1.2.5 and hdf5-1.8.7) is what we should use for GrADS 2.0.0. We've been using 4.1.2 at COLA for just about a year. I have made new builds for darwin and CentOS and posted them to our FTP server, and I changed my supplibs.html page too. Arlindo, thanks for bringing this bug to my attention. John, I guess you have no extra work since your builds are already linked with 4.1.2.
 --Jennifer

p.s. Here is what Dennis Heimbigner wrote about the performance bug:
"The performance bug has been in the code since almost
the beginning of opendap support.  It is possible
(even probable) that you will not encounter it.
It occurs when you read a large variable and specify
it as a constraint in the url. E.g. http://x.com/y.nc?var[0:1]
The bug is that the code ignores the [0:1] and reads
the whole variable.  The result is correct, but unnecessary
data is read."

I don't exactly know whether that syntax he shows is different from the requests in the gds log that look like this:
"...GET /gfs.dods?ps.ps<http://ps.ps>[0][0:360][0:719] ..."
Maybe they ARE different and that's why we don't encounter a performance change between old versions and the new snapshot. In any event, I think 4.1.2 is the way to go for now.


On Oct 5, 2011, at 1:20 PM, Arlindo da Silva wrote:



On Wed, Oct 5, 2011 at 11:51 AM, Jennifer Adams <jm...@co...<mailto:jm...@co...>> wrote:
Hi, Arlindo --
The release notes for 4.1.2 ( http://www.unidata.ucar.edu/software/netcdf/release-notes-4.1.2.html ) mention a lot of items that are relevant for GrADS, one especially that was pointed out by me (speedup in opening of large files). I will definitely not be going to back to 4.1.1. We've been using 4.1.2 at COLA for a long time.


This is good to know. If you have been using 4.1.2 for sometime than this is proof of stability. I'll follow your lead.

Here are the 7 tests I ran with the two builds -- I could not detect any difference in the time it took for these tasks to run when comparing 4.1.2 and 4.2.snapshot.

I share your concern about 4.2. First, it is very new, and second, a large portion of the opendap codebase was apparently rewritten. Let me know when you make the final decision about 4.1.2 and I'll update my supplibs accordingly.

  Cheers!

     Arlindo






--Jennifer

'reinit'
'!date'

if (0)
* The original bug
 'sdfopen http://opendap.gsfc.nasa.gov:9090/dods/GEOS-5/fp/0.25_deg/assim/inst1_2d_hwl_Nx'
 'set x 1 1152'
 'd slp'
 say result
endif

if (0)
* NOMADS GDS
 'sdfopen http://nomads.ncep.noaa.gov:9090/dods/gfs/gfs20111003/gfs_00z'
 'set x 1 360'
 'set y 1 181'
 'set z 1 26'
 'set t 1 65'
 'define foo = hgtprs'
endif

if (0)
* COLA GDS, Ensembles
 'sdfopen http://monsoondata.org:9090/dods/gfsens/gfsens.2011100400'
 'set x 1 360'
 'set y 1 181'
 'set z 1 7'
 'set e 1 22'
 'define foo = z'
endif

if (0)
* Netcdf4 high res data behind GDS
 'sdfopen http://monsoondata.org:9090/dods/nicam/sst'
 'set x 1 5120'
 'set y 1 2556'
 'd ave(sst,t=1,t=8)'
endif

if (0)
* Netcdf4 high res data on local disk
 'open /data/hdf5/grids/sst.ctl'
 'set x 1 5120'
 'set y 1 2556'
 'd ave(sst,t=1,t=8)'
endif

if (0)
* Local file, PDEF'd
 'open /data/netcdf/gswp/Albedo_mean_csu.ctl'
 'set x 1 360'
 'set y 1 150'
 'set z 1'
 'set t 1 12'
 'define foo = albedo'
endif

if (1)
* Classic netcdf
 'sdfopen /data/netcdf/rean/air.mon.mean.nc<http://air.mon.mean.nc>'
 'set x 1 144'
 'set y 1 73'
 'set z 1 17'
 'set t 1 349'
 'define foo = air'
endif

'!date'


On Oct 5, 2011, at 10:26 AM, Arlindo da Silva wrote:


On Tue, Oct 4, 2011 at 5:06 PM, Jennifer Adams <jm...@co...<mailto:jm...@co...>> wrote:
Hi, Dennis --
Using the same GrADS source code, I have created two builds, one linked with netcdf-4.1.2 and the other netcdf-4.2-snapshot2011100320. Both use hdf5-1.8.7. The bug that showed up with 4.1.3 does not reproduce with either build. Furthermore, I am unable to detect any performance difference with these two builds, having tested some bulky I/O requests via OPeNDAP from several different GDS servers, and also from local classic netcdf files and very large compressed netcdf4 files. I am inclined to use 4.1.2 for my GrADS release. Can you give me an example for how to test the performance enhancements in the 4.2 snapshot?

I have a similar experience. Comparing builds with older 4.1.1 and most recent 4.2 snapshot (with subsetting) gives essentially the same timing (if anything, 4.1.1 is a tiny bit faster: 49.69 vs 45.63, but this difference could be "natural variability").

Jennifer:  BTW, here is my grads benchmark:

     lats4d -i http://opendap.gsfc.nasa.gov:9090/dods/GEOS-5/fp/0.25_deg/assim/inst1_2d_hwl_Nx -format stats -lat 0 30 -ntimes 10

Since 4.1.1 has been working for us very reliably for over a year, I am very tempted to stick with it in name of stability. Are there any major bug fixes going from 4.1.1 to 4.1.2?

   Arlindo






--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...<mailto:jm...@co...>






--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...<mailto:jm...@co...>

Re: [Opengrads-devel] 4.1.1 vs. 4.1.2

[Arlindo da S. <arl...@na...>]

On Fri, Oct 7, 2011 at 12:27 PM, Huddleston, John <
Hud...@ci...> wrote:

> Jennifer****
>
> ** **
>
> While creating the AMSR CTL file I did some testing on the Ubuntu Linux
> build of GrADS and found that version 2.0.0 was faster than both 2.0a8 and
> 2.0a9 by some 20-30%.****
>
> ** **
>
> So, I did the same run on Windows and it was horribly slow. I re-compiled
> GrADS with the GNU profiler and found that the HDF5 HAPatom_object was the
> culprit.****
>
> ** **
>
>   %   cumulative   self              self     total****
>
> time   seconds   seconds    calls  ms/call  ms/call  name****
>
> 99.21     85.51    85.51                             HAPatom_object****
>
>   0.15     85.64     0.13   475492     0.00     0.00  gahrow****
>
>   0.07     85.70     0.06   475544     0.00     0.00  gree****
>
>   0.06     85.75     0.05                             HTPinquire****
>
>   0.05     85.79     0.04    76373     0.00     0.00  gxfill****
>
>   0.03     85.82     0.03   475559     0.00     0.00  galloc****
>
>   0.03     85.85     0.03                             DFKconvert****
>
> ** **
>
> I checked and I was using HDF5-1.8.6; so, I’m building HDF5-1.8.7 for
> Cygwin and will rebuild GrADS 2.0.0.****
>
> **
>


Please let me know that you find. I have tried my windows build of 2.0.a9 on
this large netcdf-4 file (with compression)

ftp://gma...@ft.../fp/das/Y2011/M10/D01/DAS.ops.asm.inst3_3d_asm_Np.GEOS572.20111001_0000.V01.nc4

and found that

lats4d -i DAS.ops.asm.inst3_3d_asm_Np.GEOS572.20111001_0000.V01.nc4 -format
stats

ran slightly faster on Windows 7 than Mac OS X 10.6 (48.6s vs 51.6s) on the
same hardware (different boots).  This was based on HDF5 1.8.4 and NetCDF
4.1.1 that I used for the 2.0.a9 builds.

  I will now build 2.0.0 with new libraries and rerun the benchmark.

      Arlindo





> **
>
> John****
>
> ** **
>
> ** **
>
> ** **
>
> *From:* Jennifer Adams [mailto:jm...@co...]
> *Sent:* Wednesday, October 05, 2011 3:39 PM
> *To:* Arlindo da Silva
> *Cc:* Brian Doty; Huddleston, John
> *Subject:* Re: 4.1.1 vs. 4.1.2****
>
> ** **
>
> I think netcdf-4.1.2 (with zlib-1.2.5 and hdf5-1.8.7) is what we should use
> for GrADS 2.0.0. We've been using 4.1.2 at COLA for just about a year. I
> have made new builds for darwin and CentOS and posted them to our FTP
> server, and I changed my supplibs.html page too. Arlindo, thanks for
> bringing this bug to my attention. John, I guess you have no extra work
> since your builds are already linked with 4.1.2. ****
>
>  --Jennifer****
>
> ** **
>
> p.s. Here is what Dennis Heimbigner wrote about the performance bug: ****
>
> "The performance bug has been in the code since almost****
>
> the beginning of opendap support.  It is possible
> (even probable) that you will not encounter it.
> It occurs when you read a large variable and specify
> it as a constraint in the url. E.g. http://x.com/y.nc?var[0:1]
> The bug is that the code ignores the [0:1] and reads
> the whole variable.  The result is correct, but unnecessary
> data is read." ****
>
> ** **
>
> I don't exactly know whether that syntax he shows is different from the
> requests in the gds log that look like this:****
>
> "...GET /gfs.dods?ps.ps[0][0:360][0:719] ..."****
>
> Maybe they ARE different and that's why we don't encounter a performance
> change between old versions and the new snapshot. In any event, I think
> 4.1.2 is the way to go for now. ****
>
> ** **
>
> ** **
>
> On Oct 5, 2011, at 1:20 PM, Arlindo da Silva wrote:****
>
>
>
> ****
>
> ** **
>
> On Wed, Oct 5, 2011 at 11:51 AM, Jennifer Adams <jm...@co...> wrote:
> ****
>
> Hi, Arlindo -- ****
>
> The release notes for 4.1.2 (
> http://www.unidata.ucar.edu/software/netcdf/release-notes-4.1.2.html )
> mention a lot of items that are relevant for GrADS, one especially that was
> pointed out by me (speedup in opening of large files). I will definitely not
> be going to back to 4.1.1. We've been using 4.1.2 at COLA for a long time.
> ****
>
> ** **
>
> ** **
>
> This is good to know. If you have been using 4.1.2 for sometime than this
> is proof of stability. I'll follow your lead.****
>
>  ****
>
> Here are the 7 tests I ran with the two builds -- I could not detect any
> difference in the time it took for these tasks to run when comparing 4.1.2
> and 4.2.snapshot.****
>
> ** **
>
> I share your concern about 4.2. First, it is very new, and second, a large
> portion of the opendap codebase was apparently rewritten. Let me know when
> you make the final decision about 4.1.2 and I'll update my supplibs
> accordingly.****
>
> ** **
>
>   Cheers!****
>
> ** **
>
>      Arlindo****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
>  ****
>
>  ****
>
> --Jennifer****
>
> ** **
>
> 'reinit'****
>
> '!date'****
>
> ** **
>
> if (0)****
>
> * The original bug****
>
>  'sdfopen
> http://opendap.gsfc.nasa.gov:9090/dods/GEOS-5/fp/0.25_deg/assim/inst1_2d_hwl_Nx'
> ****
>
>  'set x 1 1152'****
>
>  'd slp'****
>
>  say result****
>
> endif****
>
> ** **
>
> if (0)****
>
> * NOMADS GDS****
>
>  'sdfopen http://nomads.ncep.noaa.gov:9090/dods/gfs/gfs20111003/gfs_00z'**
> **
>
>  'set x 1 360'****
>
>  'set y 1 181'****
>
>  'set z 1 26'****
>
>  'set t 1 65'****
>
>  'define foo = hgtprs'****
>
> endif****
>
> ** **
>
> if (0)****
>
> * COLA GDS, Ensembles****
>
>  'sdfopen http://monsoondata.org:9090/dods/gfsens/gfsens.2011100400'****
>
>  'set x 1 360'****
>
>  'set y 1 181'****
>
>  'set z 1 7'****
>
>  'set e 1 22'****
>
>  'define foo = z'****
>
> endif****
>
> ** **
>
> if (0)****
>
> * Netcdf4 high res data behind GDS****
>
>  'sdfopen http://monsoondata.org:9090/dods/nicam/sst'****
>
>  'set x 1 5120'****
>
>  'set y 1 2556'****
>
>  'd ave(sst,t=1,t=8)'****
>
> endif****
>
> ** **
>
> if (0)****
>
> * Netcdf4 high res data on local disk****
>
>  'open /data/hdf5/grids/sst.ctl'****
>
>  'set x 1 5120'****
>
>  'set y 1 2556'****
>
>  'd ave(sst,t=1,t=8)'****
>
> endif****
>
> ** **
>
> if (0)****
>
> * Local file, PDEF'd****
>
>  'open /data/netcdf/gswp/Albedo_mean_csu.ctl'****
>
>  'set x 1 360'****
>
>  'set y 1 150'****
>
>  'set z 1'****
>
>  'set t 1 12'****
>
>  'define foo = albedo'****
>
> endif****
>
> ** **
>
> if (1)****
>
> * Classic netcdf****
>
>  'sdfopen /data/netcdf/rean/air.mon.mean.nc'****
>
>  'set x 1 144'****
>
>  'set y 1 73'****
>
>  'set z 1 17'****
>
>  'set t 1 349'****
>
>  'define foo = air'****
>
> endif****
>
> ** **
>
> '!date'****
>
> ** **
>
> ** **
>
> On Oct 5, 2011, at 10:26 AM, Arlindo da Silva wrote:****
>
>
>
> ****
>
> On Tue, Oct 4, 2011 at 5:06 PM, Jennifer Adams <jm...@co...> wrote:*
> ***
>
> Hi, Dennis -- ****
>
> Using the same GrADS source code, I have created two builds, one linked
> with netcdf-4.1.2 and the other netcdf-4.2-snapshot2011100320. Both use
> hdf5-1.8.7. The bug that showed up with 4.1.3 does not reproduce with either
> build. Furthermore, I am unable to detect any performance difference with
> these two builds, having tested some bulky I/O requests via OPeNDAP from
> several different GDS servers, and also from local classic netcdf files and
> very large compressed netcdf4 files. I am inclined to use 4.1.2 for my GrADS
> release. Can you give me an example for how to test the performance
> enhancements in the 4.2 snapshot? ****
>
> ** **
>
> I have a similar experience. Comparing builds with older 4.1.1 and most
> recent 4.2 snapshot (with subsetting) gives essentially the same timing (if
> anything, 4.1.1 is a tiny bit faster: 49.69 vs 45.63, but this difference
> could be "natural variability").  ****
>
> ** **
>
> Jennifer:  BTW, here is my grads benchmark:****
>
> ** **
>
>      lats4d -i
> http://opendap.gsfc.nasa.gov:9090/dods/GEOS-5/fp/0.25_deg/assim/inst1_2d_hwl_Nx-format stats -lat 0 30 -ntimes 10
> ****
>
> ** **
>
> Since 4.1.1 has been working for us very reliably for over a year, I am
> very tempted to stick with it in name of stability. Are there any major bug
> fixes going from 4.1.1 to 4.1.2?****
>
> ** **
>
>    Arlindo****
>
> ** **
>
>   ****
>
> ** **
>
>  ****
>
> ** **
>
> ** **
>
> --****
>
> Jennifer M. Adams****
>
> IGES/COLA****
>
> 4041 Powder Mill Road, Suite 302****
>
> Calverton, MD 20705****
>
> jm...@co...****
>
> ** **
>
>
>
> ****
>
> ** **
>
> ** **
>
> ** **
>
> --****
>
> Jennifer M. Adams****
>
> IGES/COLA****
>
> 4041 Powder Mill Road, Suite 302****
>
> Calverton, MD 20705****
>
> jm...@co...****
>
> ** **
>
>
>
> ****
>
> ** **
>

[Opengrads-devel] OpenGrADS Bundle 2.0.0 almost ready

[Arlindo da S. <da...@al...>]

All,

   I have uploaded sources and binaries (Linux and Mac OS X for now) to
sf.net

        https://sourceforge.net/projects/opengrads/files/grads2/2.0.0.oga.1/


   Of course, this requires the new supplibs-2.2.1:

        https://sourceforge.net/projects/opengrads/files/supplibs/2.2.1/

  Notice that for the first time I am including 2 set of binaries for the
Mac: one for Leopard (macosx_10.5, 32-bit only) and another for Snow
Leopard/Lion (macosx_10.6, both 32 and 64-bits). The Snow Leopard binaries
are universal, although the file is branded "x86_64" they include both 32
and 64 bits. The 10.5 binaries work for Mac OS X 10.5 thru 10.7; the 10.6
binaries work for Mac OS X 10.6-10.7.

  Another addition that I'll not call attention for in the user list is a
new binary called "sgrads" for "Secure GrADS". This is a special build that
disables X11, system() and other such features to make easier the deployment
of grads web servers (as Daniel's http://webgrads.me).

 Before I announce these binaries to gradsusr, I'd appreciate if some of you
could try out these binaries on your specific platforms and send me
feedback.

 I am off to prepare the windows superpack. I dread the prospect of having
to boot into windows...

 Cheers!

     Arlindo

-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] Gxshad error

[Brian D. <do...@co...>]

shade2 should work (crosses fingers)...

On Oct 6, 2011, at 7:36 PM, Jennifer Adams wrote:

> Hi, Gary --
>
> Here is the relevant code in gxshad.c:
>
> #define XYBMAX 5000
> ...
>     if (stkcnt>=XYBMAX) {
>       printf ("Buffer stack limit exceeded in gxshad\n");
>       return(1);
>     }
>
> Looks like you've bumped up against a hard-coded limit. You can  
> change XYBMAX and recompile, or try 'set gxout shade2' and see if  
> that changes anything.
> --Jennifer
>
>
> On Oct 6, 2011, at 7:07 PM, Love, Mr. Gary, Contractor, Code 7542  
> wrote:
>
>> Hi Jennifer,
>>
>> I just encountered any error I have never seen
>>
>> Buffer stack limit exceeded in gxshad
>> Error in gxshad
>>
>> I'm only shading about a dozen colors, but I get this when the plot  
>> is very dense.
>>
>> Are there any quick solutions? Reduce colors, etc?
>>
>> Gary
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jm...@co...
>
>
>

Re: [Opengrads-devel] Gxshad error

[Jennifer A. <jm...@co...>]

Hi, Gary --

Here is the relevant code in gxshad.c:

#define XYBMAX 5000
...
     if (stkcnt>=XYBMAX) {
       printf ("Buffer stack limit exceeded in gxshad\n");
       return(1);
     }

Looks like you've bumped up against a hard-coded limit. You can change  
XYBMAX and recompile, or try 'set gxout shade2' and see if that  
changes anything.
--Jennifer


On Oct 6, 2011, at 7:07 PM, Love, Mr. Gary, Contractor, Code 7542 wrote:

> Hi Jennifer,
>
> I just encountered any error I have never seen
>
> Buffer stack limit exceeded in gxshad
> Error in gxshad
>
> I'm only shading about a dozen colors, but I get this when the plot  
> is very dense.
>
> Are there any quick solutions? Reduce colors, etc?
>
> Gary

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

[Opengrads-devel] Gxshad error

[Love, M. G. C. C. 7. <gar...@nr...>]

Hi Jennifer,

I just encountered any error I have never seen

Buffer stack limit exceeded in gxshad
Error in gxshad

I'm only shading about a dozen colors, but I get this when the plot is very dense.

Are there any quick solutions? Reduce colors, etc?

Gary