opengrads-devel — OpenGrADS development issues, with focus on interfaces and extensions

Re: [Opengrads-devel] New Win32 GrADS build

[See H. O. <ax...@ya...>]

Dear Mr Arlindo,

     Thank you.

     The outcome of my testing your
grads-1.9.0-rc.win32_superpack.exe is as follows:-

    (1) as compared with you zip package, I have to
select grads classic or grads netCDF-3 or netCDF-4 in
order to run .grb or .nc file appropriately.  [In your
zip package, I just type grads in the command window
and run either .grb or .nc file easily.]
    (2) q udft shows no listing of the table at all.
    (3) your gex folder in win32 is empty.
    (4) when I run your utFish.gs,  'psi', 'chi',
'fish' are shown as not a variable or function name.
    (5) when I tried you re() function, re is shown to
be not a variable or function name.  Also, gxyat is an
unknown command.

      Lastly, yesterday I emailed you that I am
puzzled by the streamfunction output by using fish(). 
The pattern, as compared with an example from an
article, looks similar but the signs and magnitudes
are vastly different.  I am pleased to reproduce below
my email text with the attached example for your
consideration and help.

--------------------------------
Dear Mr Arlindo da Silva,

     For days, I have been puzzled by the output of
streamfunction using your fish() in the GrADS
v1.9.0-rc1 (windows).  By chance, I happened to see a
700 hPa streamfunction example in an article on Afican
Easterly Wave.  I used the NCEP/NCAR Reanalysis data
to reproduce the example shown.  As shown in my
attachment, you can notice the followings:-
     (1) Both patterns look similar.
     (2) The signs and magnitudes of streamfunction
generated from GrADS are vastly different from those
in the example figure.
     (3) In the GrADS generated example, values west
of 0 degree longitude are absent.

      Where did I go wrong ? Please help.
------------------------

      Thank you and best regards.


OOI See-hai



--- Arlindo da Silva <da...@al...> wrote:

> Hi,
> 
>   I have a new Win32 build at
> 
>
http://opengrads.org/pre-rel/win32/grads-1.9.0-rc.win32_superpack.exe
> 
> that I hope fixes several of the issues some of you
> have reported:
> 
> 1) Xmin should now start automatically (meaning some
> of the mods I made
> could solve this problem that some of you had).
> 
> 2) The "!" command in grads should work
> 
> 3) The "^" as in "DSET ^filename" in ctl files
> should work now
> 
> 4) Other niceties: Setup.exe type of installation,
> PATH is now set
> automatically, can be uninstalled from control
> pannel.
> 
> A main change is that the suggested default
> installation directory is
> "C:\GrADS19", so that v2 could later be installed
> under C:\GrADS20. I plan
> to post this new build on sf.net in the next couple
> of days, unless I hear
> from you of major problems.
> 
> BTW, despite the Setup.exe installation, the whole
> package should still be
> relocatable to mem stick or whatever.
> 
>   Let me know of any problems.
> 
>   Thank you,
> 
>      Arlindo
 


Send instant messages to your online friends http://uk.messenger.yahoo.com 

Re: [Opengrads-devel] New Win32 GrADS build

[<smc...@pl...>]

Thanks Arlindo! I removed the .zip version and replaced with this one. 
Installation was quick and simple. This time, unlike with .zip version, I 
was able to launch right away. However, when I tried to use gxyat it 
didn't work, even after I added GAUDXT as an env variable and updated the 
path. But then I noticed the \win32\gex folder was empty, although the 
gxyat application is in \win32. Were they intentially left out?

Stephen Mc



"Arlindo da Silva" <da...@al...> 
Sent by: arl...@gm...
02/23/2008 06:34 PM

To
"Michael G Bosilovich" <Mic...@na...>, "Maat, Herbert ter" 
<Her...@wu...>, "Bill Reilly" <Bil...@co...>, "See 
Hai Ooi" <ax...@ya...>, "Stephen R McMillan" 
<smc...@pl...>, "Eric Altshuler" <el...@co...>
cc
ope...@li...
Subject
New Win32 GrADS build





Hi,
 
  I have a new Win32 build at
 
http://opengrads.org/pre-rel/win32/grads-1.9.0-rc.win32_superpack.exe
 
that I hope fixes several of the issues some of you have reported:
 
1) Xmin should now start automatically (meaning some of the mods I made 
could solve this problem that some of you had).
 
2) The "!" command in grads should work
 
3) The "^" as in "DSET ^filename" in ctl files should work now
 
4) Other niceties: Setup.exe type of installation, PATH is now set 
automatically, can be uninstalled from control pannel.
 
A main change is that the suggested default installation directory is 
"C:\GrADS19", so that v2 could later be installed under C:\GrADS20. I plan 
to post this new build on sf.net in the next couple of days, unless I hear 
from you of major problems.
 
BTW, despite the Setup.exe installation, the whole package should still be 
relocatable to mem stick or whatever.
 
  Let me know of any problems.
 
  Thank you,
 
     Arlindo
 
 
 
 
 
 
 


-- 
Arlindo da Silva
da...@al... 



***************************************************
The information contained in this e-mail message is intended only for the use of the recipient(s) named above and may contain information that is privileged, confidential, and/or proprietary.  If you are not the intended recipient, you may not review, copy or distribute this message.  If you have received this communication in error, please notify the sender immediately by e-mail, and delete the original message.
***************************************************

[Opengrads-devel] New Win32 GrADS build

[Arlindo da S. <da...@al...>]

Hi,

  I have a new Win32 build at

http://opengrads.org/pre-rel/win32/grads-1.9.0-rc.win32_superpack.exe

that I hope fixes several of the issues some of you have reported:

1) Xmin should now start automatically (meaning some of the mods I made
could solve this problem that some of you had).

2) The "!" command in grads should work

3) The "^" as in "DSET ^filename" in ctl files should work now

4) Other niceties: Setup.exe type of installation, PATH is now set
automatically, can be uninstalled from control pannel.

A main change is that the suggested default installation directory is
"C:\GrADS19", so that v2 could later be installed under C:\GrADS20. I plan
to post this new build on sf.net in the next couple of days, unless I hear
from you of major problems.

BTW, despite the Setup.exe installation, the whole package should still be
relocatable to mem stick or whatever.

  Let me know of any problems.

  Thank you,

     Arlindo









-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] Ncurses added to the supplibs

[Patrice D. <per...@fr...>]

On Wed, Feb 20, 2008 at 02:08:04PM -0500, Arlindo da Silva wrote:
> All,
> 
>   I just added ncurses-5.6 to the supplibs source tree.  Readline uses
> either termcap or ncurses, whichever was present on the build machine. Since

In fact termcap is deprecated and in the future ncurses will be used
everywhere instead. In fedora, for example there is no termcap anymore
(or at least it isn't used by anything in fedora anymore).

--
Pat

[Opengrads-devel] Ncurses added to the supplibs

[Arlindo da S. <da...@al...>]

All,

  I just added ncurses-5.6 to the supplibs source tree.  Readline uses
either termcap or ncurses, whichever was present on the build machine. Since
some distributions only have termcap while others only have ncurses, this
has been a portability impedemente. A case in point is Hoop's i686 binaries
that Jennifer just announced. It fails on my mandriva box with the error

./grads: error while loading shared libraries: libtermcap.so.2: cannot open
shared object file: No such file or directory

If you have built the supplibs from sources he is how to update:

% cd supplibs/src
% cvs upd -d GNUmakefile ncurses
% rm readline.*
% make install ALLDIRS="ncurses readline"

Then rebuild grads and ncurses should link statically.

     Arlindo

-- 
Arlindo da Silva
da...@al...

[Opengrads-devel] Wgrib2: first autotoolized version

[Arlindo da S. <da...@al...>]

Hi Pat,

  I had the first crack adding a configure script to wgrib2, borrowing from
much of the stuff you did for the grads build. To check it out:

% gacvs co -P Wgrib2

I only tested it with the supplibs-2.0.1 and it builds, very much like
grads. (Ignore the HDF option: I am detecting it but have not done anything
with it yet - I am waiting for Sergey's patch of Netcdf.c).

When you have a chance:

1) Can you check whether the option "--enable-dynamic-supplibs" works?

2) "make dist" is broken; do you know what I am doing wrong? (make bin-dist
works, though)

As for the supplibs, you only need a fraction of the grads supplibs to build
wgrib2. Given the modularity of the supplibs, you can delete all the
non-necessary packages and build only what you need for wgrib2. If you
checkout this tag you will get this subset:

gacvs co -P -r mini_supplibs-2_0_1 supplibs

Wes/Sergey: you can access the opengrads cvs repo anonymously, see here:

         http://sourceforge.net/cvs/?group_id=161773

If you have problems checking stuff out let me know and I'll post  tarball.
You should be able to get by with the supplib binaries on sf.net:


http://sourceforge.net/project/showfiles.php?group_id=161773&package_id=241681

For building instructions see the INSTALL file for both wgrib2 and the
supplibs; some build info is here:

        http://opengrads.org/wiki/index.php?title=OpenGrADS_Documentation

We are almost there with this. The next step is to fix these small glitches
and add HDF support.

  Cheers!

     Arlindo

-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] GrADS 2.0.a1 - first impressions

[Mike F. <mfi...@gm...>]

hi arlindo,

please see:

ftp://ftp.nhc.noaa.gov/users/fiorino/model.tar

for a sample of 0.5 deg gfs data for tau=0,24,48,72 from today's gfs run
(2008021312) and for 850,500,200

-rw-r--r-- mfiorino/users  592 2008-02-13 23:45:37 model1.ctl
-rw-r--r-- mfiorino/users 1705 2008-02-13 23:45:48 model1.gmp1
-rwxr-xr-x mfiorino/users 6174444 2008-02-13 23:35:49 model.f000.grb1
-rw-r--r-- mfiorino/users 2176111 2008-02-13 23:35:40 model.f000.grb2
-rwxr-xr-x mfiorino/users 6401958 2008-02-13 23:36:04 model.f024.grb1
-rw-r--r-- mfiorino/users 2115250 2008-02-13 23:35:55 model.f024.grb2
-rwxr-xr-x mfiorino/users 6531918 2008-02-13 23:36:18 model.f048.grb1
-rw-r--r-- mfiorino/users 2319105 2008-02-13 23:36:09 model.f048.grb2
-rwxr-xr-x mfiorino/users 6531918 2008-02-13 23:36:33 model.f072.grb1
-rw-r--r-- mfiorino/users 2295825 2008-02-13 23:36:24 model.f072.grb2

the .grb2 data comes off the operational file system so is as "ncep" as
you get...

i used convgrb to convert to grib2 (.grb1) and the model1.ctl works
against the .grb1 data.

these files are a lot beefier than the original model.grb, but then so
are the models..  let me know if you need more/less fields, i have a
pretty slick python script that uses wgrib2 to inventory and extract out
specific fields...

best /r mike



Arlindo da Silva wrote:
> All,
>
>    I just downloaded Jennifer's latest sources, merged with my 
> previous patches, and checked back in our repo. To checked it out:
>
>    gacvs co -P -r GRADS2_DEV_BRANCH grads
>
> or else, grab sources and a couple of binaries from
>
>   http://opengrads.org/pre-rel
>
> These sources are supplibs-2.0.1 ready. I only had to do actual merges 
> in 2 files: grads.h and gasdf.c (for the gasdf.h/netcdf.h issue I 
> mentioned before - see ChangeLog). All other *source* files have been 
> replaced with Jennifer's latest "as is"; for the *build scripts* I 
> have kept what we had before as it builds out of the box with the new 
> supplibs (I also added FreeBSD support). First impressions:
>
> 1) It now passes all the HDF tests! Thanks Jennifer! I only tested on 
> Linux i686, and FreeBSD.
>
> 2) It still fails the fwrite tests, where it writes a bogus value 
> instead of the true undef value. I am not sure if Jennifer had a 
> chance to look into it yet.
>
> 3)  We still do not have a grib2 specific test in the test suite. I 
> just need a small enough file to be included with the sources, but yet 
> have a couple of variables, times and levels. Converting the model.grb 
> to grib2 with cnvgrib would be best as I could reused many of the 
> existing tests, but according to Wes our model.grb is not ncep-kosher, 
> so cnvgrib barfs at it.
>
> Mike: would you know how to make a ncep-kosher model.grb file? That 
> is, following all their standard tables, etc.
>
> Jennifer: I am not sure if you had a chance to look at my previous 
> patches. If you are still interested, just look at this one:
>
>    http://opengrads.org/pre-rel/grads-2.0.a1.oga.1-src.tar.gz
>   
>    Let me know if you need help putting out builds, otherwise I'm not 
> intending to distribute any of my builds. For this it would be good if 
> we could reconcile our codebases.
>
>   Cheers!
>
>          Arlindo
>
>
> ---------- Forwarded message ----------
> From: *Jennifer Adams* <jm...@co... <mailto:jm...@co...>>
> Date: Feb 13, 2008 7:20 PM
> Subject: GrADS 2.0.a1
> To: GRA...@li... <mailto:GRA...@li...>
>
>
> I have posted a new release -- 2.0.a1. The src and binaries are here:
>
> ftp://iges.org/grads/2.0/grads-src-2.0.a1.tar.gz
> ftp://iges.org/grads/2.0/grads-bin-2.0.a1-darwin-intel.tar.gz
> ftp://iges.org/grads/2.0/grads-bin-2.0.a1-RHEL5-x86_64.tar.gz
>
> This release has some significant bug fixes:
>
> 0. Detection of linear and wrapping axes for netcdf files opened with 
> sdfopen
> 1. GRIB2 grids that have a bitmap were being drawn upside down
> 2. Fixed display for gxout=model and gxout=grid
> 3. Fixed handling of templated netcdf data sets opened with xdfopen
> 4. Fixed detection of unlimited axis in hdf files opened with sdfopen
> 5. Fixed I/O of PDEF files
> 6. Fix to gribmap when creating index files >250mb 
> 7. Added some debug statements for detection of errors in calls to 
> seekgb() in the grib2c library. 
>
> These bugs were causing catastrophic errors (especially #6) and so it 
> seemed worthwhile to put out a new release. I have yet to make any 
> changes to the build system -- I know it's hard to build from source, 
> and I hope to make it easier, but right now fixing catastrophic bugs 
> in the GrADS core is a higher priority for me. 
>
> To take advantage of the debug statements (#7 above), you must change 
> the value of debug in line 75 of gaio.c: 
>   static gaint debug=1;
> and then build from source. The binary distributions have debug set to 0. 
>
> I will work on getting out some more binaries for: Mac PowerPC, SunOS, 
> and RHEL4 x86_64. Sorry, we don't have any 32-bit linux boxes at COLA 
> anymore. 
>
> Please post comments and questions and bug reports here, no matter how 
> technical they are. The gradsusr posts are archived and searchable, 
> which makes this the best place to report and resolve problems. 
>
> Jennifer
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jm...@co... <mailto:jm...@co...>
>
>
>
>
>
>
> -- 
> Arlindo da Silva
> da...@al... <mailto:da...@al...>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> ------------------------------------------------------------------------
>
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>   

[Opengrads-devel] GrADS 2.0.a1 - first impressions

[Arlindo da S. <da...@al...>]

All,

   I just downloaded Jennifer's latest sources, merged with my previous
patches, and checked back in our repo. To checked it out:

   gacvs co -P -r GRADS2_DEV_BRANCH grads

or else, grab sources and a couple of binaries from

  http://opengrads.org/pre-rel

These sources are supplibs-2.0.1 ready. I only had to do actual merges in 2
files: grads.h and gasdf.c (for the gasdf.h/netcdf.h issue I mentioned
before - see ChangeLog). All other *source* files have been replaced with
Jennifer's latest "as is"; for the *build scripts* I have kept what we had
before as it builds out of the box with the new supplibs (I also added
FreeBSD support). First impressions:

1) It now passes all the HDF tests! Thanks Jennifer! I only tested on Linux
i686, and FreeBSD.

2) It still fails the fwrite tests, where it writes a bogus value instead of
the true undef value. I am not sure if Jennifer had a chance to look into it
yet.

3)  We still do not have a grib2 specific test in the test suite. I just
need a small enough file to be included with the sources, but yet have a
couple of variables, times and levels. Converting the model.grb to grib2
with cnvgrib would be best as I could reused many of the existing tests, but
according to Wes our model.grb is not ncep-kosher, so cnvgrib barfs at it.

Mike: would you know how to make a ncep-kosher model.grb file? That is,
following all their standard tables, etc.

Jennifer: I am not sure if you had a chance to look at my previous patches.
If you are still interested, just look at this one:

   http://opengrads.org/pre-rel/grads-2.0.a1.oga.1-src.tar.gz

   Let me know if you need help putting out builds, otherwise I'm not
intending to distribute any of my builds. For this it would be good if we
could reconcile our codebases.

  Cheers!

         Arlindo


---------- Forwarded message ----------
From: Jennifer Adams <jm...@co...>
Date: Feb 13, 2008 7:20 PM
Subject: GrADS 2.0.a1
To: GRA...@li...


I have posted a new release -- 2.0.a1. The src and binaries are here:
ftp://iges.org/grads/2.0/grads-src-2.0.a1.tar.gz
ftp://iges.org/grads/2.0/grads-bin-2.0.a1-darwin-intel.tar.gz
ftp://iges.org/grads/2.0/grads-bin-2.0.a1-RHEL5-x86_64.tar.gz

This release has some significant bug fixes:
0. Detection of linear and wrapping axes for netcdf files opened with
sdfopen
1. GRIB2 grids that have a bitmap were being drawn upside down
2. Fixed display for gxout=model and gxout=grid
3. Fixed handling of templated netcdf data sets opened with xdfopen
4. Fixed detection of unlimited axis in hdf files opened with sdfopen
5. Fixed I/O of PDEF files
6. Fix to gribmap when creating index files >250mb
7. Added some debug statements for detection of errors in calls to seekgb()
in the grib2c library.

These bugs were causing catastrophic errors (especially #6) and so it seemed
worthwhile to put out a new release. I have yet to make any changes to the
build system -- I know it's hard to build from source, and I hope to make it
easier, but right now fixing catastrophic bugs in the GrADS core is a higher
priority for me.

To take advantage of the debug statements (#7 above), you must change the
value of debug in line 75 of gaio.c:
  static gaint debug=1;
and then build from source. The binary distributions have debug set to 0.

I will work on getting out some more binaries for: Mac PowerPC, SunOS, and
RHEL4 x86_64. Sorry, we don't have any 32-bit linux boxes at COLA anymore.

Please post comments and questions and bug reports here, no matter how
technical they are. The gradsusr posts are archived and searchable, which
makes this the best place to report and resolve problems.

Jennifer

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...






-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] errors on sdfopen with hdf

[Jennifer A. <jm...@co...>]

GrADS has got it right now:

dim_id,name,dimsize,dtype,ndatts = 327680 time 5 6 1
dim_id,name,dimsize,dtype,ndatts = 327681 longitude 72 6 2
dim_id,name,dimsize,dtype,ndatts = 327682 latitude 46 6 2
dim_id,name,dimsize,dtype,ndatts = 327683 levels 7 6 5

Thanks, Hoop!
Jennifer

Re: [Opengrads-devel] errors on sdfopen with hdf

[Jennifer A. <jm...@co...>]

Hoop pointed out that if SDdiminfo returns 0, then it's an unlimited  
dimension and requires special treatment. I will make some changes  
and that will likely solve the problem.
Jennifer

Re: [Opengrads-devel] [Junk released by Allow List] Building GrADS v2 with new supplibs: first crack

[Jennifer A. <jm...@co...>]

On Feb 2, 2008, at 3:21 PM, Arlindo da Silva wrote:

> b) When using "set gxout fvwrite" to write binary files the missing  
> value is not being written correctly, in either little-endian or  
> big-endian mode. Is this a bug or a feature (I haven't verified  
> whether v1.9 does the same)?

I found a bug in the HDF pathway in gasdf.c :
Near the end of the gadsdf() subroutine, the varids in the gavar  
structure for each variable are reset to -999 so that the undef  
values and scale factor/add offset values will be retrieved in the I/ 
O layer. I did it for ncvids, but not sdvids. Add the 2nd line below  
and see if that isn't a workable short term fix.
         newpvar->ncvid = -999; /* reset the varid so undefs will be  
read in gaio.c */
         newpvar->sdvid = -999; /* reset the varid so undefs will be  
read in gaio.c */

There have been other patches to gasdf.c in the past week, as well as  
other unrelated bug fixes, so that one change may not fix everything  
for you. I will be putting out 2.0.a1 pretty soon.
Jennifer

Re: [Opengrads-devel] errors on sdfopen with hdf

[Jennifer A. <jm...@co...>]

On Feb 6, 2008, at 3:36 PM, Arlindo da Silva wrote:

>
>
> On Feb 6, 2008 2:27 PM, Jennifer Adams <jm...@co...> wrote:
>
> On Feb 6, 2008, at 1:09 PM, Arlindo da Silva wrote:
>
>> On Feb 6, 2008 10:44 AM, Jennifer Adams <jm...@co...> wrote:
>>
>> Arlindo,
>> I have put some print statments in gasdf.c to see why this is  
>> happening.
>> The HDF routine SDdiminfo returns the following information on
>> model.hdf:
>>
>> dim_id,name,dimsize,dtype,ndatts = 327680 time 0 6 1
>> dim_id,name,dimsize,dtype,ndatts = 327681 longitude 72 6 2
>> dim_id,name,dimsize,dtype,ndatts = 327682 latitude 46 6 2
>> dim_id,name,dimsize,dtype,ndatts = 327683 levels 7 6 5
>>
>> Looks like the file has a time dimension of size 0. I remember now
>> that when I discovered this during development, I put in the
>> following line:
>>        if (dimsize==0) dimsize=1; /* would you have a coordinate  
>> axis with
>> size==0 */
>>
>> The interesting thing is that when I run ncdump (from the HDF4.2r2
>> library build) it gives me this:
>>
>> dimensions:
>>         time = UNLIMITED ; // (5 currently)
>>         longitude = 72 ;
>>         latitude = 46 ;
>>         levels = 7 ;
>>
>> It is possible that this is a bug in the SD interface. Ncdump uses  
>> the "NC" interface which is sometimes a more basic, low level  
>> interface than SD.  (I say "sometimes" because the hdf-4 is not so  
>> cleanly layered). I suspect that your ctl/"DTYPE hdf" interface  
>> will have similar problems, correct?
> The 'dtype hdfsds' works fine because the required TDEF metadata is  
> provided in the descriptor file -- is doesn't need to use SDdiminfo  
> to discover the size of the time axis.
>
> However, can it access variables for t>1? It is possible that the  
> library has checks for not allowing access beyond the dimension size.

Here's the descriptor:
dset ../../opengrads/pytests/data/model.hdf
dtype hdfsds
title test
undef -8888 missing_value
xdef 72 linear 0 5
ydef 46 linear -90 4
zdef 7 levels 1000 850 700 500 300 200 100
tdef 5 linear 00Z01JAN1987 1dy
vars 1
ua 7 t,z,y,x Eastward wind [m/s]
endvars

All 5 time steps are displayed.

>
>
>
>> The file "model.hdf" was written with the "NC" interface through  
>> LATS. Leading to this version of hdf-4, I reported a couple of  
>> bugs to THG in the hrepack utility which  did not work properly  
>> with HDF files written by the "NC" interface (all GMAO files are  
>> written that way, although LATS is not involved).  It is very  
>> possible that this is another one of those bugs.
>>
>> We have 2 options, report the bug to the THG (we will need a small  
>> example demonstrating the problem), or try to dig in the hdf-4  
>> sources and contribute a patch to them. They have usually being  
>> very responsive to my bug reports. What do you prefer?
>
> As I see it, there is a bug in the lats4d pathway that creates hdf  
> files, not in the v2 code that reads them.
>
> It is always possible, but I find it very unlikely. I am in the  
> middle of a day long meeting. I'll confirm later in day whether the  
> same problem occurs with the official GMAO files which do not  
> involves LATS at all. I expect it does.
By "lats4d pathway" I meant include the use of the HDF library to  
write out the files. It may not be in the LATS code proper, but it's  
somewhere along the line between the lats4d command at the GrADS  
command prompt and the output file model.hdf.
Jennifer

>
>
>     Arlindo.
>
> -- 
> Arlindo da Silva
> da...@al...

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] errors on sdfopen with hdf

[Arlindo da S. <da...@al...>]

On Feb 6, 2008 2:27 PM, Jennifer Adams <jm...@co...> wrote:

>
> On Feb 6, 2008, at 1:09 PM, Arlindo da Silva wrote:
>
> On Feb 6, 2008 10:44 AM, Jennifer Adams <jm...@co...> wrote:
>
> >
> > Arlindo,
> > I have put some print statments in gasdf.c to see why this is happening.
> > The HDF routine SDdiminfo returns the following information on
> > model.hdf:
> >
> > dim_id,name,dimsize,dtype,ndatts = 327680 time 0 6 1
> > dim_id,name,dimsize,dtype,ndatts = 327681 longitude 72 6 2
> > dim_id,name,dimsize,dtype,ndatts = 327682 latitude 46 6 2
> > dim_id,name,dimsize,dtype,ndatts = 327683 levels 7 6 5
> >
> > Looks like the file has a time dimension of size 0. I remember now
> > that when I discovered this during development, I put in the
> > following line:
> >        if (dimsize==0) dimsize=1; /* would you have a coordinate axis
> > with
> > size==0 */
> >
> > The interesting thing is that when I run ncdump (from the HDF4.2r2
> > library build) it gives me this:
> >
> > dimensions:
> >         time = UNLIMITED ; // (5 currently)
> >         longitude = 72 ;
> >         latitude = 46 ;
> >         levels = 7 ;
> >
>
> It is possible that this is a bug in the SD interface. Ncdump uses the
> "NC" interface which is sometimes a more basic, low level interface than
> SD.  (I say "sometimes" because the hdf-4 is not so cleanly layered). I
> suspect that your ctl/"DTYPE hdf" interface will have similar problems,
> correct?
>
> The 'dtype hdfsds' works fine because the required TDEF metadata is
> provided in the descriptor file -- is doesn't need to use SDdiminfo to
> discover the size of the time axis.
>

However, can it access variables for t>1? It is possible that the library
has checks for not allowing access beyond the dimension size.


>
> The file "model.hdf" was written with the "NC" interface through LATS. Leading
> to this version of hdf-4, I reported a couple of bugs to THG in the hrepack
> utility which  did not work properly with HDF files written by the "NC"
> interface (all GMAO files are written that way, although LATS is not
> involved).  It is very possible that this is another one of those bugs.
>
>
> We have 2 options, report the bug to the THG (we will need a small example
> demonstrating the problem), or try to dig in the hdf-4 sources and
> contribute a patch to them. They have usually being very responsive to my
> bug reports. What do you prefer?
>
>
> As I see it, there is a bug in the lats4d pathway that creates hdf files,
> not in the v2 code that reads them.
>

It is always possible, but I find it very unlikely. I am in the middle of a
day long meeting. I'll confirm later in day whether the same problem occurs
with the official GMAO files which do not involves LATS at all. I expect it
does.

    Arlindo.

-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] errors on sdfopen with hdf

[Jennifer A. <jm...@co...>]

On Feb 6, 2008, at 1:09 PM, Arlindo da Silva wrote:

> On Feb 6, 2008 10:44 AM, Jennifer Adams <jm...@co...> wrote:
>
> Arlindo,
> I have put some print statments in gasdf.c to see why this is  
> happening.
> The HDF routine SDdiminfo returns the following information on
> model.hdf:
>
> dim_id,name,dimsize,dtype,ndatts = 327680 time 0 6 1
> dim_id,name,dimsize,dtype,ndatts = 327681 longitude 72 6 2
> dim_id,name,dimsize,dtype,ndatts = 327682 latitude 46 6 2
> dim_id,name,dimsize,dtype,ndatts = 327683 levels 7 6 5
>
> Looks like the file has a time dimension of size 0. I remember now
> that when I discovered this during development, I put in the
> following line:
>        if (dimsize==0) dimsize=1; /* would you have a coordinate  
> axis with
> size==0 */
>
> The interesting thing is that when I run ncdump (from the HDF4.2r2
> library build) it gives me this:
>
> dimensions:
>         time = UNLIMITED ; // (5 currently)
>         longitude = 72 ;
>         latitude = 46 ;
>         levels = 7 ;
>
> It is possible that this is a bug in the SD interface. Ncdump uses  
> the "NC" interface which is sometimes a more basic, low level  
> interface than SD.  (I say "sometimes" because the hdf-4 is not so  
> cleanly layered). I suspect that your ctl/"DTYPE hdf" interface  
> will have similar problems, correct?
The 'dtype hdfsds' works fine because the required TDEF metadata is  
provided in the descriptor file -- is doesn't need to use SDdiminfo  
to discover the size of the time axis.

> The file "model.hdf" was written with the "NC" interface through  
> LATS. Leading to this version of hdf-4, I reported a couple of bugs  
> to THG in the hrepack utility which  did not work properly with HDF  
> files written by the "NC" interface (all GMAO files are written  
> that way, although LATS is not involved).  It is very possible that  
> this is another one of those bugs.
>
> We have 2 options, report the bug to the THG (we will need a small  
> example demonstrating the problem), or try to dig in the hdf-4  
> sources and contribute a patch to them. They have usually being  
> very responsive to my bug reports. What do you prefer?

As I see it, there is a bug in the lats4d pathway that creates hdf  
files, not in the v2 code that reads them.

Jennifer

Re: [Opengrads-devel] errors on sdfopen with hdf

[Arlindo da S. <da...@al...>]

On Feb 6, 2008 10:44 AM, Jennifer Adams <jm...@co...> wrote:

>
> Arlindo,
> I have put some print statments in gasdf.c to see why this is happening.
> The HDF routine SDdiminfo returns the following information on
> model.hdf:
>
> dim_id,name,dimsize,dtype,ndatts = 327680 time 0 6 1
> dim_id,name,dimsize,dtype,ndatts = 327681 longitude 72 6 2
> dim_id,name,dimsize,dtype,ndatts = 327682 latitude 46 6 2
> dim_id,name,dimsize,dtype,ndatts = 327683 levels 7 6 5
>
> Looks like the file has a time dimension of size 0. I remember now
> that when I discovered this during development, I put in the
> following line:
>        if (dimsize==0) dimsize=1; /* would you have a coordinate axis with
> size==0 */
>
> The interesting thing is that when I run ncdump (from the HDF4.2r2
> library build) it gives me this:
>
> dimensions:
>         time = UNLIMITED ; // (5 currently)
>         longitude = 72 ;
>         latitude = 46 ;
>         levels = 7 ;
>

It is possible that this is a bug in the SD interface. Ncdump uses the "NC"
interface which is sometimes a more basic, low level interface than SD.  (I
say "sometimes" because the hdf-4 is not so cleanly layered). I suspect that
your ctl/"DTYPE hdf" interface will have similar problems, correct?

The file "model.hdf" was written with the "NC" interface through LATS.
Leading to this version of hdf-4, I reported a couple of bugs to THG in the
hrepack utility which  did not work properly with HDF files written by the
"NC" interface (all GMAO files are written that way, although LATS is not
involved).  It is very possible that this is another one of those bugs.

We have 2 options, report the bug to the THG (we will need a small example
demonstrating the problem), or try to dig in the hdf-4 sources and
contribute a patch to them. They have usually being very responsive to my
bug reports. What do you prefer?

   Arlindo

-- 
Arlindo da Silva
da...@al...

[Opengrads-devel] errors on sdfopen with hdf

[Jennifer A. <jm...@co...>]

On Feb 2, 2008, at 3:21 PM, Arlindo da Silva wrote:
>
> 4) Of the 70 tests, gradsv2 is failing 16. They are related to 2  
> issues:
>
> a) The "sdfopen model.hdf" is not quite working: it reports the  
> file as having 1 timestep instead of 5. I verified that the RH  
> binary Jennifer posted has exactly the same problem, so it does not  
> appear to be my build.

Arlindo,
I have put some print statments in gasdf.c to see why this is happening.
The HDF routine SDdiminfo returns the following information on  
model.hdf:

dim_id,name,dimsize,dtype,ndatts = 327680 time 0 6 1
dim_id,name,dimsize,dtype,ndatts = 327681 longitude 72 6 2
dim_id,name,dimsize,dtype,ndatts = 327682 latitude 46 6 2
dim_id,name,dimsize,dtype,ndatts = 327683 levels 7 6 5

Looks like the file has a time dimension of size 0. I remember now  
that when I discovered this during development, I put in the  
following line:
	if (dimsize==0) dimsize=1; /* would you have a coordinate axis with  
size==0 */

The interesting thing is that when I run ncdump (from the HDF4.2r2  
library build) it gives me this:

dimensions:
         time = UNLIMITED ; // (5 currently)
         longitude = 72 ;
         latitude = 46 ;
         levels = 7 ;

--Jennifer

Re: [Opengrads-devel] grads vs gradsdap in v2

[Jennifer A. <jm...@co...>]

On Feb 4, 2008, at 12:51 PM, Arlindo da Silva wrote:

> On Feb 4, 2008 12:08 PM, Jennifer Adams <jm...@co...> wrote:
> In a couple of years the Unidate netcdf library will have swallowed  
> up libnc-dap (see below),
>
> That would be fantastic!
>
> so gradsdap is the one that will eventually be phased out. Until  
> then, I feel it's better to link grads with the Unidata library and  
> gradsdap with OPeNDAP version. Then when nc4 is ready (it's waiting  
> for hdf5 to come out of beta status), we can link grads with nc4  
> library and leave gradsdap alone.
>
> I have been experimenting with the current beta of netcdf4 in the  
> context of v1.9 (binary name: gradsnc4) and I'd say that, pending  
> low level hdf-5 issues which are not visible at the API level,  
> netcdf-4 is pretty ready; gradsnc4 already passes 100% of our  
> tests. I have both netcdf4 beta and the hdf-5  version it requires  
> in the supplibs. These will be replaced with the final versions as  
> they become available.  We have all the m4 discovery macros for it  
> already.
The way I see it, the whole point of moving to nc4 is to take  
advantage of the features that involve hdf5, so there's no point in  
upgrading until hdf5 is solid. I guess there are other advantages,  
such as the groups stuff which might be used for ensembles, but until  
someone steps up with a 5D netcdf file with groups that they want to  
read with GrADS, I'm in no rush to move to nc4.

>
> (The gradsnc4 binary is just as huge as gradsdap, as both libdap  
> and hdf5 are bloated. Put these 2 together we will have a very  
> heavy load, indeed.
The idea is to load up your executable at configure time with all the  
features you want so you can make your build as bulky or lean as you  
like.

> It may be time to bring old gradsc back :-)
:-(  eek! Don't even joke about it!
Actually, I find it convenient that v2 and v19 have different  
executable names -- keeps things clear for the users migrating from  
one to the other.

>
>
> I am trying to keep the data formats and their supplibs and their  
> code paths in GrADS untangled. The HDF Group is also thinking about  
> developing an OPeNDAP client library for HDF5, so that may be  
> another complicating factor.
>
> Speaking as a user, once an OPeNDAP enabled NetCDF-4 is available I  
> see no compeling reason for using HDF-5. I am trying to articulate  
> this point within NASA.
I learned at the HDF5 workshop last fall that the subset of all hdf5  
files that can be read by nc4 is very small. The hdf5 interface in  
GrADS will have to be more general and will require a descriptor  
file, especially as we work toward the quasi-regular data type and  
handling of swath data.

Jennifer

>
>
>   Thank you for the info!
>
>        Arlindo
>
>
>
>
> On Feb 4, 2008, at 11:40 AM, Arlindo da Silva wrote:
>
>> Jennifer,
>>
>>   Does the executable "grads" have any feature not covered in  
>> "gradsdap"? If not, is "grads" really necessary? Sure, nc-dap is  
>> bloated, but nc-dap should be considered robust enough by now. You  
>> could have gone all the way and have a single executable called  
>> "grads", and only on those platforms where OPeNDAP cannot be build  
>> we build it with NetCDF. What were your thoughts for not having a  
>> single executable at this point?
>>
>>         Arlindo
>>
>> -- 
>> Arlindo da Silva
>> da...@al...
>> --------------------------------------------------------------------- 
>> ----
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ 
>> _______________________________________________
>> Opengrads-devel mailing list
>> Ope...@li...
>> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>>
>> --===============1190338742==--
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jm...@co...
>
>
>
>
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>
>
>
>
> -- 
> Arlindo da Silva
> da...@al...

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

Re: [Opengrads-devel] Fonts in GrADS

[Arlindo da S. <arl...@na...>]

On Feb 2, 2008 6:04 PM, Brian Doty <do...@co...> wrote:

> Hi Matt, thanks for all the feedback on all that stuff, it is very
> helpful!  I am going to take a very close look at Cairo and probably
> hack a partially functional interface between it and gxX just to see
> how it behaves.  It looks to support a lot of the stuff we want to do
> so it'll be great if it performs as advertised.  It appears to be a
> long lived and stable project; is that your sense as well?  It'll
> probably take me a week or two to play with it and discover any
> pitfalls.  I'll keep you informed... Brian
>

Cairo uses FreeType underneath to render its fonts, so you would be using
FreeType if you go thru the cairo API, but I believe it has support for
native fonts as well, say on Windows and/or Macs. You may want to take a
look at gxyat.c

http://opengrads.cvs.sourceforge.net/opengrads/contrib/extensions/gxyat/gxyat.c?revision=1.2&view=markup

to see how I mapped the gx metadata into cairo calls; of course, no fonts at
this point. I have gxyat implemented as an external command line utility,
and the same code compiles into gxyat.gex.so and functions as a built-in
extension --- I ifdef the code using the same trick Matt used for in-lining
gxeps.

I have been tinkering with the idea of addiing an option to gxyat to render
the plot to the screen instead of to a file. This would provide a quick
prototype of how a cairo graphical backend would look like, and allow for
some experimentation before tackling the gxX.c problem.

Another thing to keep in mind is that the cairo library proper only handles
the "rendering" of vector graphics (it used to be called Xr when it lived
inside X), it does not attempt to solve the event driven mouse click/key
pressed kind of stuff that would be needed for a full gxX.c replacement.
Cairo's approach is to rely on a "backend" library for that. A common
backend is libsdl:

            http://www.libsdl.org/

>From their home page:

Simple DirectMedia Layer is a cross-platform multimedia library designed to
provide low level access to audio, keyboard, mouse, joystick, 3D hardware
via OpenGL, and 2D video framebuffer. It is used by MPEG playback software,
emulators, and many popular games, including the award winning Linux port of
"Civilization: Call To Power."

SDL supports Linux, Windows, Windows CE, BeOS, MacOS, Mac OS X, FreeBSD,
NetBSD, OpenBSD, BSD/OS, Solaris, IRIX, and QNX. The code contains support
for AmigaOS, Dreamcast, Atari, AIX, OSF/Tru64, RISC OS, SymbianOS, and OS/2,
but these are not officially supported.

SDL is written in C, but works with C++ natively, and has bindings to
several other languages, including Ada, C#, D, Eiffel, Erlang, Euphoria,
Guile, Haskell, Java, Lisp, Lua, ML, Objective C, Pascal, Perl, PHP, Pike,
Pliant, Python, Ruby, Smalltalk, and Tcl.

SDL is distributed under GNU LGPL version 2. This license allows you to use
SDL freely in commercial programs as long as you link with the dynamic
library.
---

  I haven't added cairo to the supplibs but it is on my to do list.

            Arlindo


>
> On Feb 2, 2008, at 1:07 AM, Matthias Münnich wrote:
>
> > Hi Brian
> >
> > Based on my experience, and your intention to give GrADS a major
> > overhaul, I would try out  Cairo.   The Freetype solution, at least
> > the one I started implementing,  would likely still appear like
> > patchwork and may  lead to more  maintanance trouble on the long
> > run.   I think, the basis of  a new implementation should be to
> > move away from the low-level "pin-up, pin-down" and introduce
> > higher order graphics elements  within the GrADS source to then
> > offload as much a possible to some graphics libraries like Cairo.
> >
> > That said, one advantage of going with FreeType would be that many
> > of the items you mention are actually already implemented in my
> > private version of GrADS 1.9.
> >
> >  - The metafile format was amended for strings, line-style, polygons
> >  - Freetype support was added to  gxhpng.c
> >  - String support was added to gxeps.c (these changes are actually
> > already in GrADS 1.9).
> >  - On-screen fonts are drawn with Freetype.
> >
> > You may like to  revisit the patches I sent to COLA a couple of
> > years ago. For your convenience I put a tar-file of my private
> > GrADS version at http://www.atmos.ucla.edu/~munnich/grads/<http://www.atmos.ucla.edu/%7Emunnich/grads/>
> > grads-1.9XP1mm3.tar.bz2    Go to the "oldformat" subdirectory for
> > diffs and have a look at  matts_cppflags for a summary of my
> > changes.  These changes are far from perfect. Here is a list of
> > deficiencies  off the top of my head:
> >
> >  1)  gxeps's BoundingBox comment is broken.  The main trouble here
> > is to get the extent of strings on the plot.
> > The implementation I came up with seems broken and, lacking time, I
> > began using ghostscript to get the BoundingBox.
> >
> > 2) Fonts are sometimes not displayed if one starts a session
> > remotely.  As far as I remember, FreeType  off-loads rendering to
> > the local X11 server, where FreeType may not be found.   It is
> > either this, or the event loop implementation in GrADS does not
> > play well with FreeType.
> >
> > 3) Clipping is a mess.   It is sufficiently differently done in X11
> > and PostScript to cause trouble.  I struggled with this and likely
> > implemented it badly.  Our graphics assistant at UCLA routinely
> > edits gxeps PS files using Adobe Illustrator.  However the tons of
> > clipping paths introduced by my gxeps version made it cumbersome
> > for her as she had to remove all of them by hand to before being
> > able to edit a figure.  Of course, this may be a problem of the EPS
> > importer of AI. Anyway,   I ended up creating a gxeps with disabled
> > clipping.  This meant that  strings ran out of the graphic boundary
> > but this was much easier for her to fix than dealing with the
> > clipping paths.
> >
> > 4) Subscripts and Superscripts are not working.
> >
> > Additional thoughts:
> >
> >  - Implementing clipping the way you once mentioned  -- introducing
> > a fine raster mask -- does not play nicely with
> > PostScript's idea of being resolution independent and may be
> > cumbersome to implement in gxeps.
> >
> >  - You should start supporting transparency (alpha-channel). In
> > this context  you may like to have a look  into the SVG  graphics
> > format.  It's a W3 standard and as such, all modern browsers can
> > render basic SVG files.  SVG supports alpha channel.  I always
> > thought I should write a gxsvg which should not be hard.
> >
> >  - Keep an eye on rendering speed.  At least the early versions of
> > Cairo (or was it Xft? ) I tried out  had issues  in this regard.
> >
> >  - Arlindo (cc)  asked me last May about my thoughts on
> > antialiasing and fonts in GrADS.  He came up with some solution.
> > You may like to contact him as well.  His knowledge should be more
> > up to date.   It's been a long time since I worked on this. Somehow
> > I mostly ended up thinking about a new contour filling routine,
> > when my mind drifted toward GrADS development.
> >
> >
> > ... Matt
> >
> >
> > On Jan 31, 2008, at 9:09 PM, Brian Doty wrote:
> >
> >> Hello Matt, I am beginning to think about the graphics layer of
> >> GrADS for v2.  There are a number of things on the list.
> >>
> >> One thing which we have discussed before is to put some sort of
> >> "native" font support into the package.  For screen output, I am
> >> looking at the Freetype library, and also the Cairo graphics
> >> library.  The Cairo project (formerly Xr/Xc) is of particular
> >> interest as it provides vector graphics primitives with a multi-
> >> platform backend.  This would allow for GrADS to be built native
> >> for Linux, Windows, and Mac, with no X server needed for the
> >> latter two.  But this is not something I have fully researched as
> >> yet.
> >>
> >> Anyway, the basic idea is to have a way to draw strings using
> >> raster fonts rather than the Hershey vector fonts.  The GRADS
> >> higher level utilities say they want to plot a string at a
> >> specific color, size, and rotation.  Also there is a call into the
> >> font library currently which provides back the actual length of
> >> the string to be drawn, thus taking into account proportional
> >> spacing.  For screen display we can use either X native or Cairo.
> >> For printim we can use either Freetype or Cairo.  My unknown is
> >> how to also support this via postscript output.  My knowledge
> >> there these days is limited.  I am also not sure if any of these
> >> things support rotated fonts.  If not then we just fall back to
> >> Hershey fonts when rotation is used.   GrADS puts it's contour
> >> labels horizontally by default so for most plots the rotation is
> >> not used.
> >>
> >> I am also planning to allow the "real" page size to be set on the
> >> fly.  This currently is hardwired as 8.5x11 or 11x8.5.  I want to
> >> allow this to be set arbitrarily, so the user can control the
> >> aspect ratio.  This is needed when the primary output is an image.
> >>
> >> Lastly, I would like to provide some ability to produce image
> >> output and somehow be able to print that to the postscript file.
> >> The overall design of this would depend on the hardcopy output
> >> limitations.
> >>
> >> Overall the goal continues to be to have a unified approach to the
> >> graphics such that something that can be displayed on the screen
> >> can also be printed via vector graphics with high quality
> >> reproduction.  Of course there are many things that can be done if
> >> one stays purely in the image realm, but I really want to maintain
> >> the vector hardcopy capability.  This may fade in importance with
> >> time, but for this round of planning I think it is still important
> >> enough.
> >>
> >> Needless to say many of these things create new entries in the
> >> graphics "metafile".  Thus gxeps will need to be extended.  I am
> >> planning on dropping the gv utility and probably also the old gxps.
> >>
> >> Please give me your thoughts, views, advice, etc!  Thanks... Brian
> >
> > Matthias Munnich
> > mu...@at...
> >
> >
> >
> >
>



-- 
Arlindo da Silva
NASA/GSFC Code 610.1
Tel: (301) 614-6174
Fax: (301 614-6297

Re: [Opengrads-devel] grads vs gradsdap in v2

[Arlindo da S. <da...@al...>]

On Feb 4, 2008 12:08 PM, Jennifer Adams <jm...@co...> wrote:

> In a couple of years the Unidate netcdf library will have swallowed up
> libnc-dap (see below),
>

That would be fantastic!


> so gradsdap is the one that will eventually be phased out. Until then, I
> feel it's better to link grads with the Unidata library and gradsdap with
> OPeNDAP version. Then when nc4 is ready (it's waiting for hdf5 to come out
> of beta status), we can link grads with nc4 library and leave gradsdap
> alone.
>

I have been experimenting with the current beta of netcdf4 in the context of
v1.9 (binary name: gradsnc4) and I'd say that, pending low level hdf-5
issues which are not visible at the API level, netcdf-4 is pretty ready;
gradsnc4 already passes 100% of our tests. I have both netcdf4 beta and the
hdf-5  version it requires in the supplibs. These will be replaced with the
final versions as they become available.  We have all the m4 discovery
macros for it already.

(The gradsnc4 binary is just as huge as gradsdap, as both libdap and hdf5
are bloated. Put these 2 together we will have a very heavy load, indeed. It
may be time to bring old gradsc back :-)


> I am trying to keep the data formats and their supplibs and their code
> paths in GrADS untangled. The HDF Group is also thinking about developing an
> OPeNDAP client library for HDF5, so that may be another complicating
> factor.
>

Speaking as a user, once an OPeNDAP enabled NetCDF-4 is available I see no
compeling reason for using HDF-5. I am trying to articulate this point
within NASA.

  Thank you for the info!

       Arlindo




> On Feb 4, 2008, at 11:40 AM, Arlindo da Silva wrote:
>
> Jennifer,
>
>   Does the executable "grads" have any feature not covered in "gradsdap"?
> If not, is "grads" really necessary? Sure, nc-dap is bloated, but nc-dap
> should be considered robust enough by now. You could have gone all the way
> and have a single executable called "grads", and only on those platforms
> where OPeNDAP cannot be build we build it with NetCDF. What were your
> thoughts for not having a single executable at this point?
>
>         Arlindo
>
> --
> Arlindo da Silva
> da...@al...
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
>
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>
> --===============1190338742==--
>
>
> --
> Jennifer M. Adams
> IGES/COLA
> 4041 Powder Mill Road, Suite 302
> Calverton, MD 20705
> jm...@co...
>
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>
>


-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] grads vs gradsdap in v2

[Jennifer A. <jm...@co...>]

In a couple of years the Unidate netcdf library will have swallowed  
up libnc-dap (see below), so gradsdap is the one that will eventually  
be phased out. Until then, I feel it's better to link grads with the  
Unidata library and gradsdap with OPeNDAP version. Then when nc4 is  
ready (it's waiting for hdf5 to come out of beta status), we can link  
grads with nc4 library and leave gradsdap alone. I am trying to keep  
the data formats and their supplibs and their code paths in GrADS  
untangled. The HDF Group is also thinking about developing an OPeNDAP  
client library for HDF5, so that may be another complicating factor.

Here's the announcement from James Gallagher (dated October 14th, 2007):

"We (OPeNDAP) are pleased to say that NSF has funded a
proposal for the netCDF client library software to migrate into the
Unidata code base for netCDF. This will entail rewriting the netCDF
client library to use the recently developed C API for DAP 2. The
effort will last two years and will take place both at OPeNDAP and
Unidata."


Jennifer

On Feb 4, 2008, at 11:40 AM, Arlindo da Silva wrote:

> Jennifer,
>
>   Does the executable "grads" have any feature not covered in  
> "gradsdap"? If not, is "grads" really necessary? Sure, nc-dap is  
> bloated, but nc-dap should be considered robust enough by now. You  
> could have gone all the way and have a single executable called  
> "grads", and only on those platforms where OPeNDAP cannot be build  
> we build it with NetCDF. What were your thoughts for not having a  
> single executable at this point?
>
>         Arlindo
>
> -- 
> Arlindo da Silva
> da...@al...
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ 
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>
> --===============1190338742==--

--
Jennifer M. Adams
IGES/COLA
4041 Powder Mill Road, Suite 302
Calverton, MD 20705
jm...@co...

[Opengrads-devel] grads vs gradsdap in v2

[Arlindo da S. <da...@al...>]

Jennifer,

  Does the executable "grads" have any feature not covered in "gradsdap"? If
not, is "grads" really necessary? Sure, nc-dap is bloated, but nc-dap should
be considered robust enough by now. You could have gone all the way and have
a single executable called "grads", and only on those platforms where
OPeNDAP cannot be build we build it with NetCDF. What were your thoughts for
not having a single executable at this point?

        Arlindo

-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] Fishy finite differencing

[Arlindo da S. <da...@al...>]

On Feb 4, 2008 9:22 AM, Michael Fiorino <Mic...@no...> wrote:

> hi arlindo,
>
> i'll take this on, for the lat-lon grids and if you can send me the refs
> and the algorithm, i'll give it a go.


Will do. Start mplementing the off-pole case which uses uncentered
differences. I'll send you later details about the polar cap case.


>  can you point me to the precise
> fish udx code i need to work?  thanks, also it seem more consistent to
> leave the gauss-spec version as a separte udx.
>

Start with the v1.9.0-rc1 code base:

% gacvs co -r grads-1_9_0-rc1 -P Grads   (notice capital "G")

Then look under extensions

% cd Grads/extensions
% cvs upd -A        (do this at this level the first time so it will be
easier to check in)
% cd fish

The files are:

fish.c --- fish() and the other functions are here, this is C.
fishpak.f --- the generic f77 eliptic solver
fish.udxt.in --- "function table", maps the function names as seen inside
grads
                        to the function names in fish.c; make sure you edit
fish.udxt.in and
                        not fish.udxt
ftn_fish.F --- front end for fishpak.F
GNUmakefile --- makefile
utFish.gs --- sample script, add more stuff as you implement more functions.

  Let me know if you have questions.

              Arlindo

-- 
Arlindo da Silva
da...@al...

Re: [Opengrads-devel] Fishy finite differencing

[Mike F. <mfi...@gm...>]

hi arlindo,

i'll take this on, for the lat-lon grids and if you can send me the refs
and the algorithm, i'll give it a go.  can you point me to the precise
fish udx code i need to work?  thanks, also it seem more consistent to
leave the gauss-spec version as a separte udx.

best /r mike

Arlindo da Silva wrote:
> Mike,
>
>   Or should I have said "finite differencing for fish()"?. (This is 
> intended primary for Mike; I'm cc'ing the list in case someone wants 
> to comment on the  algorithm.)
>
>    A little background. Fish() is a poisson solver that computes 
> streamfunction and velocity potential given vorticity and divervence:
>
>    psi = fish(vort)
>    chi = fish(divg)
>
> Unlike the classic UDF psichi,  I  have  separated  the computation of 
> vort/div from the possion solver proper. Like the classic psichi,  
> fish() uses fishpak.f, and old dusty-deck f77 eliptic solver that 
> stood the test of time. (Fishpak is not really f77 but rather f66, 
> with tons of goto's and arithmetic ifs, but it is very fast and the 
> numerics are solid.) Fishpak is freely available from netlib.
>
> Now, in principle vorticity and divergence could be computed with 
> grads intrinsics such as:
>
>   vort = hcurl(u,v)
>
> The problem is, hcurl() does not handles the boundaries very well, 
> placing UNDEFS in the first and last points (even if this is a global 
> periodic grid). Altough fish replaces undefs with 0. (like the classic 
> psichi), this is not the right thing to do. So, we need to provide a 
> replacement in the fish() package for properly handling the 
> boundaries, something that we could contribute later to COLA for 
> improving the current intrinsics.
>
> Here is my proposal for handling finite differencing at the boundaries:
>
> a) Zonal direction. If the grid is global, exploit the periodicidy, 
> and use centered differences for all points. If the grid (read: 
> dimension environment) is regional, use uncentered differences at the 
> boundaries.
>
> b) Meridional direction. If the grid (dimension environment) is 
> regional, use uncentered differences at the boundaries. Now, if the 
> grid is global, the handling of the extrema depends on whether the 
> pole is a grid point or not:
>
> i) Grid is off the pole by delta-lat/2. For example, on a 1 degree 
> grid the first and last gridpoints are -89.5 and 89.5. In this case 
> use uncentered differences at these points.
>
> ii) The poles are grid-points. This is the grid used by the 
> finite-volume dynamical core of S.J.-Lin used at GMAO, NCAR and GFDL.  
> The trick here is to consider these polar points as little polar caps 
> and use vector calculus to compute vorticity/divergence. Using Stokes' 
> theorem one can relate the mean vorticity in this polar cap to the 
> "circulation" at the boundaries of the polar cap, basically the sum of 
> "u"  at the boundary of the polar cap for some 2 pi factor thrown in 
> for good measure; for 1 deg grids, this boundary is at -89.5 and 89.5, 
> so "u" needs to be interpolated as the average of the first and second 
> meridion grid points (similarly for last and 1 from last gridpoints).  
> A similar algorithm is used for divergence, but now one relies on the 
> Gauss (divergence) theorem to relate the mean divergence at the polar 
> cap to the mass flux into the polar cap, basically the sum of "v" at 
> the boundary with some 2pi normalization factor.
>
> iii) Gaussian grids. The best solution would be to do a spectral 
> transform and compute vort/div that way.
>
> Poles are always messy, and usually a big source of trouble. So it 
> pays to be a bit careful.
>
> Mike: I added the stubs already to the fish() code base, perhaps more 
> than we need. I guess all that we need at this point is vort() and 
> div(). Could you add your code for the zonal direction? You can 
> implement the polar polar, and if you wish I can do it or write down 
> the algorithm more precisely (or send you the references.)
>
> I need to dig some spectral transform code for the gaussian grid case, 
> or else do a simple uncentered calculation  at the pole. If one is 
> going to bring spectral transforms in, then it is better to do the 
> calculation of psi/chi spectrally and not use fish() which is intended 
> for lat/lon grids. This may be another extension altogether.
>
> I'm cc'ing Brian in case he has any suggestions for us, as he may want 
> to incorporate some of this into hcur/divg later.
>
>     As always, any suggestions greatly appreciated.
>
>       Cheers!
>
>          Arlindo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> -- 
> Arlindo da Silva
> da...@al... <mailto:da...@al...>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> ------------------------------------------------------------------------
>
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>   

Re: [Opengrads-devel] Fishy finite differencing

[Michael F. <Mic...@no...>]

hi arlindo,

i'll take this on, for the lat-lon grids and if you can send me the refs 
and the algorithm, i'll give it a go.  can you point me to the precise 
fish udx code i need to work?  thanks, also it seem more consistent to 
leave the gauss-spec version as a separte udx. 

best /r mike

Arlindo da Silva wrote:
> Mike,
>
>   Or should I have said "finite differencing for fish()"?. (This is 
> intended primary for Mike; I'm cc'ing the list in case someone wants 
> to comment on the  algorithm.)
>
>    A little background. Fish() is a poisson solver that computes 
> streamfunction and velocity potential given vorticity and divervence:
>
>    psi = fish(vort)
>    chi = fish(divg)
>
> Unlike the classic UDF psichi,  I  have  separated  the computation of 
> vort/div from the possion solver proper. Like the classic psichi,  
> fish() uses fishpak.f, and old dusty-deck f77 eliptic solver that 
> stood the test of time. (Fishpak is not really f77 but rather f66, 
> with tons of goto's and arithmetic ifs, but it is very fast and the 
> numerics are solid.) Fishpak is freely available from netlib.
>
> Now, in principle vorticity and divergence could be computed with 
> grads intrinsics such as:
>
>   vort = hcurl(u,v)
>
> The problem is, hcurl() does not handles the boundaries very well, 
> placing UNDEFS in the first and last points (even if this is a global 
> periodic grid). Altough fish replaces undefs with 0. (like the classic 
> psichi), this is not the right thing to do. So, we need to provide a 
> replacement in the fish() package for properly handling the 
> boundaries, something that we could contribute later to COLA for 
> improving the current intrinsics.
>
> Here is my proposal for handling finite differencing at the boundaries:
>
> a) Zonal direction. If the grid is global, exploit the periodicidy, 
> and use centered differences for all points. If the grid (read: 
> dimension environment) is regional, use uncentered differences at the 
> boundaries.
>
> b) Meridional direction. If the grid (dimension environment) is 
> regional, use uncentered differences at the boundaries. Now, if the 
> grid is global, the handling of the extrema depends on whether the 
> pole is a grid point or not:
>
> i) Grid is off the pole by delta-lat/2. For example, on a 1 degree 
> grid the first and last gridpoints are -89.5 and 89.5. In this case 
> use uncentered differences at these points.
>
> ii) The poles are grid-points. This is the grid used by the 
> finite-volume dynamical core of S.J.-Lin used at GMAO, NCAR and GFDL.  
> The trick here is to consider these polar points as little polar caps 
> and use vector calculus to compute vorticity/divergence. Using Stokes' 
> theorem one can relate the mean vorticity in this polar cap to the 
> "circulation" at the boundaries of the polar cap, basically the sum of 
> "u"  at the boundary of the polar cap for some 2 pi factor thrown in 
> for good measure; for 1 deg grids, this boundary is at -89.5 and 89.5, 
> so "u" needs to be interpolated as the average of the first and second 
> meridion grid points (similarly for last and 1 from last gridpoints).  
> A similar algorithm is used for divergence, but now one relies on the 
> Gauss (divergence) theorem to relate the mean divergence at the polar 
> cap to the mass flux into the polar cap, basically the sum of "v" at 
> the boundary with some 2pi normalization factor.
>
> iii) Gaussian grids. The best solution would be to do a spectral 
> transform and compute vort/div that way.
>
> Poles are always messy, and usually a big source of trouble. So it 
> pays to be a bit careful.
>
> Mike: I added the stubs already to the fish() code base, perhaps more 
> than we need. I guess all that we need at this point is vort() and 
> div(). Could you add your code for the zonal direction? You can 
> implement the polar polar, and if you wish I can do it or write down 
> the algorithm more precisely (or send you the references.)
>
> I need to dig some spectral transform code for the gaussian grid case, 
> or else do a simple uncentered calculation  at the pole. If one is 
> going to bring spectral transforms in, then it is better to do the 
> calculation of psi/chi spectrally and not use fish() which is intended 
> for lat/lon grids. This may be another extension altogether.
>
> I'm cc'ing Brian in case he has any suggestions for us, as he may want 
> to incorporate some of this into hcur/divg later.
>
>     As always, any suggestions greatly appreciated.
>
>       Cheers!
>
>          Arlindo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> -- 
> Arlindo da Silva
> da...@al... <mailto:da...@al...>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> ------------------------------------------------------------------------
>
> _______________________________________________
> Opengrads-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opengrads-devel
>   

[Opengrads-devel] Fishy finite differencing

[Arlindo da S. <da...@al...>]

Mike,

  Or should I have said "finite differencing for fish()"?. (This is intended
primary for Mike; I'm cc'ing the list in case someone wants to comment on
the  algorithm.)

   A little background. Fish() is a poisson solver that computes
streamfunction and velocity potential given vorticity and divervence:

   psi = fish(vort)
   chi = fish(divg)

Unlike the classic UDF psichi,  I  have  separated  the computation of
vort/div from the possion solver proper. Like the classic psichi,  fish()
uses fishpak.f, and old dusty-deck f77 eliptic solver that stood the test of
time. (Fishpak is not really f77 but rather f66, with tons of goto's and
arithmetic ifs, but it is very fast and the numerics are solid.) Fishpak is
freely available from netlib.

Now, in principle vorticity and divergence could be computed with grads
intrinsics such as:

  vort = hcurl(u,v)

The problem is, hcurl() does not handles the boundaries very well, placing
UNDEFS in the first and last points (even if this is a global periodic
grid). Altough fish replaces undefs with 0. (like the classic psichi), this
is not the right thing to do. So, we need to provide a replacement in the
fish() package for properly handling the boundaries, something that we could
contribute later to COLA for improving the current intrinsics.

Here is my proposal for handling finite differencing at the boundaries:

a) Zonal direction. If the grid is global, exploit the periodicidy, and use
centered differences for all points. If the grid (read: dimension
environment) is regional, use uncentered differences at the boundaries.

b) Meridional direction. If the grid (dimension environment) is regional,
use uncentered differences at the boundaries. Now, if the grid is global,
the handling of the extrema depends on whether the pole is a grid point or
not:

i) Grid is off the pole by delta-lat/2. For example, on a 1 degree grid the
first and last gridpoints are -89.5 and 89.5. In this case use uncentered
differences at these points.

ii) The poles are grid-points. This is the grid used by the finite-volume
dynamical core of S.J.-Lin used at GMAO, NCAR and GFDL.  The trick here is
to consider these polar points as little polar caps and use vector calculus
to compute vorticity/divergence. Using Stokes' theorem one can relate the
mean vorticity in this polar cap to the "circulation" at the boundaries of
the polar cap, basically the sum of "u"  at the boundary of the polar cap
for some 2 pi factor thrown in for good measure; for 1 deg grids, this
boundary is at -89.5 and 89.5, so "u" needs to be interpolated as the
average of the first and second meridion grid points (similarly for last and
1 from last gridpoints).  A similar algorithm is used for divergence, but
now one relies on the Gauss (divergence) theorem to relate the mean
divergence at the polar cap to the mass flux into the polar cap, basically
the sum of "v" at the boundary with some 2pi normalization factor.

iii) Gaussian grids. The best solution would be to do a spectral transform
and compute vort/div that way.

Poles are always messy, and usually a big source of trouble. So it pays to
be a bit careful.

Mike: I added the stubs already to the fish() code base, perhaps more than
we need. I guess all that we need at this point is vort() and div(). Could
you add your code for the zonal direction? You can implement the polar
polar, and if you wish I can do it or write down the algorithm more
precisely (or send you the references.)

I need to dig some spectral transform code for the gaussian grid case, or
else do a simple uncentered calculation  at the pole. If one is going to
bring spectral transforms in, then it is better to do the calculation of
psi/chi spectrally and not use fish() which is intended for lat/lon grids.
This may be another extension altogether.

I'm cc'ing Brian in case he has any suggestions for us, as he may want to
incorporate some of this into hcur/divg later.

    As always, any suggestions greatly appreciated.

      Cheers!

         Arlindo
















-- 
Arlindo da Silva
da...@al...