ncap_initial_scan

[henry Butowsky]

Hi Charlie, I thought up a method of how we can get rid of the truly horrible ncap_initial scan.  What we can do is enable ncap_var_init() to create the dimensions on the FLY in the output file. When the parser is complete it should return to ncap.c :-
a) A list of the new attributes to be written.
b) A list of the dimensions defined.

what happens next depends on the -v flag ( PROCESS_ALL-VARS)

1) -v false
Need to add vars from the input file which correspond to any attribute "orphans" .
Add co-ordinate vars if necessary ( -c and -C ) flags
Write the new attributes

2) -v true
Add all the non-processed vars
Add co-ordinate vars (-c and -C)
write the new attributes.

There is a subtle problem that arises with this method.
If the user redefines a variable in the script file , and this redefinition fails then if (-v true)  then the original variable will appear in the output file.

Implementing 1) and 2) could get quite messy but I think it will still be much more clear than the existing code. Also it enables us  at a future date to allow new dimension definition in the script file.

FUNCTIONS
--------

I see you have done a lot of work getting the right maths functions for the corresponding machine architectures. What about having an addition option e.g -f that prints out the symbol table sym_tbl ?

[Charlie Zender]

Hi Henry,

All this sounds great and I'm glad to hear from you again.
I'm just running out the door for four days of camping so
can't respond point by point. Being able to add new
dimensions in output file is very important. Please proceed
if you are interested and I'll comment more when I return.

Charlie

[Nobody/Anonymous]

Hi Henry,

I'm back from my trip and have re-read this message.
I really like all the ideas you raise:

>There is a subtle problem that arises with this method.
>If the user redefines a variable in the script file , and >this redefinition fails then if (-v true) then the original >variable will appear in the output file.

Something that has been bothering me for a short
while is that ncap is too-fault-tolerant. It was useful
to have ncap ignore errors initially, but now that
it is more stable it should, in my opinion, die horribly
when something illegal is attempted. That means
the handling of symbols needs to be tighted up a bit,
and the error handling routine and eprovoke_skip
(which is stuff I have not look at too carefully) should
also be tightened up. In the end, there should never
be a case where a definition of a variable fails and
the script continues. ncap should just print the line
number of the script where it fails and exit.
Do you think you can make those changes?
Then the initial_scan() replacement
would not have to worry so much about pathological
cases.

>What about having an addition option e.g -f that >prints out the symbol table sym_tbl ?

Good idea! I would welcome that. Also, the code
which handles the symbols needs a re-write.
Right now one must manually count the symbols
that exist, etc., whenever adding a new symbol.
We need a mechanism to register new symbols
that does automatic symbol table allocation and
keeps track of number and name of symbols.

The float-valued symbols can be a real PITA,
because, as implemented, they prevent some
valid symbols from being defined. It should be possible to register double symbols
that do not have a float version, but right now that
does not seem to be possible.  Any work on this
issue would be greatly appreciated too!

Thanks,
Charlie

[henry Butowsky]

Hi Charlie,
I have done some work on dynamic dimensions and commited ncap.c,ncap.h ,ncap_utl.c ncap_lex.l .   Its still a bit buggy and ncap.c needs a good clean up -- There is a curious bug
ncap -s lon@test=9999 in.nc foo2.nc works but
ncap -s lon@test=9999 -D 3 in.nc foo2.nc dies coz of some problems with the printing of some messages .

The way is now open for user defined dims -- also we will able to  treat 1 -dimensional attributes as variables --

I'm still doing some tidy ups so let me know if your gonna do a major commit with the ncap files

regards Henry

[Nobody/Anonymous]

Hi Henry,

Glad you've chosen to tackle such a cool feature...

I have no plans to work on ncap for a while so go ahead and make major changes. I glanced at the code and clearly you've already done that!
Note that we're using ncap for data reduction here
and it's working great.

You'll notice I spent a lot of time cleaning
up the code in the last few months.
Naturally I don't want to do that again!
Since you are making great changes,
it would be best if you followed these guidelines
(at least once the dust has settled):

1. Use doc/ChangeLog to describe what you've just commited.
2. If you add user-visible features, also add a
corresponding regression test to ncap.in that
demonstrates its use. We want to make sure
that new features keep working and do not break
old features. In that sense ncap.in is not quite
robust because it does not ring alarm bells when
something does not behave as expected.
That's one reason I want to make stricter error
checking in ncap. Ideally we do not check in
code that does not pass the regression test.
3. Update nco.texi with new features, too.
You'll see the ncap section is there and not
quite complete but quite credible.
4. Clean up your code, dude! Use consistent
indentation with what Emacs gives, if possible.
Put a short /* [type] Descritpion */ comment after
new variable declarations. There are now so many
lists and counters that my head is spinning!
When we make mods, I want them to be
readable and understandable by others so we
can appreciate the fine hackery.

Thanks for contributing so much!
Charlie