[Tcl-bugs] [ tcl-Bugs-3401704 ] expr parses numbers before function names

[SourceForge.net <no...@so...>]

Bugs item #3401704, was opened at 2011-08-31 18:22
Message generated for change (Comment added) made by ferrieux
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3401704&group_id=10894

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: 45. Parsing and Eval
Group: current: 8.5.10
Status: Open
Resolution: None
Priority: 8
Private: No
Submitted By: Will Duquette (duquette)
Assigned to: Don Porter (dgp)
Summary: expr parses numbers before function names

Initial Comment:
You cannot define expr functions with names that begin with numbers (including "inf" and "nan").  The expression parser looks for numbers before looking for function names.  For example,

    expr {2*nancy(5)}

will throw the following error:

    missing operator at _@_ 
    in expression "nan_@_cy(5)"

Function names beginning with a digit have the same problem.

Note: it doesn't matter whether the tcl::mathfunc::nancy command is actually defined or not.


----------------------------------------------------------------------

>Comment By: Alexandre Ferrieux (ferrieux)
Date: 2011-08-31 23:35

Message:
I went through all call sites of TclParseNumber (in the core; not bothered
about extensions).
What I gather from this is:
  - there are many call sites, so fixing them would be
tricky/unmaintainable
  - fixing the callee only improves existing error cases


----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-08-31 23:33

Message:

Just for the record, this issue is not a regression.

% info patch
7.6p2
% expr {nancy(2)}
unknown math function "cy"
% expr {influence(1.2)}
unknown math function "luence"

similar results all along the way.

----------------------------------------------------------------------

Comment By: Alexandre Ferrieux (ferrieux)
Date: 2011-08-31 23:13

Message:
As another example:

 string is double -failindex x nancy

formerly returned 3 in $x, now zero. Same remark.
Do you have counter examples ? A true regression ?

----------------------------------------------------------------------

Comment By: Alexandre Ferrieux (ferrieux)
Date: 2011-08-31 21:41

Message:
With the patch this becomes:

  % expr {infeqinf}
  invalid bareword "infeqinf"
  in expression "infeqinf";

do you prefer the old behavior ? FWIW I don't :}

----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-08-31 20:49

Message:

Yes but TclParseNumber has other callers too.

FWIW, 8.6b2 produces this:
% expr {1eq1}
1
% expr {infeqinf}
invalid bareword "eqinf"
in expression "inf_@_eqinf";
should be "$eqinf" or "{eqinf}" or "eqinf(...)" or ...
% expr {infeq inf}
1




----------------------------------------------------------------------

Comment By: Alexandre Ferrieux (ferrieux)
Date: 2011-08-31 20:43

Message:
Yes. Rationale: ParseLexeme uses TclParseNumber pretty early, as a
tokenization first pass. So if I'd fixed the caller, I'd have gotten
"inbalid bareword 'cy'". So I had to fix the callee.

----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-08-31 20:39

Message:

Oh.  You revised TclParseNumber()? Not at
all what I was expecting.  My unexamined opinion
is that the flaw is in the parsing of expressions, not
numbers.  Will look it over though.


----------------------------------------------------------------------

Comment By: Alexandre Ferrieux (ferrieux)
Date: 2011-08-31 20:27

Message:
Branch bug-3401704 now has the cleaner 'isalnum' predicate instead of
hard-coded tests.
Also includes a new test case expr-22.10
Removing attached patch now out of sync.

----------------------------------------------------------------------

Comment By: Alexandre Ferrieux (ferrieux)
Date: 2011-08-31 19:50

Message:
Done :)

----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-08-31 19:30

Message:

Can you commit the patch to a branch bug-3401704
for more convenient review please?  Thanks!


----------------------------------------------------------------------

Comment By: Alexandre Ferrieux (ferrieux)
Date: 2011-08-31 19:15

Message:
It looks like TclParseNumber tries to mimic strtod. Quoting the manpage:

 The  strtod(),  strtof(),  and  strtold() functions convert the initial
 portion of the string pointed to by nptr to  double,  float,  and  long
 double representation, respectively.

The attached patch fixes the issue by detecting trailers after Nan and
Inf[inity].
Trailers currently defined as [0-9A-Za-z_], should be generalized (is
there a function for that ?).

In case they are detected, the number parser bails out, and [expr] reports
the correct "invalid bareword nancy" or "invalid command name
::tcl::mathfunc::nancy depending on the context. 




----------------------------------------------------------------------

Comment By: Will Duquette (duquette)
Date: 2011-08-31 18:22

Message:
Gave this a high priority, per DKF on c.l.t.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3401704&group_id=10894