#162 Handling of hex literals in .def files

autogen
pending
Bruce Korb
None
1
2014-06-30
2014-06-26
Erik
No

I've been using autogen for about ten years as part of the build chain for my project libsndfile. Now, after all this time I must report that I have run into a problem. My definition file contains hexadecimal literals. See the files:

https://github.com/erikd/libsndfile/blob/master/tests/floating_point_test.def
https://github.com/erikd/libsndfile/blob/master/tests/floating_point_test.tpl

Running autogen as:

autogen --writable floating_point_test.def

I get the error:

FSM Error:  in state 10 (have_value), event 3 (var_name) is invalid
invalid transition:  in scale_clip_test.def on line 22
    token in error:  var_name:  ''x7FFFF ;
    } ;

I'm using autogen 5.18.3 from Debian testing on an amd64 machine.

Any idea what has changed? Is there a workaround or some other fix? Is this actually a bug?

Discussion

  • Erik
    Erik
    2014-06-27

    It seems Debian went from 5.12 (which was working for me) to 5.18.3 which has the problem described above.

     
  • Erik
    Erik
    2014-06-27

    I just compiled 5.18.1 from source and it works, while 5.18.2 (also compiled from source) doesn't.

     
    Last edit: Erik 2014-06-27
  • Bruce Korb
    Bruce Korb
    2014-06-28

    I am surprised it ever worked. The scanner sees a digit (a zero) and says, "I'm processing a number token, so as long as I see decimal digits, I accept the next character." Since it has intended to be that way since forever, I am surprised it ever worked.

    e33cd672 (bkorb      2005-01-08 22:56:19 +0000 568) static char*
    4e40be23 (Bruce Korb 2013-10-12 13:54:25 -0700 569) gather_name(char * scan, te_dp_event * ret_val)
    de4b4d61 (bruce      1999-10-14 00:33:53 +0000 570) {
    de4b4d61 (bruce      1999-10-14 00:33:53 +0000 571)     /*
    de4b4d61 (bruce      1999-10-14 00:33:53 +0000 572)      *  Check for a number.
    4e40be23 (Bruce Korb 2013-10-12 13:54:25 -0700 573)      *  Scan it in and advance "scan".
    de4b4d61 (bruce      1999-10-14 00:33:53 +0000 574)      */
    4e40be23 (Bruce Korb 2013-10-12 13:54:25 -0700 575)     if (  IS_DEC_DIGIT_CHAR(*scan)
    4e40be23 (Bruce Korb 2013-10-12 13:54:25 -0700 576)        || ((*scan == '-') && IS_DEC_DIGIT_CHAR(scan[1])) ) {
    4e40be23 (Bruce Korb 2013-10-12 13:54:25 -0700 577)         token_str = scan;
    4e40be23 (Bruce Korb 2013-10-12 13:54:25 -0700 578)         *ret_val  = DP_EV_NUMBER;
    4e40be23 (Bruce Korb 2013-10-12 13:54:25 -0700 579)         return SPN_DEC_DIGIT_CHARS(scan + 1);
    de4b4d61 (bruce      1999-10-14 00:33:53 +0000 580)     }
    

    It looks like I fiddled the code last fall, but I didn't intentionally change the functioning. That it worked before was a fluke. But accepting a hex number is reasonable enough, so before I detect a number, I'll look for a hex number. Anyway, just quote it:

    int-max = '0x7FFFF';
    
     
  • Bruce Korb
    Bruce Korb
    2014-06-28

    • summary: Regression: Handling of hex literals in .def files --> Handling of hex literals in .def files
    • assigned_to: Bruce Korb
     
  • Bruce Korb
    Bruce Korb
    2014-06-28

    The following patch is being applied to defLex.c:

    @@ -569,13 +569,19 @@ static char*
     gather_name(char * scan, te_dp_event * ret_val)
     {
         /*
    -     *  Check for a number.
    -     *  Scan it in and advance "scan".
    +     *  Check for a number.  We don't care about leading zeros, that
    +     *  is a user's problem, so no distinction between octal and decimal.
    +     *  However, we look for "0x" and "0X" prefixes for hex numbers.
          */
         if (  IS_DEC_DIGIT_CHAR(*scan)
            || ((*scan == '-') && IS_DEC_DIGIT_CHAR(scan[1])) ) {
             token_str = scan;
             *ret_val  = DP_EV_NUMBER;
    +
    +        if (  (scan[0] == '0')
    +           && ((scan[1] == 'x') || (scan[1] == 'X')) )
    +            return SPN_HEX_DIGIT_CHARS(scan + 2);
    +
             return SPN_DEC_DIGIT_CHARS(scan + 1);
         }
    
     
  • Bruce Korb
    Bruce Korb
    2014-06-28

    • status: open --> pending
     
  • Erik
    Erik
    2014-06-30

    You're right, if I enclose the hex literals in quotes and it works with 5.18.3.

     
  • Bruce Korb
    Bruce Korb
    2014-06-30

    I still fixed it so it will work again in 5.18.4. It seems I used to use strtoul() and discarded the result to scan across the text. I changed it to use the SPN_DEC_DIGIT_CHARS() skip-over-digits macro function. "Sorry."