[Swig-devel] [ swig-Patches-3468362 ] SWIG preprocessor bug: %include nested in %defined macros

[SourceForge.net <no...@so...>]

Patches item #3468362, was opened at 2012-01-02 02:56
Message generated for change (Comment added) made by wsfulton
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=301645&aid=3468362&group_id=1645

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: Karl Wette (kwwette)
Assigned to: William Fulton (wsfulton)
Summary: SWIG preprocessor bug: %include nested in %defined macros

Initial Comment:
The patch intends to address the following issue with the SWIG preprocessor: when an %include directive is nested within more than one macro defined with %define, the included file is parsed by the preprocessor more than once. For example:

$ cat header.h 
#define A 1
#define B 2
#define C 3
$ cat test.i 
%module test

%define nested1()
%include <header.h>
%enddef

%define nested2()
nested1();
%enddef

#if defined(NESTED1)
nested1();
#elif defined(NESTED2)
nested2();
#endif
$ swig -E -DNESTED1 test.i | grep . | tail -10
%includefile(maininput="test.i") "test.i" [
%module test
/*@SWIG:test.i,3,nested1@*/
%includefile "header.h" [
%constant A = 1;
%constant B = 2;
%constant C = 3;
]
/*@SWIG@*/;
]
$ swig -E -DNESTED2 test.i | grep . | tail -12
%includefile(maininput="test.i") "test.i" [
%module test
/*@SWIG:test.i,7,nested2@*/
/*@SWIG:test.i,3,nested1@*/
%includefile "header.h" [
%constant 1 = 1;
%constant 2 = 2;
%constant 3 = 3;
]
/*@SWIG@*/;
/*@SWIG@*/;
]

In the second case, when SWIG is run with -DNESTED2, the included file header.h is parsed by the SWIG preprocessor twice - once when the file is included when nested1() is expanded, and once when the included file contents is parsed again when nested2() is expanded. As a result, the names of the #defined constants A, B, and C are replaced with their values, which have already been defined as preprocessor symbols, and invalid code is generated.

This patches replaces the [ and ] brackets used by the internal %includefile directive with two new internal directives: %beginfile and %endoffile, e.g.

%includefile "header.h" %beginfile
...
%endoffile

These directives are recognised as balanced tokens by the preprocessor, in a manner similar to how %define and %enddef are parsed. When encountered, all containing text is skipped by the preprocessor. The directives are then recognised by the SWIG parser's "include_directive" rule. With this patch applied, the result of the second test above is now:

$ swig -E -DNESTED2 test.i | grep . | tail -12
%includefile(maininput="test.i") "test.i" %beginfile
%module test
/*@SWIG:test.i,7,nested2@*/
/*@SWIG:test.i,3,nested1@*/
%includefile "header.h" %beginfile
%constant A = 1;
%constant B = 2;
%constant C = 3;
%endoffile
/*@SWIG@*/;
/*@SWIG@*/;
%endoffile

i.e. the names of the constants A, B, and C are now correctly preserved.

File changed:
 * Source/Preprocessor/cpp.c: modifies preprocessor to generate and parse new %beginfile and %endoffile directives.
 * Source/CParse/cscanner.c: recognises new directives in scanner.
 * Source/CParse/parser.y: adds new directives as tokens, and modifies definition of "include_directive" rule.
 * Examples/test-suite/preproc_include.i: adds new tests of using %include inside nested %defined macros.
 * Examples/test-suite/preproc_include_h{1,2,3}.i: new files for preprocessing test.


----------------------------------------------------------------------

>Comment By: William Fulton (wsfulton)
Date: 2012-03-15 13:33

Message:
I'm not entirely convinced we should be supporting %include within a
%define macro at all given the C preprocessor does not support a #include
within a #define macro. Nevertheless, I've committed your patch. Thanks for
a nice complete patch, test cases and all. 

----------------------------------------------------------------------

Comment By: Karl Wette (kwwette)
Date: 2012-03-13 04:11

Message:
I chose %beginfile and %endoffile because they have the same number of
characters. That way I could base the preprocessor parsing code (cpp.c,
line 1922) on the equivalent code for %define and %enddef, which seems to
rely on the identifiers having the same length. I thought it would be
better to try to reuse existing preprocessor code that trying to write my
potentially buggy version.

----------------------------------------------------------------------

Comment By: William Fulton (wsfulton)
Date: 2012-03-13 00:36

Message:
Certainly looks like a valid problem. Am still looking, is there any reason
why the naming isn't more consistent and the new directives instead being
called %beginoffile and %endoffile or %beginfile and %endfile ?



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=301645&aid=3468362&group_id=1645