#906 Template instances generate incorrect C++

[Göran Uddeborg]

Description

When generating interface for an instance of for example list<...>, the resulting C++ code fails to compile on AIX. The reason is that a member function asptr of a class template traits_asptr_stdseq calls SWIG_Python_ConvertPtrAndOwn. The latter function is static. And according to section 14.6.4.2 the standard, only functions with external linkage are considered for lookup of unqualified names.

To repeat

1. Create a file y.i with this content:

%module y
%include "std_list.i"
%template(IntList) std::list<int>;

2. Process with swig:

swig -c++ -python y.i

3. Compile with the AIX compiler:

xlC -qrtti=all -I /usr/local/include/python2.4 -c y_wrap.cxx

The compiler gives the following error messages:

"y_wrap.cxx", line 3898.13: 1540-0274 (S) The name lookup for "SWIG_Python_ConvertPtrAndOwn" did not find a declaration.
"y_wrap.cxx", line 3898.13: 1540-1292 (I) Static declarations are not considered for a function call if the function is not qualified.
"y_wrap.cxx", line 3895.16: 1540-0700 (I) The previous message was produced while processing "swig::traits_asptr_stdseq<std::list<int,std::allocator<int> >,int>::asptr(PyObject *, sequence **)".
"y_wrap.cxx", line 3963.16: 1540-0700 (I) The previous message was produced while processing "swig::traits_asptr<std::list<int,std::allocator<int> > >::asptr(PyObject *, std::list<int,std::allocator<int> > **)".
"y_wrap.cxx", line 3069.14: 1540-0700 (I) The previous message was produced while processing "swig::asptr<std::list<int,std::allocator<int> > >(PyObject *, list<int,std::allocator<int> > **)".
"y_wrap.cxx", line 5288.12: 1540-0700 (I) The previous message was produced while processing "_wrap_new_IntList__SWIG_1(PyObject *, PyObject *)".

Notes

The AIX compiler is the only one I've seen actually enforcing the restriction en section 14.6.4.2.

One way to solve the problem would be to make the name qualified by simply prepending :: to the name. See the attachement for the modifications I made to 1.3.34 to make this work for Python. Presumably, similar changes will need to be done for other examples and other target languages if you choose this solution.

[Göran Uddeborg]

Patch to make the test case work.

[Olly Betts]

Logged In: YES
user_id=14972
Originator: NO

This seems to be the relevant GCC bug:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19092

This says there's an open defect for this, so the bug is suspended.

But anyway, this patch looks good to me, as it's simple and should be harmless for compilers which currently handle the code.

Could you try running the testsuite with this compiler to see if we can automatically catch at least some other instances of this?

make check-test-suite
make check-examples

[Göran Uddeborg]

Output of test results

[Göran Uddeborg]

Logged In: YES
user_id=55884
Originator: YES

After a bit of tweaking flags and such I've been able to run the perl, python and guile tests. That's the targets we have installed. I BELIEVE I've solved things that depend on local configurations, and that the remaining issues are because of real bugs in the Swig code and/or bugs in the compiler.

But I haven't analysed them very much yet, and it might be a while before I have time to do that. In case anyone wants to have a look in advance, I thought I could attach the output of the runs.
File Added: swigtest.tar.gz

[Göran Uddeborg]

Logged In: YES
user_id=55884
Originator: YES

I've looked at the first two errors.

The first one seems to be a bug in the AIX compiler. I've sent a report to IBM.

The second one is namespace pollution. When you include EXTERN.h and perl.h, (using the IBM perl package perl.rte) the symbol BSIZE gets defined to 4096. You could argue who's at fault here, but the simplest probably is to change the names used in Examples/test-suite/arrays_scope.i.

[Olly Betts]

Logged In: YES
user_id=14972
Originator: NO

I've committed a workaround for the BSIZE issue (renamed it to BBSIZE). Interestingly, the reason for CCSIZE rather than CSIZE in that file is due to a clash on OS X, according to the SVN logs. Looking at the Perl 5.8.8 headers I have here, there's no trace of BSIZE, so I guess this must be from system headers, or else you have a different Perl version...

Did you need to patch to get the perl and guile tests to run?

[Olly Betts]

Logged In: YES
user_id=14972
Originator: NO

I've now applied the "swigdiff" patch to SVN. If you find further issues, please attach further patches. If everything seems to be working, please close this bug.

[Göran Uddeborg]

Diffs done in some makefiles

[Göran Uddeborg]

Logged In: YES
user_id=55884
Originator: YES

> Did you need to patch to get the perl and guile tests to run?

I did some tweaks. I wasn't focusing on getting this part correct, so I did do some changes directly in the generated Makefiles. See the attached diff for details.

1. I used these options to configure:

./configure CC=xlc_r CFLAGS=-D_ALL_SOURCE CXX=xlC_r 'CXXSHARED=makeC++SharedLib_r -p 0' CXXFLAGS=-D_ALL_SOURCE

2. I added the compiler flag -qrtti to get run time type information in the tests.

3. I used "$(CC) -G" rather than "$(srcdir)/ld_so_aix $(CC)" to generate shared libraries. There is no $(srcdir)/ld_so_aix.

4. I commented out the second CXXSHARED in Examples/Makefile so the one I gave on the command line would have effect.

5. I added *_DLNK flags for all languages I tested, see the diff for details.

6. I added an explicit reference to our locally installed boost header files in Examples/test-suite/common.mk
File Added: swig-config.diff

[Göran Uddeborg]

Logged In: YES
user_id=55884
Originator: YES

> If you find further issues, please attach further patches. If everything seems to be working, please
> close this bug.

My plan is to go through all the issues in the log file. It will be a while before I've found the time for them all.

If you like to keep things separate we can close this bug, and I can open a separate for each case. Or I can add to this one as I come to them. It doesn't matter to me, I'll let you decide.

[Göran Uddeborg]

Logged In: YES
user_id=55884
Originator: YES

I investigated one more case. AIX does not seem to define PTHREAD_MUTEX_RECURSIVE_NP. It does, however, define PTHREAD_MUTEX_RECURSIVE, which also is the name Posix specifies.

I removed the _NP and then the li_boost_shared_ptr test succeeded for all of perl5, python, and guile. (I attach the obvious patch.) But maybe that would break some other system?
File Added: swigdiff.2

[Göran Uddeborg]

Patch removing _NP from PTHREAD_MUTEX_RECURSIVE_NP

[Göran Uddeborg]

Logged In: YES
user_id=55884
Originator: YES

Two more problems: The perl and python header files pulls in other header files that define int8 et.al and M_PI. So when these are defined in li_windows and member_pointer respectively, there is a compilation error.

In the case of M_PI we are talking about a definition in the preprocessor sense, so it can be guarded as shown in the attached swigdiff.3. uint8 et.al are typedef definitions. In that case I suppose you either have to change the name, or to do some kind of platform checking and avoid including these typedefs when they are already done in standard header files.
File Added: swigdiff.3

[Göran Uddeborg]

Ifdef guards around M_PI

[Olly Betts]

Just noticed this was still assigned to me in the old SF tracker which we've stopped actively using.

Fortunately I think almost everything here was actually dealt with years ago:

swigdiff.2 was addressed not long after by f70f2760e3baf740cc825a72303070d15b640436

swigdiff.3 was actually addressed a month before you posted it by ec4491556fd698d0e973c47586dc76cee85c7b68

Most of swig-config.diff looks like it can be achieved by passing suitable arguments to configure.

Someone's opened a number of issues in the github tracker for AIX this year:

https://github.com/swig/swig/issues?q=is%3Aissue+is%3Aopen+aix

These include the non-existent ld_so_aix script.

I don't have access to AIX so can't check if there's anything still outstanding, but given it's been 13 years and someone's clearly trying to sort out problems with a current AIX version, I think it's appropriate to close this. But if you're still hitting issues, please open a new ticket (or tickets) in the github tracker.

[Göran Uddeborg]

It's not a problem for us any more. We used any necessary workarounds until we stopped supporting AIX some time ago. So I'm fine with closing it.

Some issues stay open longer than other! :-)