#978 Allow automatic %template injection when reading header file

[James Masters]

Currently %template declarations must be made *after* reading in header files. Additionally, %template declarations must be made *before* their usage as a base class. For example, to wrap the three classes below (ArrayBase<int>, Array<int>, IntArray) *with* proper inheritance, I would need the following %template declaration positioning:

template <class T> class ArrayBase {};
template <class T> class Array : public ArrayBase<T> {};
%template("IArrayBase") ArrayBase<int>;
%template("IArray") Array<int>;
class IntArray : public Array<int> {};

Unfortunately, if the header files cannot be edited directly, then a copy of the header files is necessary and manual injection (or through a script) of %template is needed to conform with this precedence requirement (for details see: http://www.nabble.com/Is-there-a-workaround-to-use--template-after-reading-all-header-files--td20861858.html\)

A possible solution would be to somehow allow SWIG to inject %template into the header files automatically during preprocessing immediately after a template is defined - perhaps by some kind of SWIG interface macro pre-declaration before reading header files.

[David Piepgrass]

I agree with this suggestion. One of the difficulties for users using SWIG is getting declarations in the correct order. For example if you give SWIG this:

Foo Bar()
%typemap(out) Foo { ... }

The typemap doesn't work, and SWIG offers no warning. It would be nice if users could be given a simple rule-of-thumb like: "always put SWIG directives ABOVE the types and functions to which they apply". In most cases this rule works, but in the case of %template it does not.

[David Piepgrass]

I'd like to add one more nuance. If SWIG's developers add this feature (which would be nice because even if you are able to modify the original header files, it's weird putting SWIG directives in the middle of them) it is necessary to wait not just until the template itself is defined, but until the template arguments are also defined. For example:

%template(FooOfBar) Foo< Bar<Baz> >;
template<class T> class Foo { ... };
template<class T> class Bar { ... };
class Baz { ... };

SWIG shouldn't create FooOfBar until Foo, Bar and Baz are all defined. Most likely SWIG should simply wait until the template is needed before expanding it. For example if SWIG encounters

Foo< Bar<Baz> > x;

or

class DerivedFoo : public Foo< Bar<Baz> > { ... };

that's when it should expand the template (and give a warning, maybe, if Bar and Baz are undefined). At the end of the master .i file if unexpanded templates are still pending, then they should be expanded.

[James Masters]

If nobody has done work on this yet, then I'd like to give it a shot myself. I'm not a full-fledged developer; however, this feature is extremely important to me. If someone could just point out to me where the change should go, then I'll get started. Giving me a pointer will help quite a bit given that I am not familiar with the SWIG code structure. Thanks...

[Olly Betts]

It's nearly 13 years later, but if you (or anyone else) are still interested I can offer pointers - let me know.

Probably as David suggests %template(FooOfBar) could just be stored when seen and the processing of done lazily so it's deferred until referenced (with any left processed after reading all inputs).

[Olly Betts]

SWIG shouldn't create FooOfBar until Foo, Bar and Baz are all defined.

At least in general, that's too conservative I think - see this example from https://sourceforge.net/p/swig/bugs/1316/:

template <typename I>
struct Downcastable
{
    Downcastable<I>() {}
    virtual ~Downcastable<I>() {}

    I* derived()
    {
        return static_cast<I*>(this);
    }
};
// The %template needs to be exactly here to work it seems.
%template(DowncastImpl) Downcastable<Downcasted>;
struct Downcasted : public Downcastable<Downcasted>
{
    Downcasted() {}
    ~Downcasted() {}

    Downcasted* me()
    {
        return derived();
    }
};

[James Masters]

Hi Olly,

Thanks for your replies. As you noted, it's been several years and I've worked around this issue since the original posting. I still use SWIG regularly, but as a user and not contributor. I could try to take this on if you have suggestions on where to look/edit in the source code.

Best regards
James

[Olly Betts]

OK, so currently %template is processed in the parser rule for it in Source/CParse/parser.y:

template_directive: SWIGTEMPLATE LPAREN idstringopt RPAREN idcolonnt LESSTHAN valparms GREATERTHAN SEMI {

This is bison, so $1, $2, etc mean the values of the things after the : above - e.g. $3 is the idstringopt which is the (optional) string inside the () after the %template.

The things in capitals are just tokens, so I think only $3, $5 and $7are relevant here (and only those seem to be referred to in the rule). So probably storing $5 and $7 in a Hash keyed on $3 would work, then moving the code here to its own function which gets called lazily.

The tricky part is probably working out exactly when that lazy call should happen.

This patch would definitely need test coverage, but we have some cases above so that's OK. I can help slotting them into the testsuite if you can't see how to do that - to start with I'd just process them by hand (you can run the in-tree SWIG for manual testing via the preinst-swig wrapper in the top level of the source tree).

[William Fulton]

The documentation, https://swig.org/Doc4.1/SWIGPlus.html#SWIGPlus_template_class_inheritance, does clearly state that you need to instantiate a template before it is used as a base class. Any change from this behaviour will be a major change.

[Olly Betts]

wsfulton: With the change discussed above the template would still be instantiated before being used as a base class - the idea is to allow having the %template EARLIER, in particular before the C++ template definition. This would address the current awkward situation where a %template directive needs to be in the middle of a third-party C++ header which is being wrapped.

This means the %template directive would need to be stored and processed later. The exact point to process seems a bit tricky, since the template may be specialised so it probably can't just be when we see a matching template definition in case there's a specialisation which effectively overrides that, but it does need to happen before anything which needs to come after the %template, such as use as a subclass.