John Lenz wrote:
> On 11/14/06 18:40, David Beazley wrote:
>> Now that SWIG-1.3.30 has been put out, I'd like to discuss a few
>> issues related to SWIG development that I've been thinking about
>> lately. These are somewhat general, but I'd be curious to get
>> people's thoughts.
>> 1. SWIG-2.0? Now? Soon? Never?
> Rather than try and respond in the correct spot in the other threads, I
> will put all my thoughts here.
>> The whole point of the SWIG-1.3.x branch of SWIG (back in prehistoric
>> time) was to do work that eventually led to a SWIG-2.0 release. For
>> a variety of reasons, this has never happened. However, I'm
>> wondering if this is something that we ought to actually address at
>> some point. As far as I'm concerned, there are only two central
>> requirements for making 2.0 happen. First, I think the SWIG core
>> should be stable. That is, the interface between the SWIG core and
>> target language modules should be well-defined and stable enough not
>> to change between subsequent versions. Second, all of the
>> documentation needs to be up to date. There are probably a variety
>> of minor things we could throw in, but those are the two biggies in
>> my book.
> Yes, when we were discussing the possibility of loadable swig module
> extensions a few months ago on this list, the lack of a well defined API
> between the swig core and the modules came up as a major problem. From
> my point of view, the modules call such a large number of functions from
> Source/Swig, and Modules/lang.cxx and such.
> see below for more...
>> 2. Testing hell
>> I am extremely frustrated with the difficulty of testing SWIG.
>> Frankly, the problem has just grown completely out of hand. There
>> are too many language modules, different versions of target
>> languages, platforms, compilers, linkers, and everything else. On
>> top of that, running the test suite is a rather time-consuming
>> process. Because of this, I'm wondering if we should rethink this
>> whole problem and come up with something more useful. Can we use
>> something like BuildBot to deal with this? Is there any way that an
>> end user could run the test suite and have results reported to a
>> centralized database? I'm just trying to think of ways to make this
>> easier for developers.
> I have had success using debian (and now ubuntu). I basicly search for
> the packages and the dev packages, and I can most of the time get the
> test suite running. The problem is, some languages have errors and you
> don't know if the errors are problems in the build chain or errors in
> the language module. This would be helped by some way to compare the
> output of the test suite. Compare 1.3.29 to 30 to current CVS.
> I think something like a qemu image would be great to test from. There
> are several projects that automate building, for example
> http://www.openembedded.org/, or using buildd and wanna-build from
> debian (http://www.debian.org/devel/buildd/) in a chroot, or all the
> work on the testing environment William mentioned.
> I have a computer here that I can set up as a test machine... I don't
> have a lot of bandwidth though, but enough to upload results to the web
> page. What I think we first need to do is get some scripts that compare
> the run of the test suite. Because there are a few errors, but if the
> errors are not new errors that is information you want to see. See the
> section of the article called "Buried in warnings".
> http://lwn.net/Articles/206352/ has some discussion of this.
The main 6 languages pass the complete test-suite. If they don't then it
is a problem that needs addressing. These are what I called the first
class languages, the most popular languages. For the remaining
languages, identifying what should be working and what is not working is
a problem. But the problem in the first place is that they are not
mature. This is why they are not in the set of first class languages and
until the maintainer brings them up to scratch will remain that way. To
ease things, maybe we should have a list somewhere of what these 1st
class languages are as then you'd know what should be passing. I think
that scripts that compare the output for these broken modules is a waste
time. Consider what happens if something breaks - the maintainer isn't
going to do anything, not if they can't be bothered to get the
test-suite working in the first place. And who is going to maintain the
expected outputs, considering new tests are being added all the time? If
you disagree and want to get a better handle on the status for these
second class languages, then I think we should add in the capability of
these languages maintaining a list of known broken tests and then
modifying the scripts so that it does not run these broken tests. So
basically something similar to the CPP_TEST_BROKEN in common.mk.
The "Buried in warnings" article was interesting. I've actually tried
hard to ensure we don't suffer that problem as many of my commits are
done just to remove unnecessary warnings with the aim of running the
test-suite warning free. That way all the useful warnings are not
overlooked. If anyone ever proposes changes to the test-suite that
removes either compiler or SWIG warnings, it will be opposed by me as
warnings are a good thing. On my forever expanding todo list is to up
the compiler warnings on gcc, at least, when running the test-suite to
something approaching what the users typically use.
> sourceforge does have testing build servers, and we can even kick off
> the build from a subversion commit hook... if we get the test script
> completely automated, we can have it compile once a week on all the
> In any case, this is something that needs further investigation.
Yes that sounds good, this service could be useful. I think the main
problem is getting a centralised way of reporting results is the
important bit as finding servers to run things on probably isn't too
tricky. You mentioned you've got a box and so have I and as I said, it
is possible to configure one box with many virtual machines covering a
wide array of systems.
>> 3. Batteries included.... or not
>> SWIG seems to include a large number of target language modules, but
>> it's not clear to me how many of these modules are being actively
>> maintained. Does it make sense for us to be maintaining all of this
>> stuff in the CVS repository if there are few users and no active
>> maintenance? To be honest, I've always been kind of in favor of
>> stripping SWIG down to a more common set of target modules and
>> allowing third party modules to be easily imported via dynamic
>> loading. If we had that, it would be possible for lesser-known
>> modules to be maintained separately from the main SWIG distribution
>> yet still work (since they could simply be compiled and imported as
>> extension modules). I just don't know if this is possible or not,
>> but I often think about it.
> The tradeoff you have to make is between how much benefit loadable
> modules are, versus how complicated you make everything. Right now, the
> swig binary links against libc, libm, libstdc++ and nothing else. You
> can pass it a SWIG_LIB environment var pointing to anywhere. This makes
> swig very self contained, it can be run from anywhere with essentially
> no configuration if you use the preinst-swig script. It is a single
> binary. In addition, the resulting files can be compiled from anywhere
> and don't require swig at all. This makes swig very minimal.
> Moving to loadable modules, you will need additional configuration
> (where the modules are, LD_LIBRARY_PATH, etc...) building becomes more
> complicated, libtool needs to be used, and to build modules the headers
> need to be available and the .so files need to be installed, so headers
> need to be packaged and separated out, and swig needs to be installed.
> All these problems are solvable, but a pain in the ass to deal with!
> And in my opinion, the benefits do not outweigh the costs. We don't
> have any modules that are maintained out of tree. Also, language
> modules are not something you just sit down and code up overnight, and
> the extra effort to maintain it in tree as a compiled in language module
> is very small (because swig can be run right from the build directory,
> and never needs to be installed).
> Now, the factoring out and defining of a real public API between the
> SWIG core and the modules is a great idea. But I think that separating
> out the modules from the core in code only, but still linking it all
> together in one binary is the best way to go.
> If we want to differentiate between modules that are maintained, vs old
> modules that might not work anymore (and we don't test), we can issue a
> warning when swig is run or something. Since the public API of the core
> to the language modules should stay stable, old modules (whose
> maintainer has moved on) should still be able to compile and ideally
> keep working, even if the core is changed.
Nicely put John, I agree 100% with this.
> All in all, if we get the testing figured out (see above), then I think
> we should keep as many modules in tree as possible, and keep them all
> compiled in. We can maintain a status on how complete each module is:
> like set up a Level system... for example, new modules start at Level
> 0. Once they get all the basic typemaps covered and some basic
> documentation, Level 1. SWIGTYPE * pointer support, Level 2.
> std::vector, and more documentation gets Level 3, directors Level 5,
> and so on. Something like
> http://worldforge.org/dev/eng/clients/compliance Also look at
> http://worldforge.org/dev/eng/clients for a list of clients, many
> abandoned at various levels.
Yup, the level system is a good idea and formalises what happens in
reality and takes the 1st class and 2nd class language modules a step
> I am probably saying this over and over, but if we have a better
> organization of in-tree language modules, we can keep all the modules
> compiled into one binary and be open about accepting new modules. As
> long as we are up front and say this old module is at Level 1 or
> whatever. The thing is, a single <lang>.cxx adds so little code to the
> binary. Most stuff is in the Lib typemaps, the Examples, docs, etc...
> which can already be changed on the fly by external stuff.
> I think out of tree third party modules will die a quick death. This is
> certainly the case with linux kernel development, where out of tree
> modules require a large amount of work, way more work than in-tree
> modules. And you can find a lot of out of tree modules that have been
> abandoned, because of the extra workload. Sure it is different, but the
> general idea is out of tree modules are harder to maintain.
> I think we should make every effort to include maintained (and new)
> modules in SWIG, with a Level system. The only users of the dynamic
> module loading will be old unmaintained modules, but by their very
> definition they will not be maintained and quickly die without a website
> or project or anything like that.
> Old modules can stick around and hopefully keep working as long as the
> SWIG core doesn't change, and the API stays stable. If a new version of
> SWIG breaks it, well then the module is broken. That is no worse than
> the module being removed from SWIG and not working at all.
Yup, yup, yes and yes, fully agreed.
> Note too that with subversion, it is much easier to maintain a branch,
> and merge changes between the trunk and the branch. So modules not yet
> ready for "release" can be developed in a separate branch. It is
> possible to do a svn switch on the Lib directory, so the branch keeps
> all the trunk changes, or merge stuff between the trunk and the branch.
>> 4. Documentation.
>> I'd like to spend some time bringing the documentation up-to-date
>> with new features. Since I wrote most of the original documentation
>> I'd like to volunteer for this. Besides, it would be a good way for
>> me to get caught up on new features.
> Yes, great. I'm willing to help here too, but I think I will let you
> decide the "vision" so to say. Some new chapters will need to be added,
> and I think the features chapter should probably be broken up. The
> typemaps chapter is also very long...
> The docs probably need to be reorganized a little bit. If you can
> change around the structure, I can help fill in the details.
>> 5. ????
>> Anything else I'm missing.
> I think we should stabilize and essentially "freeze" the SWIG core, but
> not put a freeze on the language modules. I have been waiting for UTL
> to become less murky before transitioning chicken to it. I partially
> converted chicken to the UTL a while back, but haven't done anything
> with it for a while. If the core gets stable and well documented, then
> we should let the various language modules "catch up". This also can be
> made explicit with the Level system I mentioned above.
May I suggest we use doxygen comments for documenting the core? Every
method in a header file should have a doxygen comment, explaining what
it does? Each method can then have a doxygen 'group' tag indicating
which module/API it is in. Or even better use C++ namespaces to indicate
this. Documentation in the source is the only practical way of
documenting code that has any chance of keeping up to date.