From: Vadim Z. <vz...@ze...> - 2016-11-25 01:30:00
|
Hello, This has been discussed several times in the past, but nothing has been really decided, AFAIK, so I'd like to return to this again in the hope that we finally do something about the problem of having too many Python build variants. Right now we have the following major options: - builtin - classic - modern without speaking about minor ones such as -extranative or a bunch of -fastxxx. According to the documentation, -builtin provides a significant performance improvement and the limitations listed there don't seem significant at all, so I propose making -builtin the default and only way of generating wrappers. It's less clear which benefits does -modern bring as the documentation, but its only requirement seems to be Python 2.2, so it seems perfectly reasonable to turn it on by default and remove the option. Similarly, -classic is only useful for pre-2.2, apparently, so let's drop it and forget it ever existed. Could we please agree to do this? I'd do my best to make the changes doing this that should help a lot with comprehensibility of the Python module code and facilitate further changes in the future as currently it's just too annoying to play whack-a-mole with all these options (which is a game when you do some change working with default options, run the unit test that finds a failure with some non default combination, fix this failure, but this now results in a failure of another test for another options combinations and after fixing this one, things don't work with the default options again). Thanks in advance, VZ |
From: William S F. <ws...@fu...> - 2016-11-25 19:23:24
|
On 25 November 2016 at 01:29, Vadim Zeitlin <vz...@ze...> wrote: > Hello, > > This has been discussed several times in the past, but nothing has been > really decided, AFAIK, so I'd like to return to this again in the hope that > we finally do something about the problem of having too many Python build > variants. > > Right now we have the following major options: > > - builtin > - classic > - modern > > without speaking about minor ones such as -extranative or a bunch of > -fastxxx. > > According to the documentation, -builtin provides a significant > performance improvement and the limitations listed there don't seem > significant at all, so I propose making -builtin the default and only way > of generating wrappers. > > It's less clear which benefits does -modern bring as the documentation, > but its only requirement seems to be Python 2.2, so it seems perfectly > reasonable to turn it on by default and remove the option. Similarly, > -classic is only useful for pre-2.2, apparently, so let's drop it and > forget it ever existed. > > Agreed. All code prior to 2.6 should be dropped, as seems to be the conclusion on https://github.com/swig/swig/issues/701. > Could we please agree to do this? I'd do my best to make the changes doing > this that should help a lot with comprehensibility of the Python module > code and facilitate further changes in the future as currently it's just > too annoying to play whack-a-mole with all these options (which is a game > when you do some change working with default options, run the unit test > that finds a failure with some non default combination, fix this failure, > but this now results in a failure of another test for another options > combinations and after fixing this one, things don't work with the default > options again). > > Yes we should. The page at https://github.com/swig/swig/wiki/SWIG-3.1-Development was started to discuss which options to remove and which should be the default. I suggest we keep this up to date with any suggestions and it will form the documentation/release notes too. I don't think -builtin should be the default though. I've been using it recently on a large code base that has previously used non-builtin for some time. I've been fixing builtin bugs for swig-3.0.11, but there are still more that need fixing. There are also a number of features that don't work with builtin. I'm not sure I have any great solutions for all of them. A lot of customisations needs to be implemented differently with -builtin and so in total I think the pain for users would be too much to make it the default, and certainly not while it is not quite up to par. I think for swig-3.1 non-builtin should remain the default. I would like to see all the options that are used by -O turned on by default and no way to turn them off. That will result in quite a long list of removed options. I think that would keep someone busy for quite a while removing the resulting dead code. How about putting it in a branch if you want to work on this? William |
From: Vadim Z. <vz...@ze...> - 2016-11-25 20:34:09
|
On Fri, 25 Nov 2016 19:22:35 +0000 William S Fulton <ws...@fu...> wrote: WSF> Yes we should. The page at WSF> https://github.com/swig/swig/wiki/SWIG-3.1-Development was started to WSF> discuss which options to remove and which should be the default. I suggest WSF> we keep this up to date with any suggestions and it will form the WSF> documentation/release notes too. Ah, this is where it was. I looked for quite some time among the issues as I completely forgot it was in the wiki instead, thanks for the reminder. WSF> I don't think -builtin should be the default though. I admit I don't have much experience with -builtin, but why not? Having two ways to do something doesn't seem to be ideal, even if it's much better than having 17 (or whatever, I lost count) different ways to do something as is the case now. Using -builtin is supposed to bring clear performance improvements, so we need to keep it for people for whom performance is the top concern, but what is the reason for keeping the Python proxy classes variant? WSF> There are also a number of features that don't work with builtin. What are they? The ones mentioned in the documentation don't seem to be of utmost importance, is there something else/more? WSF> I'm not sure I have any great solutions for all of them. A lot of WSF> customisations needs to be implemented differently with -builtin Hmm, do you mean extending Python generation classes or is there even more to it? WSF> so in total I think the pain for users would be too much to make it WSF> the default, and certainly not while it is not quite up to par. I WSF> think for swig-3.1 non-builtin should remain the default. I guess I could live with this, getting rid of modern/classic/etc would already be a big gain. But the holy grail would still be having just one version... WSF> I would like to see all the options that are used by -O turned on by WSF> default and no way to turn them off. That will result in quite a long list WSF> of removed options. I think that would keep someone busy for quite a while WSF> removing the resulting dead code. How about putting it in a branch if you WSF> want to work on this? My experience of working on long-lived SWIG branches hasn't been uniformly positive so far, to put it mildly, so I'd prefer to make individual small(er) PRs instead in the hope that they could be accepted faster. Maybe we could fork a 3.0.x branch from master instead right now and start preparing for 3.1 on master? Thanks, VZ |
From: William S F. <ws...@fu...> - 2016-11-25 23:30:33
|
On 25 November 2016 at 20:34, Vadim Zeitlin <vz...@ze...> wrote: > On Fri, 25 Nov 2016 19:22:35 +0000 William S Fulton < > ws...@fu...> wrote: > > WSF> Yes we should. The page at > WSF> https://github.com/swig/swig/wiki/SWIG-3.1-Development was started to > WSF> discuss which options to remove and which should be the default. I > suggest > WSF> we keep this up to date with any suggestions and it will form the > WSF> documentation/release notes too. > > Ah, this is where it was. I looked for quite some time among the issues as > I completely forgot it was in the wiki instead, thanks for the reminder. > > WSF> I don't think -builtin should be the default though. > > I admit I don't have much experience with -builtin, but why not? Having > two ways to do something doesn't seem to be ideal, even if it's much better > than having 17 (or whatever, I lost count) different ways to do something > as is the case now. Using -builtin is supposed to bring clear performance > improvements, so we need to keep it for people for whom performance is the > top concern, but what is the reason for keeping the Python proxy classes > variant? > Python is the largest user base and while -builtin is not up to scratch I think it would be crazy to piss off a huge portion of SWIG's users with something substandard. A long term goal may be to have this on as default for swig-3.2 perhaps once -builtin has fully matured? > WSF> There are also a number of features that don't work with builtin. > > What are they? The ones mentioned in the documentation don't seem to be of > utmost importance, is there something else/more? > I've probably forgotten some, but native exceptions, %shadow, %pythoncode don't work. Any special method, eg __str__ will be ignored unless a suitable %feature("python:slot") is added. Also all builtin types are convertible to int which is wrong. Builtin types are also immutable unlike proxy classes. > WSF> I'm not sure I have any great solutions for all of them. A lot of > WSF> customisations needs to be implemented differently with -builtin > > WSF> I would like to see all the options that are used by -O turned on by > WSF> default and no way to turn them off. That will result in quite a long > list > WSF> of removed options. I think that would keep someone busy for quite a > while > WSF> removing the resulting dead code. How about putting it in a branch if > you > WSF> want to work on this? > > My experience of working on long-lived SWIG branches hasn't been uniformly > positive so far, to put it mildly, so I'd prefer to make individual > small(er) PRs instead in the hope that they could be accepted faster. Maybe > we could fork a 3.0.x branch from master instead right now and start > preparing for 3.1 on master? > > Yes great. This new version 3.1 isn't going to happen without a branch for it. I'm afraid I'm not got too much time to focus on 3.1, but happy for others to get going. Anything that is in https://github.com/swig/swig/wiki/SWIG-3.1-Development ought to go into the branch. so you may want to put the doxygen stuff in first as that introduces a pile of code changes. Small patches are great, but if they are at all risky or change default behaviour then they won't be any good for the stable swig-3.0.x branch, but will be suitable for 3.1. I'd say the main reason for not merging big long lived branches is the amount of change is too great for a stable branch and we don't have enough resources to focus on big new releases. William |
From: Vadim Z. <vz...@ze...> - 2016-11-26 00:39:40
|
On Fri, 25 Nov 2016 23:29:44 +0000 William S Fulton <ws...@fu...> wrote: WSF> > WSF> There are also a number of features that don't work with builtin. WSF> > WSF> > What are they? The ones mentioned in the documentation don't seem to be of WSF> > utmost importance, is there something else/more? WSF> > WSF> I've probably forgotten some, but native exceptions, %shadow, %pythoncode WSF> don't work. Any special method, eg __str__ will be ignored unless a WSF> suitable %feature("python:slot") is added. Also all builtin types are WSF> convertible to int which is wrong. Builtin types are also immutable unlike WSF> proxy classes. OK, this does look like a lot of potential breakage. The documentation probably should mention those instead of speaking about compatibility problems with Python 2.[234] that nobody cares about any more... WSF> > My experience of working on long-lived SWIG branches hasn't been uniformly WSF> > positive so far, to put it mildly, so I'd prefer to make individual WSF> > small(er) PRs instead in the hope that they could be accepted faster. Maybe WSF> > we could fork a 3.0.x branch from master instead right now and start WSF> > preparing for 3.1 on master? WSF> > WSF> Yes great. This new version 3.1 isn't going to happen without a branch for WSF> it. I'm afraid I'm not got too much time to focus on 3.1, but happy for WSF> others to get going. Anything that is in WSF> https://github.com/swig/swig/wiki/SWIG-3.1-Development ought to go into the WSF> branch. so you may want to put the doxygen stuff in first as that WSF> introduces a pile of code changes. I really don't want to predicate merging anything else on the merge of the Doxygen branch. I have no idea when, or if, it's going to happen, so why take the risk of not merging the other changes too? WSF> Small patches are great, but if they are at all risky or change default WSF> behaviour then they won't be any good for the stable swig-3.0.x branch, but WSF> will be suitable for 3.1. I'd say the main reason for not merging big long WSF> lived branches is the amount of change is too great for a stable branch and WSF> we don't have enough resources to focus on big new releases. But if this is so, what hope do we have for merging your proposed 3.1 branch which will *combine* even more changes? I was worried about merging just the branch with Python options changes because I am very much afraid that when it's ready to be merged in a few months time, there will be too many changes in it to review. But reviewing it and Doxygen together is even more impossible. Sorry for being direct, but do you realistically believe in being able to do it yourself? Why not create a 3.0 branch as I'm proposing for the future, stopgap 3.0.x releases and start merging stuff into master, testing it and release once it's in a good enough state? Like this it would allow to merge the pieces in reasonably reviewable parts and sooner or later we will make a new release from master. Again, I realize that we all have limited time and I am not trying to create more work for you (or anybody else). But knowing this, I just don't see how would this putative 3.1 branch ever get merged. OTOH I do think that releasing from master, after a period of stabilization, is feasible. Do you still insist on creating a separate branch for all these changes? VZ |
From: William S F. <ws...@fu...> - 2016-11-26 16:53:16
|
On 26 November 2016 at 00:39, Vadim Zeitlin <vz...@ze...> wrote: > On Fri, 25 Nov 2016 23:29:44 +0000 William S Fulton < > ws...@fu...> wrote: > > WSF> > My experience of working on long-lived SWIG branches hasn't been > uniformly > WSF> > positive so far, to put it mildly, so I'd prefer to make individual > WSF> > small(er) PRs instead in the hope that they could be accepted > faster. Maybe > WSF> > we could fork a 3.0.x branch from master instead right now and start > WSF> > preparing for 3.1 on master? > WSF> > > WSF> Yes great. This new version 3.1 isn't going to happen without a > branch for > WSF> it. I'm afraid I'm not got too much time to focus on 3.1, but happy > for > WSF> others to get going. Anything that is in > WSF> https://github.com/swig/swig/wiki/SWIG-3.1-Development ought to go > into the > WSF> branch. so you may want to put the doxygen stuff in first as that > WSF> introduces a pile of code changes. > > I really don't want to predicate merging anything else on the merge of the > Doxygen branch. I have no idea when, or if, it's going to happen, so why > take the risk of not merging the other changes too? > > If I havn't made it clear enough elsewhere, then I'll make it clear now that the doxygen branch is currently in a good enough state for inclusion in 3.1. I don't see any risk of basing other changes on it. > WSF> Small patches are great, but if they are at all risky or change > default > WSF> behaviour then they won't be any good for the stable swig-3.0.x > branch, but > WSF> will be suitable for 3.1. I'd say the main reason for not merging big > long > WSF> lived branches is the amount of change is too great for a stable > branch and > WSF> we don't have enough resources to focus on big new releases. > > But if this is so, what hope do we have for merging your proposed 3.1 > branch which will *combine* even more changes? I was worried about merging > just the branch with Python options changes because I am very much afraid > that when it's ready to be merged in a few months time, there will be too > many changes in it to review. But reviewing it and Doxygen together is even > more impossible. Sorry for being direct, but do you realistically believe > in being able to do it yourself? > > Why not create a 3.0 branch as I'm proposing for the future, stopgap 3.0.x > releases and start merging stuff into master, testing it and release once > it's in a good enough state? Like this it would allow to merge the pieces > in reasonably reviewable parts and sooner or later we will make a new > release from master. > > Again, I realize that we all have limited time and I am not trying to > create more work for you (or anybody else). But knowing this, I just don't > see how would this putative 3.1 branch ever get merged. OTOH I do think > that releasing from master, after a period of stabilization, is feasible. > > Do you still insist on creating a separate branch for all these changes? > > Yes. It is all about priorities and emphasis given the limited number of people willing to get involved. master is for the stable 3.0.x releases, which I will continue to release every few months until 3.1 goes out. With regards to priorities, I want the constant stream of numerous small bug fixes and improvement patches to be pushed out asap. The reality of things means it is best done with the Travis testing and a stable master and 3.0.x is the only thing we have that resembles stable. When 3.1 stops being vapourware and a real branch with the planned and discussed features actually implemented, then, we can do as you suggest and switch master to 3.1.x releases. I'm keen on 3.1 happening soon, but we need active participants, other than Vadim and myself, so anyone else who is willing to help make this happen please let us know. William |
From: Nikita S. <sim...@gm...> - 2016-11-29 23:00:08
|
William, Vadim, Thank you both for your leads on the C++ to C++ swig module. At the moment I do not have an immediate plan to jump in and write the module. My curiosity was mostly about the complexity of such undertaking. Should there be a justified necessity for this, you will hear more from me :-) Best, Nikita On Sat, Nov 26, 2016 at 9:52 AM, William S Fulton <ws...@fu...> wrote: > > > On 26 November 2016 at 00:39, Vadim Zeitlin <vz...@ze...> wrote: > >> On Fri, 25 Nov 2016 23:29:44 +0000 William S Fulton < >> ws...@fu...> wrote: >> >> WSF> > My experience of working on long-lived SWIG branches hasn't been >> uniformly >> WSF> > positive so far, to put it mildly, so I'd prefer to make individual >> WSF> > small(er) PRs instead in the hope that they could be accepted >> faster. Maybe >> WSF> > we could fork a 3.0.x branch from master instead right now and >> start >> WSF> > preparing for 3.1 on master? >> WSF> > >> WSF> Yes great. This new version 3.1 isn't going to happen without a >> branch for >> WSF> it. I'm afraid I'm not got too much time to focus on 3.1, but happy >> for >> WSF> others to get going. Anything that is in >> WSF> https://github.com/swig/swig/wiki/SWIG-3.1-Development ought to go >> into the >> WSF> branch. so you may want to put the doxygen stuff in first as that >> WSF> introduces a pile of code changes. >> >> I really don't want to predicate merging anything else on the merge of >> the >> Doxygen branch. I have no idea when, or if, it's going to happen, so why >> take the risk of not merging the other changes too? >> >> If I havn't made it clear enough elsewhere, then I'll make it clear now > that the doxygen branch is currently in a good enough state for inclusion > in 3.1. I don't see any risk of basing other changes on it. > > >> WSF> Small patches are great, but if they are at all risky or change >> default >> WSF> behaviour then they won't be any good for the stable swig-3.0.x >> branch, but >> WSF> will be suitable for 3.1. I'd say the main reason for not merging >> big long >> WSF> lived branches is the amount of change is too great for a stable >> branch and >> WSF> we don't have enough resources to focus on big new releases. >> >> But if this is so, what hope do we have for merging your proposed 3.1 >> branch which will *combine* even more changes? I was worried about merging >> just the branch with Python options changes because I am very much afraid >> that when it's ready to be merged in a few months time, there will be too >> many changes in it to review. But reviewing it and Doxygen together is >> even >> more impossible. Sorry for being direct, but do you realistically believe >> in being able to do it yourself? >> >> Why not create a 3.0 branch as I'm proposing for the future, stopgap >> 3.0.x >> releases and start merging stuff into master, testing it and release once >> it's in a good enough state? Like this it would allow to merge the pieces >> in reasonably reviewable parts and sooner or later we will make a new >> release from master. >> >> Again, I realize that we all have limited time and I am not trying to >> create more work for you (or anybody else). But knowing this, I just don't >> see how would this putative 3.1 branch ever get merged. OTOH I do think >> that releasing from master, after a period of stabilization, is feasible. >> >> Do you still insist on creating a separate branch for all these changes? >> >> Yes. It is all about priorities and emphasis given the limited number of > people willing to get involved. master is for the stable 3.0.x releases, > which I will continue to release every few months until 3.1 goes out. With > regards to priorities, I want the constant stream of numerous small bug > fixes and improvement patches to be pushed out asap. The reality of things > means it is best done with the Travis testing and a stable master and 3.0.x > is the only thing we have that resembles stable. When 3.1 stops being > vapourware and a real branch with the planned and discussed features > actually implemented, then, we can do as you suggest and switch master to > 3.1.x releases. I'm keen on 3.1 happening soon, but we need active > participants, other than Vadim and myself, so anyone else who is willing to > help make this happen please let us know. > > William > > > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > Swig-devel mailing list > Swi...@li... > https://lists.sourceforge.net/lists/listinfo/swig-devel > > |