From: David N. <co...@kl...> - 2010-09-15 00:09:08
|
Hello all, during the last few months, I have been working on a SWIG module for the D programming language (http://www.digitalmars.com/d/). Even though I have only been able to spend time on it rather sporadically, it has progressed to a state where including it into SWIG trunk seems like a reasonable thing to do. It started out as a fork of the C# module and currently supports D1/Tango and D2/Phobos. Currently, the test suite passes (except for »preproc_line_file« which almost certainly fails on other languages too since it uses the Java-specific %javaconst without testing for SWIGJAVA) and almost all runtime tests have been ported from C# to both D 1.0 and D 2.0. From my point of view there are a few things left which I will sort out in the next few days (basic documentation, checking if the test suite works on Windows too), but I thought I'd ask if there is a chance that the module will be accepted into trunk, and if so, how that process would look like. The current state can be found at GitHub: http://github.com/klickverbot/swig/tree/swigd. David |
From: David N. <co...@kl...> - 2010-09-15 00:23:54
|
P.S.: Forget that comment about preproc_line_file – I was still using trunk from last week, it has been fixed in the meantime. |
From: William S F. <ws...@fu...> - 2010-09-15 22:23:25
|
David Nadlinger wrote: > Hello all, > > during the last few months, I have been working on a SWIG module for the > D programming language (http://www.digitalmars.com/d/). Even though I > have only been able to spend time on it rather sporadically, it has > progressed to a state where including it into SWIG trunk seems like a > reasonable thing to do. It started out as a fork of the C# module and > currently supports D1/Tango and D2/Phobos. > > Currently, the test suite passes (except for »preproc_line_file« which > almost certainly fails on other languages too since it uses the > Java-specific %javaconst without testing for SWIGJAVA) and almost all > runtime tests have been ported from C# to both D 1.0 and D 2.0. > > From my point of view there are a few things left which I will sort out > in the next few days (basic documentation, checking if the test suite > works on Windows too), but I thought I'd ask if there is a chance that > the module will be accepted into trunk, and if so, how that process > would look like. > > The current state can be found at GitHub: > http://github.com/klickverbot/swig/tree/swigd. It sounds like you have met the prerequisites for adding a new language - http://www.swig.org/Doc2.0/Extending.html#Extending_prerequisites, so I don't envisage any problems accepting this into trunk. I'd need to review it first, but can't promise quite when. Does github have a decent interface to view what is different to SWIG head, that is, easily show just the changes you have made as one patch? Assuming the review works out okay, I'll get you svn access whereupon you can commit the changes, provided you are willing to maintain the D language module for the foreseeable future. When I take a look, I'd appreciate information on which D packages to install on Debian/Ubuntu to try it out. BTW, if you are looking at porting runtime tests, the Java tests are usually very easily converted into C# tests, and hence I suspect D tests. The python runtime tests are of course the most comprehensive and are also not too hard to port to a curly brace language. William |
From: David N. <co...@kl...> - 2010-09-27 07:10:26
|
Oh, William just pointed me to the fact that I must have accidentally hit the wrong button before, as I replied to him off-list only. Sorry for that, here is my original answer: --- Sorry for the late reply, but I was busy tracking down bugs in the C++/D toolchain fox OS X the last days (which cause some of the runtime tests to fail on OS X 10.6, but more about that later). > Does github have a decent interface to view what is different to SWIG > head, that is, easily show just the changes you have made as one patch? Usually, you could use the built-in compare interface (http://github.com/klickverbot/swig/compare/master...swigd), but since it chokes on the size of the diff, the easiest way is probably just to view the differences locally. I don't know how familiar you are with Git, but these commands should help you there (one-time ~20 MiB download, since it includes the whole Git'ified history of the SWIG SVN repository): git clone http://github.com/klickverbot/swig.git cd swig git diff master..swigd While looking at the diff output, I just noticed that I forgot to turn off the auto-stripping of trailing whitespace while editing some files. If necessary, I can remove the whitespace changes from the commit. > […]provided you are willing to maintain the D language module for the > foreseeable future. This should not be an issue. While I will not be able to work big new features, I will certainly be able to spare some time to deal with the usual maintenance business. > When I take a look, I'd appreciate information on which D packages to > install on Debian/Ubuntu to try it out. There are currently two major versions: D1, which is in maintenance mode, and D2, which is spec-frozen, but actively worked one. It's a long time since I used a Debian-based system as my development box, but for D1, the easiest way to get a working setup is probably installing the LLVM D Compiler and Tango (the de-facto standard library for D1) from this PPA: https://launchpad.net/~d-language-packagers/+archive/ppa. For D2, your only choice is the DigitalMars D compiler (the reference implementation), you can find a .deb at http://www.digitalmars.com/d/download.html. The Phobos standard library should be included. By default, the examples and the test-suite are built in D1 mode, pass D_VERSION={1,2} to make to override this. A couple of notes on the current state: Is there any standard way of creating an output directory? Currently, I am just calling system("mkdir -p …"), since there is no standard way to do this in C, but this is not very elegant – it should work for most users though, since SWIG requires a UNIX-like environment for the build system anyway. Forcing the user to manually create the needed directories and move the generated D files to it (packages in the D import path are mapped to file system directories) does not seem like a good choice either, especially in combination with the nspace feature (which I've had no time to implement for D yet, but the issue remains). Since D supports (C-style) pointers, pointers to primitive types and pointers to functions only having (pointers to) primitive types in their signature are translated to native D pointers by default (any number of indirections, only if there is no other typemap overriding that behavior). To implement this feature, I had to add another C-to-C++ hack called Language_unmatched_typemap_request_handler allowing a language module to hook into the typemap resolution process (the invoking code is at typemap.c:780). Is there any neater way of doing this? A related question: To avoid a crash occurring after I inserted the native pointer handling code, I had to add a check for null in Swig_ref_call (cwrap.c:628). Without it, SWIG crashes when fed with the D test suite. I am not quite sure, however, if this is the correct place for this check (I added a short TODO note). Unfortunately I don't know enough about the inner semantics of the SWIG typemap system to be able to find out which higher-level conditions trigger the bug, and, thus, what would be the correct way of avoiding the crash. Regarding the test-suite: Unfortunately, I had to rename two test cases (template to template_basic, immutable to immutable_values) because their names clashes with D keywords. I think this is still a cleaner solution than adding two special cases in the D test-suite build scripts to choose other directory names for the two test cases. There did not seem to be any runtime test cases for these, so the name change should not cause any trouble. About the OS X issues mentioned above: It turned out that somehow the Apple-specific -flat_namespace linker option causes crashes in the C/C++ code because of relocation issues when a C dynamic library is loaded from D. The details are not quite clear yet, but I hope to be able to find out on which side (Apple's custom ld or the D toolchain) the bug lies in the next few days. Do you have any idea about why the »-flat_namespace -undefined suppress« options where actually added to the configure script in the first place (I am all new to OS X development)? There is also another strange OS X-specific issue where an exception is not caught at the C++ side (in the »throw_exception« test case). I'll investigate that, although it is probably a bug in Apple's fairly old version of GCC since the piece of code works perfectly fine on Linux. Another bug outside the scope of SWIG affects the »threads« test case: The generated code used to complete the test perfectly well, but some recent changes to the D garbage collector introduced a bug, occasionally causing the test case to hit a deadlock in the D runtime/GC code. This is investigated by the D community. Other than that, the test-suite should build and run fine with the current versions of DMD, LDC and Tango on Linux x86_64 and Mac OS X 10.6 (I will probably set up a Windows VM to run the tests there soon). Okay, now sorry for this way too long message. I'd be glad to answer any remaining questions though! David |
From: William S F. <ws...@fu...> - 2010-09-27 18:31:53
|
David Nadlinger wrote: > Sorry for the late reply, but I was busy tracking down bugs in the C++/D > toolchain fox OS X the last days (which cause some of the runtime tests > to fail on OS X 10.6, but more about that later). > > > > Does github have a decent interface to view what is different to SWIG > > head, that is, easily show just the changes you have made as one patch? > Usually, you could use the built-in compare interface > (http://github.com/klickverbot/swig/compare/master...swigd), but since > it chokes on the size of the diff, the easiest way is probably just to > view the differences locally. > > I don't know how familiar you are with Git, but these commands should > help you there (one-time ~20 MiB download, since it includes the whole > Git'ified history of the SWIG SVN repository): > > git clone http://github.com/klickverbot/swig.git > cd swig > git diff master..swigd > It's all new to me, but learning some git ain't a bad thing. > While looking at the diff output, I just noticed that I forgot to turn > off the auto-stripping of trailing whitespace while editing some files. > If necessary, I can remove the whitespace changes from the commit. > The odd whitespace change is okay, but as soon as there become a lot it gets too intrusive, so if there are many of them please remove. I wouldn't have thought there'd be too many trailing whitespaces though. > > > […]provided you are willing to maintain the D language module for the > > foreseeable future. > This should not be an issue. While I will not be able to work big new > features, I will certainly be able to spare some time to deal with the > usual maintenance business. > That's okay, thanks. > > > When I take a look, I'd appreciate information on which D packages to > > install on Debian/Ubuntu to try it out. > There are currently two major versions: D1, which is in maintenance > mode, and D2, which is spec-frozen, but actively worked one. > > It's a long time since I used a Debian-based system as my development > box, but for D1, the easiest way to get a working setup is probably > installing the LLVM D Compiler and Tango (the de-facto standard library > for D1) from this PPA: > https://launchpad.net/~d-language-packagers/+archive/ppa. > > For D2, your only choice is the DigitalMars D compiler (the reference > implementation), you can find a .deb at > http://www.digitalmars.com/d/download.html. The Phobos standard library > should be included. > > By default, the examples and the test-suite are built in D1 mode, pass > D_VERSION={1,2} to make to override this. > Okay, useful thanks, I'll go with one of these. > > A couple of notes on the current state: > > Is there any standard way of creating an output directory? Currently, I > am just calling system("mkdir -p …"), since there is no standard way to > do this in C, but this is not very elegant – it should work for most > users though, since SWIG requires a UNIX-like environment for the build > system anyway. Forcing the user to manually create the needed > directories and move the generated D files to it (packages in the D > import path are mapped to file system directories) does not seem like a > good choice either, especially in combination with the nspace feature > (which I've had no time to implement for D yet, but the issue remains). > Swig_new_subdirectory() is what you could use, but probably not. The only time SWIG should create subdirectories is for the nspace feature. Any other reason to go around creating subdirectories needs to be justified as the general philosophy is for users to create any directories they need. D should behave as much as other modules as possible and in this it sounds like it should mirror the Java and C# usage where files can be generated into a different package / namespace directory created by the user. > Since D supports (C-style) pointers, pointers to primitive types and > pointers to functions only having (pointers to) primitive types in their > signature are translated to native D pointers by default (any number of > indirections, only if there is no other typemap overriding that > behavior). To implement this feature, I had to add another C-to-C++ hack > called Language_unmatched_typemap_request_handler allowing a language > module to hook into the typemap resolution process (the invoking code is > at typemap.c:780). Is there any neater way of doing this? > I'm not following 100% and an example might help. Are you saying that functions that only have primitive pointers as parameters are handled differently to when the function has a mix of pointers and non-pointers? So there is a change in marshalling/type handling depending on other types in the function? If you are saying that pointers to primitive types are being handled differently to pointers to non-primitive types, then you ought to be adding in specific primtive pointer typemaps, one for each primitive type - that shouldn't be too much to ask. > A related question: To avoid a crash occurring after I inserted the > native pointer handling code, I had to add a check for null in > Swig_ref_call (cwrap.c:628). Without it, SWIG crashes when fed with the > D test suite. I am not quite sure, however, if this is the correct place > for this check (I added a short TODO note). Unfortunately I don't know > enough about the inner semantics of the SWIG typemap system to be able > to find out which higher-level conditions trigger the bug, and, thus, > what would be the correct way of avoiding the crash. > Line 628 is: if (ref) { so maybe you aren't refering to trunk HEAD. If you mean the line above: String *ref = Swig_rflag_search(cn, "feature:ref", "feature:noref"); cn should always be non-null. If it isn't I'd say it is a bug somewhere. > Regarding the test-suite: Unfortunately, I had to rename two test cases > (template to template_basic, immutable to immutable_values) because > their names clashes with D keywords. I think this is still a cleaner > solution than adding two special cases in the D test-suite build scripts > to choose other directory names for the two test cases. There did not > seem to be any runtime test cases for these, so the name change should > not cause any trouble. Renaming test cases is preferable. > > About the OS X issues mentioned above: It turned out that somehow the > Apple-specific -flat_namespace linker option causes crashes in the C/C++ > code because of relocation issues when a C dynamic library is loaded > from D. The details are not quite clear yet, but I hope to be able to > find out on which side (Apple's custom ld or the D toolchain) the bug > lies in the next few days. Do you have any idea about why the > »-flat_namespace -undefined suppress« options where actually added to > the configure script in the first place (I am all new to OS X development)? > > There is also another strange OS X-specific issue where an exception is > not caught at the C++ side (in the »throw_exception« test case). I'll > investigate that, although it is probably a bug in Apple's fairly old > version of GCC since the piece of code works perfectly fine on Linux. > Sorry, someone else will have to help, I don't do OSX. > Another bug outside the scope of SWIG affects the »threads« test case: > The generated code used to complete the test perfectly well, but some > recent changes to the D garbage collector introduced a bug, occasionally > causing the test case to hit a deadlock in the D runtime/GC code. This > is investigated by the D community. > Glad I'm not looking at that ;) > Other than that, the test-suite should build and run fine with the > current versions of DMD, LDC and Tango on Linux x86_64 and Mac OS X 10.6 > (I will probably set up a Windows VM to run the tests there soon). > Superb! > > Okay, now sorry for this way too long message. I'd be glad to answer any > remaining questions though! > Probably we should aim to get this in SWIG-2.0.2 and so you should aim to commit this direct to trunk after I've found time to review further. Right now I want to concentrate on a new release. William |
From: David N. <co...@kl...> - 2010-10-06 11:46:54
|
On 9/27/10 8:31 PM, William S Fulton wrote: >> Since D supports (C-style) pointers, pointers to primitive types and >> pointers to functions only having (pointers to) primitive types in >> their signature are translated to native D pointers by default (any >> number of indirections, only if there is no other typemap overriding >> that behavior). […] >> > I'm not following 100% and an example might help. Are you saying that > functions that only have primitive pointers as parameters are handled > differently to when the function has a mix of pointers and non-pointers? > So there is a change in marshalling/type handling depending on other > types in the function? Okay, let me elaborate a bit. D supports the C ABI and has all C primitive types too, so you can directly use C pointers from within D to access data. This obviously only works if the data is stored the same on a bit level, i.e. there is no additional conversion needed. If it was only for the quite limited number of primitive types, defining the typemaps per hand would certainly be feasible. However, this works for an arbitrary number of indirections. So, e.g. if a C function returned int***, you could directly use that result in D without any conversion code. Additionally, and this is where defining the typemaps per hand is not much of an option anymore, D also has C-style function pointers. For example, consider the following C function declaration: void add_callback(void (* pointer)(char *name), int *was_added); Using the feature I implemented by providing a hook in the typemap lookup process of SWIG before falling back to SWIGTYPE, this is directly translated to the corresponding D signature (omitting the extern(C) attributes for simplicity): void add_callback( void function( char* name ), int* was_added ); If there were no default typemaps for these cases, interfacing with C code would be a lot more difficult than necessary. Yes, this breaks as soon as types wrapped with a proxy class enter the picture, but the D module just falls back to a SWIGTYPE_… type wrapper class then. But it is surprising how many »plain old C« libraries can be used directly even with this restriction in place. Oh, in theory it would be possible to wrap C structs or C++ POD structs/classes as D structs as well (as mentioned above, the whole C ABI is supported), providing bit-level compatibility for these too, but this is a quite intricate and error-prone process – even more so as there is, as far as I know, no other SWIG module which does something comparable. David |
From: William S F. <ws...@fu...> - 2010-10-15 20:47:59
|
David Nadlinger wrote: > On 9/27/10 8:31 PM, William S Fulton wrote: >>> Since D supports (C-style) pointers, pointers to primitive types and >>> pointers to functions only having (pointers to) primitive types in >>> their signature are translated to native D pointers by default (any >>> number of indirections, only if there is no other typemap overriding >>> that behavior). […] >>> >> I'm not following 100% and an example might help. Are you saying that >> functions that only have primitive pointers as parameters are handled >> differently to when the function has a mix of pointers and non-pointers? >> So there is a change in marshalling/type handling depending on other >> types in the function? > > Okay, let me elaborate a bit. D supports the C ABI and has all C > primitive types too, so you can directly use C pointers from within D to > access data. This obviously only works if the data is stored the same on > a bit level, i.e. there is no additional conversion needed. > > If it was only for the quite limited number of primitive types, defining > the typemaps per hand would certainly be feasible. However, this works > for an arbitrary number of indirections. So, e.g. if a C function > returned int***, you could directly use that result in D without any > conversion code. > > Additionally, and this is where defining the typemaps per hand is not > much of an option anymore, D also has C-style function pointers. For > example, consider the following C function declaration: > > void add_callback(void (* pointer)(char *name), int *was_added); > > Using the feature I implemented by providing a hook in the typemap > lookup process of SWIG before falling back to SWIGTYPE, this is directly > translated to the corresponding D signature (omitting the extern(C) > attributes for simplicity): > I think you can just use the normal special variable replacement mechanism that is in place already. Look for this sort of code in the other language modules: if ((tm = Getattr(p, "tmap:in"))) { Replaceall(tm, "$input", arg); p = Getattr(p, "tmap:in:next"); So you can search and replace any special variables in the typemap with whatever you want, almost always something based on the type. So you'd have a special variable just for D which gets replaced by the D signature for the type. Also you can pick up typemap attributes, eg, "tmap:in:disown" in python specified in the typemap as %typemap(in, disown=1) type "..." William |
From: David N. <co...@kl...> - 2010-10-15 22:29:21
|
On 10/15/10 10:41 PM, William S Fulton wrote: > I think you can just use the normal special variable replacement > mechanism that is in place already. Look for this sort of code in the > other language modules: > > if ((tm = Getattr(p, "tmap:in"))) { > Replaceall(tm, "$input", arg); > p = Getattr(p, "tmap:in:next"); Excuse me, but could you elaborate a bit? I don't really see how this would solve my problem of matching arbitrary levels of pointer indirections, but only to types with bit-compatible representations in C and D, i.e. primitive types. I already use typemap attributes to specify these »target« types, but I still need the hook in the lookup process to be able to match variable levels of indirection (while still allowing the user to override the behavior with specific typemaps, e.g. mapping an output parameter using typemaps.i). David |
From: Ian L. T. <ia...@go...> - 2010-10-15 23:06:46
|
David Nadlinger <co...@kl...> writes: > On 10/15/10 10:41 PM, William S Fulton wrote: >> I think you can just use the normal special variable replacement >> mechanism that is in place already. Look for this sort of code in the >> other language modules: >> >> if ((tm = Getattr(p, "tmap:in"))) { >> Replaceall(tm, "$input", arg); >> p = Getattr(p, "tmap:in:next"); > > Excuse me, but could you elaborate a bit? I don't really see how this > would solve my problem of matching arbitrary levels of pointer > indirections, but only to types with bit-compatible representations in C > and D, i.e. primitive types. > > I already use typemap attributes to specify these »target« types, but I > still need the hook in the lookup process to be able to match variable > levels of indirection (while still allowing the user to override the > behavior with specific typemaps, e.g. mapping an output parameter using > typemaps.i). For Go I handle this in the C++ code. The actual code is probably fairly impenetrable at this point, but the basic implementation is pretty simple. It's in goTypeWithInfo in go.cxx, and the relevant parts looks like this: } else if (SwigType_ispointer(t) || SwigType_isarray(t)) { SwigType *r = Copy(t); if (SwigType_ispointer(r)) { SwigType_del_pointer(r); } else { SwigType_del_array(r); } String *base = goTypeWithInfo(n, r, &is_interface); ret = NewString("*"); Append(ret, base); Delete(base); Ian |
From: David N. <co...@kl...> - 2010-10-17 21:26:54
|
On 10/16/10 1:06 AM, Ian Lance Taylor wrote: > For Go I handle this in the C++ code. The actual code is probably > fairly impenetrable at this point, but the basic implementation is > pretty simple. It's in goTypeWithInfo in go.cxx, and the relevant parts > looks like this: […] From a quick glance at the Go module (I have not gotten around to have a look at Go itself yet) I'd guess that you are only dealing with the Go type name in goTypeWithInfo(), and not with multiple typemaps, right? This is the problem I have in the D module: I need to be able to override the typemap lookup fallback for several typemaps kinds, just as if they were defined for all pointer types translatable to D in the D SWIG library files. If it was possible to define typemaps for types given via regex-like wildcards (including referring to the matches in the typemap body itself), it would be possible to implement this in the library only, but since this is not the case, the hook into the typemap lookup function was the cleanest solution I could find… David |
From: William S F. <ws...@fu...> - 2010-10-21 19:36:20
|
David Nadlinger wrote: > On 10/16/10 1:06 AM, Ian Lance Taylor wrote: >> For Go I handle this in the C++ code. The actual code is probably >> fairly impenetrable at this point, but the basic implementation is >> pretty simple. It's in goTypeWithInfo in go.cxx, and the relevant parts >> looks like this: […] > > From a quick glance at the Go module (I have not gotten around to have > a look at Go itself yet) I'd guess that you are only dealing with the Go > type name in goTypeWithInfo(), and not with multiple typemaps, right? > > This is the problem I have in the D module: I need to be able to > override the typemap lookup fallback for several typemaps kinds, just as > if they were defined for all pointer types translatable to D in the D > SWIG library files. > > If it was possible to define typemaps for types given via regex-like > wildcards (including referring to the matches in the typemap body > itself), it would be possible to implement this in the library only, but > since this is not the case, the hook into the typemap lookup function > was the cleanest solution I could find… > Sorry for slow replies, am busy. Can you explain why the normal typemap matching rules as documented do not work as I am not following your use cases? From what I can see if I put a debug hook into D::unmatchedTypemapRequestHandler, you are translating pointers into void * for cwtype and dwtype and keeping the pointer as is for dptype. SWIGTYPE * should handle this fine, like: %typemap(cwtype) SWIGTYPE * "void *" %typemap(dwtype) SWIGTYPE * "void *" %typemap(dptype) SWIGTYPE * "$1_type" Here, $1_type is a special variable which SWIG replaces with int * - standard documented behaviour. But you can provide your own special variables and do your own special variable replacement as I mentioned in my previous email. In other words, instead of hooking in your custom code by modifying typemaps.c, you work with the typemap system and the way the other language modules work by using special variables and replacing them with your custom code (related to that type). The $javaclassname is very closely related to the problems you need to solve, as it gets replaced with the Java proxy class, if the type is known, or otherwise the typewrapper class if it is not known. Also look at $javainput in the javain typemap. Currently, a user cannot even override the behaviour by providing a SWIGTYPE * typemap - it gets ignored, so the typemap system is broken for D. btw: make SWIGOPT=-debug-tmsearch in one of the examples directory should work like the other modules, there is something wrong in your makefiles. Also the results of the typemap search are not being displayed. Likewise with -debug-tmused. The java and C# typemaps are grouped together in the way I think you want them. For example, see the ctype, imtype and cstype typemaps - these must always be specified together as what one puts into the ctype, must be aligned with what is in the imtype and cstype as well as in and csin, for that matter. For function pointers from your earlier example, where the C type is: void (* pointer)(char *name) and you want the dptype to be: void function(char*) Use: %typemap(dptype) SWIGTYPE (*)(ANY) "$dfuncptrtype" then substitute the new D specific $dfuncptrtype special variable with 'void function(char*)' as I mentioned in my previous email - use "tmap:dptype" for the typemap method name. BTW, -debug-tmsearch is your friend here (where it works with the other language modules) - it tells you what types it is searching for and lists the generic: SWIGTYPE(*)(ANY) type amongst those considered. It is all recently documented - probably after you first started working on D. William |
From: David N. <co...@kl...> - 2010-10-21 20:45:27
|
On 10/21/10 9:08 PM, William S Fulton wrote: > make SWIGOPT=-debug-tmsearch > in one of the examples directory should work like the other modules, > there is something wrong in your makefiles. Also the results of the > typemap search are not being displayed. Likewise with -debug-tmused. Okay, I'll have a look at that – unfortunately, SWIGOPT is not documented anywhere, so I assumed that it's just for internal use. Thanks for providing the detailed explanation of your special variable suggestion – at the first glance, replaceSpecialVariables could work as a hook instead of the custom one indeed. Just to make it clear what the pointer-related code does: --- %inline %{ class SomeClass { }; class OpaqueClass; typedef void (*FuncA)(int **x, char ***y); typedef void (*FuncB)(int **x, SomeClass *y); int *a( int *value ); float **b( float **value ); char ***c( char ***value ); SomeClass *d( SomeClass *value ); SomeClass **e( SomeClass **value ); OpaqueClass *f( OpaqueClass *value ); FuncA g( FuncA value ); FuncB* h( FuncB* value ); %} // Generated D functions: // --- // int* a(int* value); // float** b(float** value); // char*** c(char*** value); // SomeClass d(SomeClass value); // SWIGTYPE_p_p_SomeClass e(SWIGTYPE_p_p_SomeClass value); // SWIGTYPE_p_OpaqueClass f(SWIGTYPE_p_OpaqueClass value); // void function(int**, char***) g(void function(int**, char***) value); // SWIGTYPE_p_f_p_p_int_p_SomeClass__void h(SWIGTYPE_p_f_p_p_int_p_SomeClass__void value); --- Note that the SomeClass* case isn't handled by the code in lookupNativePointerTypemap(), but by the »SWIGTYPE *« typemaps in Lib/d/dclasses.swg. I'm not sure why overriding that typemap shouldn't work, but I'll have another look at it when I try to replace the D-specific hook by extensions to replaceSpecialVariables(). David |
From: William S F. <ws...@fu...> - 2010-10-22 22:07:46
|
David Nadlinger wrote: > On 10/21/10 9:08 PM, William S Fulton wrote: >> make SWIGOPT=-debug-tmsearch >> in one of the examples directory should work like the other modules, >> there is something wrong in your makefiles. Also the results of the >> typemap search are not being displayed. Likewise with -debug-tmused. > > Okay, I'll have a look at that – unfortunately, SWIGOPT is not > documented anywhere, so I assumed that it's just for internal use. > Yup, but I like to keep the build system as common as possible for maintainability. I'm a strong believer in consistency where possible. A couple of languages have gone and done their own thing and now they don't work. Please don't add to that list. > Thanks for providing the detailed explanation of your special variable > suggestion – at the first glance, replaceSpecialVariables could work as > a hook instead of the custom one indeed. > > Just to make it clear what the pointer-related code does: > > --- > %inline %{ > class SomeClass { > }; > class OpaqueClass; > typedef void (*FuncA)(int **x, char ***y); > typedef void (*FuncB)(int **x, SomeClass *y); > > int *a( int *value ); > float **b( float **value ); > char ***c( char ***value ); > SomeClass *d( SomeClass *value ); > SomeClass **e( SomeClass **value ); > OpaqueClass *f( OpaqueClass *value ); > FuncA g( FuncA value ); > FuncB* h( FuncB* value ); > %} > > // Generated D functions: > // --- > // int* a(int* value); > // float** b(float** value); > // char*** c(char*** value); > // SomeClass d(SomeClass value); > // SWIGTYPE_p_p_SomeClass e(SWIGTYPE_p_p_SomeClass value); > // SWIGTYPE_p_OpaqueClass f(SWIGTYPE_p_OpaqueClass value); > // void function(int**, char***) g(void function(int**, char***) value); > // SWIGTYPE_p_f_p_p_int_p_SomeClass__void > h(SWIGTYPE_p_f_p_p_int_p_SomeClass__void value); > --- > > Note that the SomeClass* case isn't handled by the code in > lookupNativePointerTypemap(), but by the »SWIGTYPE *« typemaps in > Lib/d/dclasses.swg. I'm not sure why overriding that typemap shouldn't > work, but I'll have another look at it when I try to replace the > D-specific hook by extensions to replaceSpecialVariables(). Handling all primitive types differently to non-primitive types like this is a bit painful. However, it isn't that hard to write specific typemaps. You can use just 1 line of code per typemap method per pointer level. For example normal pointers: %typemap(dptype) bool *, signed char *, unsigned char*, short *, unsigned short *, int *, unsigned int *, long *, unsigned long *, long long *, unsigned long long *, float *, double * "$1_type" then copy paste and add in another * for pointer to pointers. I'd expect you don't need to deal with anything more than 3 levels of pointers, but just in case you can use SWIGTYPE **** to print out an unsupported warning if used: %typemap(out, warning="700:Unsupported type") SWIGTYPE **** %{#error "not supported"%} Replace 700 with an appropriate new D warning number. The above will catch all types using 4 or more pointers, eg short ***** You can also put your typemaps into a single simple macro to cut the code size down even further. This is much better than hard coded handling in d.cxx. Note that I left out char * in the list above and added signed char * in instead. This is because char * should always be handled as a C string and signed char * as a pointer to a type char. Separate typemaps for char * would be needed as probably your string handling is going to be different to pointer handling. char ** or char *** shouldn't be treated any differently to other primitive pointers though. So when you implement this, you won't even need much special handling or need for hooks or extra special variables, except you'd need to handle the equivalent to the $javaclassname special variable for the type wrapper classes or proxy classes - but here you can do the same as in Java or C#. Hope that helps. William |
From: David N. <co...@kl...> - 2010-10-30 00:22:08
|
On 10/23/10 12:07 AM, William S Fulton wrote: > Yup, but I like to keep the build system as common as possible for > maintainability. I'm a strong believer in consistency where possible. A > couple of languages have gone and done their own thing and now they > don't work. Please don't add to that list. Yes, I am a very strong supporter of consistency as well – if I know what being consistent means at the first place, which is not always the case for SWIG, since I had only very limited experience with it before starting the module. Anyway, I just pushed a commit to my repo at GitHub bringing the D module in line with the other languages; with my comment, I just wanted to point at the fact that »grep -r SWIGOPT Doc/« returns zero results… > So when you implement this, you won't even need much special handling or > need for hooks or extra special variables, except you'd need to handle > the equivalent to the $javaclassname special variable for the type > wrapper classes or proxy classes - but here you can do the same as in > Java or C#. Okay, thanks to your hints, I have finally been able to come up with a solution which requires no non-standard hooks of any kind, and has only a minimal amount of actual codegen logic in the D module: I introduced a »nativepointer« typemap attribute which replaces the typemap contents when looking up a primitive type, somewhat similar to the »out« attributes used for C#. This also greatly reduces the impact of the D module on common code. I have pushed the related commits to GitHub as well. Thanks for your patience, David |
From: William S F. <ws...@fu...> - 2010-11-09 07:23:09
|
David Nadlinger wrote: > On 10/23/10 12:07 AM, William S Fulton wrote: >> Yup, but I like to keep the build system as common as possible for >> maintainability. I'm a strong believer in consistency where possible. A >> couple of languages have gone and done their own thing and now they >> don't work. Please don't add to that list. > > Yes, I am a very strong supporter of consistency as well – if I know > what being consistent means at the first place, which is not always the > case for SWIG, since I had only very limited experience with it before > starting the module. > Hah, admittedly reverse engineering the consistency isn't easy > Anyway, I just pushed a commit to my repo at GitHub bringing the D > module in line with the other languages; with my comment, I just wanted > to point at the fact that »grep -r SWIGOPT Doc/« returns zero results… > >> So when you implement this, you won't even need much special handling or >> need for hooks or extra special variables, except you'd need to handle >> the equivalent to the $javaclassname special variable for the type >> wrapper classes or proxy classes - but here you can do the same as in >> Java or C#. > > Okay, thanks to your hints, I have finally been able to come up with a > solution which requires no non-standard hooks of any kind, and has only > a minimal amount of actual codegen logic in the D module: I introduced a > »nativepointer« typemap attribute which replaces the typemap contents > when looking up a primitive type, somewhat similar to the »out« > attributes used for C#. > > This also greatly reduces the impact of the D module on common code. I > have pushed the related commits to GitHub as well. > These changes are perfect. Thanks. William |
From: David N. <co...@kl...> - 2010-10-30 00:51:09
|
On 10/8/10 7:58 AM, William S Fulton wrote: > The files in the Lib directory should not have any license headers > (these were removed for swig-2.0). Fixed. > wchar.i is in a mixed Unix/Windows file format (I've just fixed the > csharp version in respect of this) wchar.i and std_wstring.i were not yet ported from C#, I have removed them until I get around to have a proper go at wide strings. > In Doc/Manual chapters and Sections.html - D is not in alphabetical order. Fixed. > wrapperloader.swg license - why this license? We should only be using > the SWIG license. This is a bit of an issue – the license applies not really to wrapperloader.swg itself, but rather to the generated code. The problem is that the D2 standard library does not currently include a reliable cross-platform way to load symbols from dynamic libraries. Derelict, a game-development centric bindings project, have developed such a loader for their own bindings and made it available under the Boost Software License. I adapted the code from their loader, and thus put the license note in there. I guess I could attempt an … uhm … clean room implementation of the loader code, but I am not sure if this would make any sense at all – the D standard library is Boost-licensed as well, and the license should not require any attribution from SWIG users besides not removing that notice from the generated source files (I am no lawyer though). Basically, the Boost license was developed for (standard) libraries which need a »do whatever you like with it, just don't claim you wrote that source code«-style license, so I really don't see how it could be an issue for any SWIG users, but when I finally manage to find some time for implementing static linking, dynamically loading the wrapper library (and thus the inclusion of this piece of code) will become fully optional anyway… David |
From: William S F. <ws...@fu...> - 2010-11-09 07:18:20
|
David Nadlinger wrote: > On 10/8/10 7:58 AM, William S Fulton wrote: > > wrapperloader.swg license - why this license? We should only be using > > the SWIG license. > This is a bit of an issue – the license applies not really to > wrapperloader.swg itself, but rather to the generated code. > > The problem is that the D2 standard library does not currently include a > reliable cross-platform way to load symbols from dynamic libraries. > Derelict, a game-development centric bindings project, have developed > such a loader for their own bindings and made it available under the > Boost Software License. I adapted the code from their loader, and thus > put the license note in there. > > I guess I could attempt an … uhm … clean room implementation of the > loader code, but I am not sure if this would make any sense at all – the > D standard library is Boost-licensed as well, and the license should not > require any attribution from SWIG users besides not removing that notice > from the generated source files (I am no lawyer though). > But this extra license in the generated code is contrary to the SWIG license which says the generated code can be modified without restriction. You have inadvertently introduced non-conformance with the license and I don't want to get involved in providing exceptions to the license. > Basically, the Boost license was developed for (standard) libraries > which need a »do whatever you like with it, just don't claim you wrote > that source code«-style license, so I really don't see how it could be > an issue for any SWIG users, but when I finally manage to find some time > for implementing static linking, dynamically loading the wrapper library > (and thus the inclusion of this piece of code) will become fully > optional anyway… I'm not sure what the best approach to solve this is but some ideas are, admittedly none are particularly attractive: 1) Get dynamic loading of shared libraries into D proper and use this from the generated code - all the other languages have support for this so that users don't have to write platform specific code to load their modules. It's hard to believe D doesn't provide this functionality. 2) Rewrite the code for just the platforms you are interested in and expand over time. 3) Move it into a 3rd party project and require users to get it from there. 4) Obtain a very permissive license from Derelict for the code. I've been taking a close look at the D module and will post some review comments in a day or two for some minor issues. The license issue though is the only thing that is preventing incorporation into trunk for release. William |
From: David N. <co...@kl...> - 2010-11-10 18:43:51
|
On 11/9/10 8:18 AM, William S Fulton wrote: > But this extra license in the generated code is contrary to the SWIG > license which says the generated code can be modified without > restriction. You have inadvertently introduced non-conformance with the > license and I don't want to get involved in providing exceptions to the > license. Good news: I have contacted the author of the Derelict loading code, and he has generously agreed to his code being used in SWIG without any special license; I'll remove the notice from the generated code. David |
From: William S F. <ws...@fu...> - 2010-11-10 21:05:28
|
David Nadlinger wrote: > On 11/9/10 8:18 AM, William S Fulton wrote: >> But this extra license in the generated code is contrary to the SWIG >> license which says the generated code can be modified without >> restriction. You have inadvertently introduced non-conformance with the >> license and I don't want to get involved in providing exceptions to the >> license. > > Good news: I have contacted the author of the Derelict loading code, and > he has generously agreed to his code being used in SWIG without any > special license; I'll remove the notice from the generated code. Excellent. Below is the promised code review... - In namespace_class.i, something seems wrong in the library interface files if you have had to add: %warnfilter(SWIGWARN_IGNORE_OPERATOR_LT); - In smart_pointer_templatemethods.i, just change Object to Objct throughout the interface file, ie prefer not to have #ifdef SWIGD wherever possible. There are no smart_pointer_templatemethods_runme files so this won't affect any runtime tests. - super.dispose(); can go up a couple of lines into the if statement in boost_shared_ptr.i and dclassgen.swg. - The convention is not to use underscores in special variables, so suggest importtype() instead of import_type(). - The typemaps in dclasses.swg can be used for non-class types including any opaque (including primitive) type - perhaps a better name is dgeneric.swg or dswigtype.swg - it looks similar to Lib/typemaps/swigtype.swg used by the scripting languages. - %dstripprefix - this is non-standard and probably breaks %rename. Please check the normal %rename directive is a suitable replacement and remove, in particular have a look at the strip:[prefix] and regex support and %isenumitem as mentioned in SWIG.html#SWIG_advanced_renaming. Note this is relatively new documentation that wasn't around when you started. - The automatic rename in memberFunctionHandler is not how SWIG generally works... a user should provide the appropriate %rename they want, but SWIG can emit a helpful warning with an automatic ignore or error out if it knows the output won't compile. So use the same approach for functions that is used for classes in assertClassNameValidity(). - This won't compile under all C++ compilers (it is best to keep with what is used elsewhere in the source): for (unsigned int i = 0; p; i++) { - There are a couple of calls to Printf(stderr, ... which don't error out... they should be calling Swig_error() instead to provide a conformant file/line error message, as controlled by the -F command line options. - Out of source builds don't work in the test-suite, eg: mkdir ../testbuild && cd ../testbuild && ../trunk/configure && make && make check-d-test-suite. I suggest you update the Examples/test-suite/d/Makefile.in from the latest C# version and don't create an extra d1 and d2 directory. - Please remove the subdirectory creation for when -package is used - users must create this directory themselves as with other target languages. SWIG emulates a compiler in this area. Consistency observations: - Why dnativeconst and not dconst? - Why use m_swigCObject and m_swigOwnCObject instead of swigCPtr and swigCMemOwn etc? Not essential, but having these the same is just much nicer for those who use more than one target language - which is one of SWIG's strengths. - Why cwtype, dwtype, dptype instead of ctype, imtype and dtype? Note that I chose the ctype and imtype names to be language neutral, but it was too late to change the Java names to conform as the Java language had been around too long. Also the term 'intermediary' has been dropped to use 'd wrapper' instead, can't we use the common terminology? You could argue that din should then be dpin, dout => dpout etc, but I would rather see the C# style conventions used throughout. Also $dpcall should be $dcall. - Also csdestruct is the destructor wrapper and 'destruct' is a C++ (target language neutral) term, so instead of introducing a D term (dispose) in ddispose, you could use ddestruct. - Why not use $imcall instead of $wcall - using 'w' for wrapper doesn't add any information or help much here as there are wrappers at the C level, D intermdiary level and D proxy level - everything that SWIG generates at all layers is a wrapper!! - Why have example_wrap.foo() instead of just example.foo() - adding '_wrap' seem unnecessary to me. Also other languages will expect to shared object called libexample.so not libexample_wrap.so by default. Documentation: - Can you elaborate in the docs on "Support of pointers to primitive types." in the "Other Features" section? - The nativepointer attribute and cprimitive attribute for the dptype typemap need documenting, they seem to be quite crucial to the way primitive pointers work. Multiple versions: Do we need support for both d1 and d2? It might be a whole lot easier to avoid getting bogged down supporting d1 when it seems quite different to d2? Presumably both are really needed for some reason, in which case, I'd like to see the examples and test-suite work like -py3 for Python. Just an idea: Consider finding/writing something similar to the python 2to3 program which converts from version 2 to 3. Probably a simple sed script could suffice for your needs, then just one version of the runme.d files needs to be maintained. - I'd also prefer that the examples under Examples/d have a runme.1.d and runme.2.d instead of using d1/runme.d and d2/runme.d, like you have in the test-suite. If you output into the current directory it will be consistent with other languages... one day I'm going to rework the examples so the inputs are all common and I'd like the build system to be as consistent across all languages as possible to make that easier. Again, see if you copy the Python 2 and 3 approach instead for handling different versions of D. Please send me your sourceforge ID and I'll set you up with commit access. What do you think is the best way to apply the patch? Replay the changes from git, or one big commit? I'd prefer to see changes like immutable.i being renamed to immutable_values.i as a separate commit, so we can see what happened. Please don't commit any documentation changes apart from the new D chapter. I like to keep the chapter numbering changes separate to real changes as much as possible. There were some whitespace changes - please commit separately. Although I have quite a few fairly minor comments, the overall picture for your D module is excellent, it is about as advanced as all the best SWIG modules, nice work! William |
From: David N. <co...@kl...> - 2010-11-10 21:52:31
|
Thanks for your review, I will work through your list tomorrow. On 11/10/10 10:05 PM, William S Fulton wrote: > Please send me your sourceforge ID and I'll set you up with commit > access. What do you think is the best way to apply the patch? Replay > the changes from git, or one big commit? […] My sourceforge ID is »klickverbot«, as in the sender domain of this mail. Replaying the commits from Git to SVN would not really work well since I have quite a lot of tiny commits (including some overthrown concepts, etc.) – counting 357 in total, which would be way too much undesirable noise, I suppose. Thus, I would suggest merging the bulk of the changes in one big commit, with things like testcase renaming, whitespace changes, etc. in separate revisions. David |
From: David N. <co...@kl...> - 2010-11-18 23:14:38
|
I committed the code to SVN trunk a few hours ago. To (hopefully) not break the test-suite at any time, I put the renaming and special-casing changes first; I hope this doesn't cause too much confusion. As requested, the changes from regenerating the manual index are not included – should I also commit them, or do you want to wait until the next release, William? On 11/10/10 10:05 PM, William S Fulton wrote: > - In namespace_class.i, something seems wrong in the library interface > files if you have had to add: %warnfilter(SWIGWARN_IGNORE_OPERATOR_LT); The warning suppresses by that comes having »%ignoreoperator(LT) operator<;« in the lib files, because comparison operators are not implemented for D1 yet (there is a single »opCmp« for all the order relations, I'll have yet to come up with a solution to elegantly map it to the C++ operators). How do I avoid that the ignoring %rename and the one from the test-case conflict? > - In smart_pointer_templatemethods.i, just change Object to Objct > throughout the interface file, ie prefer not to have #ifdef SWIGD > wherever possible. There are no smart_pointer_templatemethods_runme > files so this won't affect any runtime tests. Done. > - super.dispose(); can go up a couple of lines into the if statement in > boost_shared_ptr.i and dclassgen.swg. Fixed. > - The convention is not to use underscores in special variables, so > suggest importtype() instead of import_type(). Fixed. > - The typemaps in dclasses.swg can be used for non-class types including > any opaque (including primitive) type - perhaps a better name is > dgeneric.swg or dswigtype.swg - it looks similar to > Lib/typemaps/swigtype.swg used by the scripting languages. I just renamed it to dswigtype.swg, thanks for the suggestion. > - %dstripprefix - this is non-standard and probably breaks %rename. > Please check the normal %rename directive is a suitable replacement and > remove, in particular have a look at the strip:[prefix] and regex > support and %isenumitem as mentioned in > SWIG.html#SWIG_advanced_renaming. Note this is relatively new > documentation that wasn't around when you started. Oh, I had somehow managed to miss %$isenumitem – I removed the feature entirely and will add a hint to the D documentation page shortly. > - The automatic rename in memberFunctionHandler is not how SWIG > generally works... a user should provide the appropriate %rename they > want, but SWIG can emit a helpful warning with an automatic ignore or > error out if it knows the output won't compile. So use the same approach > for functions that is used for classes in assertClassNameValidity(). Fixed – I haven't really used SWIG much before starting work on the D module, so thanks for calling my attention to things like this. > - This won't compile under all C++ compilers (it is best to keep with > what is used elsewhere in the source): > for (unsigned int i = 0; …; i++) { I am not sure about this one because I always thought that declaring variables in the initializer was specified legal C++ syntax – is this really not supported by some C++ compilers? Were you referring to something different? In any case, I would appreciate it if you could fix issues like this, since I have hardly any experience in dealing with non-spec-compliant compilers. > - There are a couple of calls to Printf(stderr, ... which don't error > out... they should be calling Swig_error() instead to provide a > conformant file/line error message, as controlled by the -F command line > options. Fixed, at least if I correctly understood the way it should be. > - Out of source builds don't work in the test-suite, eg: mkdir > ../testbuild && cd ../testbuild && ../trunk/configure && make && make > check-d-test-suite. Fixed, out-of-source builds now work, at least for the example you gave. If there are some other cases which still don't work, please let me know. > - Please remove the subdirectory creation for when -package is used - > users must create this directory themselves as with other target > languages. SWIG emulates a compiler in this area. Done. > Consistency observations: > - Why dnativeconst and not dconst? You are right, »dnativeconst« would not be any better than just »dconst«, because it's just as unclear as »dconst«, with the disadvantage of being inconsistent with the other languages. The thing is that one of the signature parts of D (well, D2) is its const system, similar to const correctness in C++, but much more sophisticated. The term »const« is heavily linked to a read-only view on data, not to manifest constants, which is what the feature is obviously about (manifest constants are declared with the »enum« keyword in D2). Thus, I renamed the feature to »dmanifestconst«, which better describes what it is actually about and avoids ambiguities. > - Why use m_swigCObject and m_swigOwnCObject instead of swigCPtr and > swigCMemOwn etc? Not essential, but having these the same is just much > nicer for those who use more than one target language - which is one of > SWIG's strengths. Fixed – if I remember correctly, I renamed them for some name collision issues, but as these don't apply anymore, being consistent with the other modules is obviously the better choice. > - cwtype, dwtype, dptype […] 'intermediary' […] 'd wrapper' […] ddispose While working on the D module, I was quite annoyed by having to remember what all the terms actually meant (I was still using all the names with »cs« in it from the C# module at that time), so I decided to create a clear naming scheme which mirrors the actual organization of the generated D code. Here is what I came up with: The generated code basically consists of two parts, the wrapper part, which is the C layer around the C++ library, and the proxy part, which, well, proxies the calls through the C layer. Thus, there are two types in the generated code for a given C/C++ type: the type used in the wrapper layer, and the type exposed in the proxy layer (thus, calling the C wrapper function is done via $w[rapper]call, and so on). Please note that there is nothing like a »D wrapper« – there is just a D module, called »<modulename>_wrap« by default, which is basically just a C header file in D syntax. Incidentally, this module (i.e. file in D parlance) also contains a few other helper functions, but that's just because I have not found a better place for it. There isn't something like an »intermediary« layer, you could actually link the D proxy layer and the C wrapper layer together statically (the C wrapper code could even be inlined into the proxy functions and classes with an LTO-capable linker, that's one of the reasons why I intend to add a static linking mode). If I didn't know SWIG, and would see the term »intermediary« in the context of the D module, I would probably think of the wrapper/generated/glue/… C code, since this is what is actually in between the D proxy interface and the C++ library. Thus, I chose the names »cwtype« and »dwtype« to hint at the fact that they are in fact the very same type, just with slightly different syntax in some cases. »dptype« could really be called just »ptype«, I suppose, but I found that explicitly including the language in the type name makes things clearer. As for »ddispose«: This one is a bit of a misnormer. A shortcoming in the current D object lifetime model is that destruction is completely non-deterministic and you cannot rely e.g. on objects referenced by member variables still being in a valid state. Even though D supports e..g the »scope« statement, which provides RAII-style handling of reference types, similar to e.g. »using« in C#, and thus deterministic destruction, there is no way to differentiate between the two different circumstances in the destructor. Thus, a reserved-name method called »dispose()« was added to Tango (the de-facto standard library for D1) which would be called when the object would be destructed in a deterministic fashion. Unfortunately, this never made it into the official language specification, so calling this typemap »ddestruct« would be confusing – the »destructor« in D is something different, even if it is equivalent to the notion of a »finalizer« in C#. But, as for all the topics discussed here, it's not like I would have deep feelings for my code or something, so if you think that another naming scheme would be a better fit with regard to inter-language consistency, please feel free to go ahead and fire up sed to apply it. > - Why have example_wrap.foo() instead of just example.foo() - adding > '_wrap' seem unnecessary to me. Also other languages will expect to > shared object called libexample.so not libexample_wrap.so by default. The »<modulename>_wrap« suffix is needed because just »<modulename>« is already taken by the user visible proxy module (in split proxy mode, it just contains free functions, global (manifest) constants, and the likes. As for the dynamic library name, appending a »_wrap« suffix frankly just made sense to me: Suppose you are using SWIG to generate bindings for a library called »foo«, which will probably reside in a shared object called »libfoo.so« or similar. You will probably want to name the SWIG module »foo« as well, because this is the name which is used in the D proxy layer. Now, it seemed pretty straightforward to use »<modulename>_wrap« as the default shared library name, since just »modulename« is already taken by the C++ library, »libfoo.so«, and the »_wrap« suffix hints at the fact that it contains the C wrapper code, linked to by the D module also called »<modulename>_wrap«. Sorry if this might sound ignorant, but as I have not really used SWIG before starting work on the D module: How do other languages handle this? > Documentation: > - Can you elaborate in the docs on "Support of pointers to primitive > types." […] cprimitive […]nativepointer I will add a section on native pointer support shortly (I have refrained from adding documentation for this feature up to now because I did not want to end up writing that section two or more times, as it took quite a few attempts to actually get the pointer support right). David |
From: William S F. <ws...@fu...> - 2010-11-25 19:30:04
|
David Nadlinger wrote: > I committed the code to SVN trunk a few hours ago. To (hopefully) not > break the test-suite at any time, I put the renaming and special-casing > changes first; I hope this doesn't cause too much confusion. > > As requested, the changes from regenerating the manual index are not > included – should I also commit them, or do you want to wait until the > next release, William? > Not to worry, I'll do that sometime soon. > > On 11/10/10 10:05 PM, William S Fulton wrote: >> - In namespace_class.i, something seems wrong in the library interface >> files if you have had to add: %warnfilter(SWIGWARN_IGNORE_OPERATOR_LT); > The warning suppresses by that comes having »%ignoreoperator(LT) > operator<;« in the lib files, because comparison operators are not > implemented for D1 yet (there is a single »opCmp« for all the order > relations, I'll have yet to come up with a solution to elegantly map it > to the C++ operators). How do I avoid that the ignoring %rename and the > one from the test-case conflict? > Not sure. Hopefully there is something in one of the other languages that you can copy. >> - This won't compile under all C++ compilers (it is best to keep with >> what is used elsewhere in the source): >> for (unsigned int i = 0; …; i++) { > I am not sure about this one because I always thought that declaring > variables in the initializer was specified legal C++ syntax – is this > really not supported by some C++ compilers? Were you referring to > something different? In any case, I would appreciate it if you could fix > issues like this, since I have hardly any experience in dealing with > non-spec-compliant compilers. > VC++6 is the culprit here, although to be honest, I've stopped testing with it now. >> - There are a couple of calls to Printf(stderr, ... which don't error >> out... they should be calling Swig_error() instead to provide a >> conformant file/line error message, as controlled by the -F command line >> options. > Fixed, at least if I correctly understood the way it should be. > >> Consistency observations: >> - Why dnativeconst and not dconst? > You are right, »dnativeconst« would not be any better than just > »dconst«, because it's just as unclear as »dconst«, with the > disadvantage of being inconsistent with the other languages. > > The thing is that one of the signature parts of D (well, D2) is its > const system, similar to const correctness in C++, but much more > sophisticated. The term »const« is heavily linked to a read-only view on > data, not to manifest constants, which is what the feature is obviously > about (manifest constants are declared with the »enum« keyword in D2). > > Thus, I renamed the feature to »dmanifestconst«, which better describes > what it is actually about and avoids ambiguities. > Okay. >> - cwtype, dwtype, dptype […] 'intermediary' […] 'd wrapper' […] ddispose > While working on the D module, I was quite annoyed by having to remember > what all the terms actually meant (I was still using all the names with > »cs« in it from the C# module at that time), so I decided to create a > clear naming scheme which mirrors the actual organization of the > generated D code. Here is what I came up with: > > The generated code basically consists of two parts, the wrapper part, > which is the C layer around the C++ library, and the proxy part, which, > well, proxies the calls through the C layer. Thus, there are two types > in the generated code for a given C/C++ type: the type used in the > wrapper layer, and the type exposed in the proxy layer (thus, calling > the C wrapper function is done via $w[rapper]call, and so on). > > Please note that there is nothing like a »D wrapper« – there is just a D > module, called »<modulename>_wrap« by default, which is basically just a > C header file in D syntax. Incidentally, this module (i.e. file in D > parlance) also contains a few other helper functions, but that's just > because I have not found a better place for it. There isn't something > like an »intermediary« layer, you could actually link the D proxy layer > and the C wrapper layer together statically (the C wrapper code could > even be inlined into the proxy functions and classes with an LTO-capable > linker, that's one of the reasons why I intend to add a static linking > mode). > > If I didn't know SWIG, and would see the term »intermediary« in the > context of the D module, I would probably think of the > wrapper/generated/glue/… C code, since this is what is actually in > between the D proxy interface and the C++ library. > > Thus, I chose the names »cwtype« and »dwtype« to hint at the fact that > they are in fact the very same type, just with slightly different syntax > in some cases. »dptype« could really be called just »ptype«, I suppose, > but I found that explicitly including the language in the type name > makes things clearer. > > As for »ddispose«: This one is a bit of a misnormer. A shortcoming in > the current D object lifetime model is that destruction is completely > non-deterministic and you cannot rely e.g. on objects referenced by > member variables still being in a valid state. Even though D supports > e..g the »scope« statement, which provides RAII-style handling of > reference types, similar to e.g. »using« in C#, and thus deterministic > destruction, there is no way to differentiate between the two different > circumstances in the destructor. > > Thus, a reserved-name method called »dispose()« was added to Tango (the > de-facto standard library for D1) which would be called when the object > would be destructed in a deterministic fashion. Unfortunately, this > never made it into the official language specification, so calling this > typemap »ddestruct« would be confusing – the »destructor« in D is > something different, even if it is equivalent to the notion of a > »finalizer« in C#. > > But, as for all the topics discussed here, it's not like I would have > deep feelings for my code or something, so if you think that another > naming scheme would be a better fit with regard to inter-language > consistency, please feel free to go ahead and fire up sed to apply it. > Barring the ddispose, I would prefer to have this consistent with the other languages, but don't have the inclination to edit this in. As a 2nd choice, given that it breaks with the Java/C# naming in quite a big way, I'd like to see your logic/naming convention that you have described above documented if we keep the naming as it is, especially, as it differs with the C#/Java docs which the D module refers to. >> - Why have example_wrap.foo() instead of just example.foo() - adding >> '_wrap' seem unnecessary to me. Also other languages will expect to >> shared object called libexample.so not libexample_wrap.so by default. > The »<modulename>_wrap« suffix is needed because just »<modulename>« is > already taken by the user visible proxy module (in split proxy mode, it > just contains free functions, global (manifest) constants, and the likes. > > As for the dynamic library name, appending a »_wrap« suffix frankly just > made sense to me: Suppose you are using SWIG to generate bindings for a > library called »foo«, which will probably reside in a shared object > called »libfoo.so« or similar. You will probably want to name the SWIG > module »foo« as well, because this is the name which is used in the D > proxy layer. Now, it seemed pretty straightforward to use > »<modulename>_wrap« as the default shared library name, since just > »modulename« is already taken by the C++ library, »libfoo.so«, and the > »_wrap« suffix hints at the fact that it contains the C wrapper code, > linked to by the D module also called »<modulename>_wrap«. > > Sorry if this might sound ignorant, but as I have not really used SWIG > before starting work on the D module: How do other languages handle this? > <modulenname>_wrap is only used for the C/C++ layer. The other languages take a variety of approaches for the native language layer, but none of them use '_wrap'. For example, Java uses <modulename>JNI.java for the intermediary layer (contains the Java side for the JNI code) and C# uses <modulename>PINVOKE.cs for the intermediary layer (contains the unmanaged function declaratons for PInvoke) and <modulename>.java and <modulename>.cs for globals. Python uses <modulename>.py for the proxy classes, php uses <modulename>.php for the proxy classes, Perl generates <modulename>.pl for proxy/blessed classes, Ruby does not generate any ruby code, likewise for Lua, Tcl and Octave. Thanks for explaining/addressing all the various other points. I'm yet to run the test-suite from trunk to try it out with a working copy of D, but will let you know how this goes. William |
From: William S F. <ws...@fu...> - 2010-11-27 11:58:09
|
On 25 November 2010 19:29, William S Fulton <ws...@fu...> wrote: > Thanks for explaining/addressing all the various other points. I'm yet to > run the test-suite from trunk to try it out with a working copy of D, but > will let you know how this goes. Hi David, I installed gdc on Ubuntu and get this running any of the examples/test-suite: $ make check-d-test-suite checking d test-suite checking testcase d_exception_name under d d_exception_name/d_exception_name_wrap.d:13: Error: module Exception cannot read file 'tango/core/Exception.d' Probably I need to find and install tango. However, the configure script should only enable the test-suite / examples if all the prerequisites are installed. If there is a requirement for more than just the D compiler, ie tango, the configure script needs to check that tango is installed before enabling the language. Could you add this in please else it breaks 'make check' for some. Actually, could you advise what to install? There is a libtango-ldc-dev package but that looks like it is for the LLVM version. Or maybe tango is installed and the Makefiles aren't working... how do I tell? This is what is happening in one of the examples: make[1]: Entering directory `/home/william/swig/trunk/Examples/d/class/d1' gdmd -ofrunme *.d example_wrap.d:13: Error: module Exception cannot read file 'tango/core/Exception.d' make[1]: *** [d_compile] Error 1 Thanks William |
From: William S F. <ws...@fu...> - 2010-09-27 06:01:34
|
David Nadlinger wrote: > Ah, another question: Where should I put the D-specific waring codes? > The docs have »Language module specific (800-899)«, but that space has > been exhausted with the addition of the Go module… > I've put aside 700-799 as also available for target languages, so suggest you take 700-709 or 700-719. William |
From: David N. <co...@kl...> - 2010-11-25 19:53:46
|
On 11/25/10 8:29 PM, William S Fulton wrote: > <modulenname>_wrap is only used for the C/C++ layer. The other languages > take a variety of approaches for the native language layer, but none of > them use '_wrap'. Okay, but since I have to use some postfix, at least for the wrapper/intermediary module (and there is no »natural« one, like PINVOKE for C# or JNI for java), what do you suggest? Should I just replace it with »_im«, in combination with $dwtype->$imtype, $wcall->$imcall (and the other changes you suggested) in order to make the D module easier to use for experienced SWIG users? David |