From: Vladimir K. <vka...@op...> - 2013-01-08 14:45:46
|
Hello William, I work for Open Design Alliance (www.opendesign.com) (We are not listed on the projects page, and that may be fixed. Actually, I was not aware of that page existence until recently.) We are using SWIG for quite a while now, and with some success. Last version of our product contains stable version of .NET wrappers generated by SWIG. Of course, due to the size of our API (hundreds of header files), there are some difficult places that SWIG couldn't quite handle. We use directors extensively, and there are few issues in directors support that were very limiting. The main problem is lacking "csdirectorin" "pre:" and "post" code attributes in C# module. Without them it is not trivial to marshal strings and smart-pointers back and forth between user callback code and native code. (especially by reference) I propose a patch introducing this support (with test). It also fixes 2 minor issues in director code generation that are difficult to come by until "csdirectorin" attribute is extended. The first is that "ref" types used in directors lead to invalid signature generation (the type array used to match methods possibly overloaded by user). typeof(ref T) is used instead of typeof().MakeByRefType() The second is that ignored director methods are not completely ignored - if there was a %typemap(imtype, "directorinattributes") it is not skipped for ignored method. The issue was discussed in a mailing list long ago (see e.g. http://permalink.gmane.org/gmane.comp.programming.swig/16385) I proposed a patch a year and a half ago http://sourceforge.net/p/swig/patches/268/ Attached to this mail is a git version of the patch, which is easier to review. The second problem is Unicode string literals. (Strings like L"Str" ) In our code all the strings are wchar_t based and such literals occur in headers from time to time. I proposed a workaround some time ago: http://sourceforge.net/p/swig/bugs/971/, but recently I put some work to it and implemented a full solution. See the patch attached. The wide literals may occur in default arguments and in expressions. In both cases they are correctly parsed. Wide char literals are not unescaped while parsing, because they may contain escaped characters not fitting into "char" type. If the target language wants to use them but cannot handle C escape codes, language module itself may do the necessary conversion. Usage example: %module(directors="1") MyModule %include "wchar.i" %feature("director") MyClass; %csconst(1); class MyClass { public: static const wchar_t * const myConst = L"MyWideCharConst\x234"; virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); virtual ~MyClass(); }; Generated C# wrapper code: public class MyClass : IDisposable { ... public const string myConst = "MyWideCharConst\x234"; } Generated C++ proxy code: class SwigDirector_MyClass : public MyClass, public Swig::Director { public: SwigDirector_MyClass(); virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); ... }; These cases may seem marginal to non C# users, but I think we are not unique with our requirements and these patches may benefit other users. (Of course we may continue using locally forked swig version, but it is rather inconvenient, especially for our clients, some of which may want to regenerate .net wrappers from sources.) Now, the question: Currently I'm working at the nested classes support for C# module. I already fixed parser.y, typepass and lang.cxx and working on the csharp module itself. One thing I don't understand - why was it disabled at the parser level? I mean why the pain with that weird "nestedworkaround" approach, reentering the parser (obviously not designed for reentrance)? I fear that I'm missing something essential. Who is the right person to discuss this with? (I may commit my current state of work to some remote repository to review. Not that there is much to look at, but at least the symtables are generated correctly (or so it seems)) (I didn't touch the templates yet, that is where main parsing problems are supposed to be.) -- Best regards, Vladimir Kalinin mailto:vka...@op... |
From: William S F. <ws...@fu...> - 2013-01-08 22:59:13
|
On 08/01/13 14:20, Vladimir Kalinin wrote: > Hello William, > > I work for Open Design Alliance (www.opendesign.com) > (We are not listed on the projects page, and that > may be fixed. Actually, I was not aware of that page existence until > recently.) You should be able to post a patch quite easily to the relevant page at https://github.com/swig/www/blob/master/projects.ht containing a description of Open Design Alliance that you'd like to see on the web page. > > We are using SWIG for quite a while now, and with some success. Last > version of our product contains stable version of .NET wrappers > generated by SWIG. Of course, due to the size of our API (hundreds of > header files), there are some difficult places that SWIG couldn't quite handle. > We use directors extensively, and there are few issues in directors > support that were very limiting. The main problem is lacking > "csdirectorin" "pre:" and "post" code attributes in C# module. Without them it is > not trivial to marshal strings and smart-pointers back and forth > between user callback code and native code. (especially by reference) > I propose a patch introducing this support (with test). > It also fixes 2 minor issues in director code generation that are > difficult to come by until "csdirectorin" attribute is extended. > The first is that "ref" types used in directors lead to invalid > signature generation (the type array used to match methods possibly > overloaded by user). typeof(ref T) is used instead of > typeof().MakeByRefType() > The second is that ignored director methods are not completely ignored > - if there was a %typemap(imtype, "directorinattributes") it is not > skipped for ignored method. > > The issue was discussed in a mailing list long ago (see e.g. > http://permalink.gmane.org/gmane.comp.programming.swig/16385) > I proposed a patch a year and a half ago http://sourceforge.net/p/swig/patches/268/ > Attached to this mail is a git version of the patch, which is easier > to review. Thanks very much, I'll review that very shortly. > > The second problem is Unicode string literals. (Strings like L"Str" ) > In our code all the strings are wchar_t based and such literals occur > in headers from time to time. I proposed a workaround some time ago: > http://sourceforge.net/p/swig/bugs/971/, > but recently I put some work to it and implemented a full solution. > See the patch attached. The wide literals may occur in default > arguments and in expressions. In both cases they are correctly parsed. > Wide char literals are not unescaped while parsing, because they may > contain escaped characters not fitting into "char" type. If the target > language wants to use them but cannot handle C escape codes, language > module itself may do the necessary conversion. > > Usage example: > > %module(directors="1") MyModule > %include "wchar.i" > %feature("director") MyClass; > %csconst(1); > > class MyClass > { > public: > static const wchar_t * const myConst = L"MyWideCharConst\x234"; > virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); > virtual ~MyClass(); > }; > > Generated C# wrapper code: > > public class MyClass : IDisposable { > ... > public const string myConst = "MyWideCharConst\x234"; > } > > Generated C++ proxy code: > class SwigDirector_MyClass : public MyClass, public Swig::Director { > public: > SwigDirector_MyClass(); > virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); > ... > }; > > > These cases may seem marginal to non C# users, but I think we are not unique > with our requirements and these patches may benefit other users. (Of course we may > continue using locally forked swig version, > but it is rather inconvenient, especially for our clients, some of > which may want to regenerate .net wrappers from sources.) > No doubt someone will benefit, so let's aim to get these into the main SWIG distribution, it is better for all. It might be more complicated than you wish though. This support is already in the gsoc2009-matevz branch: https://github.com/swig/swig/blob/gsoc2009-matevz/Doc/Manual - See Cpp0x.html#Cpp0x_New_string_literals. This branch is my highest priority large integration into the mainline. So could I ask you to use this branch for this particular problem and provide any additional change in this area to that branch, or wait until I've merged it into master, probably by early Feb. Aside... anyone know how to provide a direct link so that the above .html file in github is rendered in a browser in github (it was possible in svnweb). William |
From: William S F. <ws...@fu...> - 2013-01-08 22:59:18
|
On 08/01/13 14:20, Vladimir Kalinin wrote: > Now, the question: > Currently I'm working at the nested classes support for C# module. I > already fixed parser.y, typepass and lang.cxx and working on > the csharp module itself. One thing I don't understand - why was it > disabled at the parser level? I mean why the pain with that weird > "nestedworkaround" approach, reentering the parser (obviously not > designed for reentrance)? I fear that I'm missing something essential. Who is the > right person to discuss this with? (I may commit my current state of > work to some remote repository to review. Not that there is much to look at, > but at least the symtables are generated correctly (or so it seems)) > > (I didn't touch the templates yet, that is where main parsing problems are > supposed to be.) > > Nested class support will be a big step forward for SWIG, so good to hear about this. I don't know why the nested class support in the parser is a gross hack (the reentering parser bit), probably ease of implementation targeting C in the first instance. I think back in about 2000 when SWIG was going through a lot of development, this problem was put on the TODO later list, but it never got attended to. Dave Beazley probably knows more, but he isn't involved much these days. The nestedworkaround was and still is a workaround which was easily implemented. The full nested class support needn't necessarily support this feature. I'd see the full nested class support as one of the few reasons to increment the major version number and hence indicate some broken backwards compatibility for nested classes. William |
From: David B. <da...@da...> - 2013-01-08 23:35:35
|
On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: > On 08/01/13 14:20, Vladimir Kalinin wrote: >> Now, the question: >> Currently I'm working at the nested classes support for C# module. I >> already fixed parser.y, typepass and lang.cxx and working on >> the csharp module itself. One thing I don't understand - why was it >> disabled at the parser level? I mean why the pain with that weird >> "nestedworkaround" approach, reentering the parser (obviously not >> designed for reentrance)? I fear that I'm missing something essential. Who is the >> right person to discuss this with? (I may commit my current state of >> work to some remote repository to review. Not that there is much to look at, >> but at least the symtables are generated correctly (or so it seems)) >> >> (I didn't touch the templates yet, that is where main parsing problems are >> supposed to be.) >> >> > Nested class support will be a big step forward for SWIG, so good to > hear about this. I don't know why the nested class support in the parser > is a gross hack (the reentering parser bit), probably ease of > implementation targeting C in the first instance. I think back in about > 2000 when SWIG was going through a lot of development, this problem was > put on the TODO later list, but it never got attended to. Dave Beazley > probably knows more, but he isn't involved much these days. > The implementation of nested classes in SWIG has always been complicated due to two historical factors. First, many of the target languages supported by SWIG don't actually support nested classes themselves (or if they do, the support is weak/flaky). Second, the implementation of SWIG's various language modules didn't allow class code to be emitted in a manner that was reentrant. That is, if you were in the middle of emitting code for a class, you couldn't just suspend it temporarily and enter a nested code generation context. The "solution" to both of these problems was to have the parser forcefully unnest the class definitions and play some hacks with parsing and typenames. Specifically, if a nested class was found, the parser would simply capture the whole class definition body and set it aside. After the enclosing class was fully processed, the body of the nested class would then be processed as if it were a separate unnested class definition. A typedef like "typedef Outer::Inner Outer_Inner" would be made somewhere in the emitted code and the class wrapped as if it were actually called "Outer_Inner". Yes, a hack. A major reason why nested class support has never been implemented to this day is due to the fact that making it work would probably require a major refactoring of all of the target language modules to allow reentrant class wrapping. Given the large number of modules, it's a major task. I just don't think anyone has been motivated enough to tackle it. I'm not sure if this helps, but it's a bit of history about why there are some really gross hacks in there ;-). Cheers, Dave |
From: Vladimir K. <vka...@op...> - 2013-01-09 11:36:20
|
>> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>> Now, the question: >>> Currently I'm working at the nested classes support for C# module. I >>> already fixed parser.y, typepass and lang.cxx and working on >>> the csharp module itself. One thing I don't understand - why was it >>> disabled at the parser level? I mean why the pain with that weird >>> "nestedworkaround" approach, reentering the parser (obviously not >>> designed for reentrance)? I fear that I'm missing something essential. Who is the >>> right person to discuss this with? (I may commit my current state of >>> work to some remote repository to review. Not that there is much to look at, >>> but at least the symtables are generated correctly (or so it seems)) >>> >>> (I didn't touch the templates yet, that is where main parsing problems are >>> supposed to be.) >>> >>> >> Nested class support will be a big step forward for SWIG, so good to >> hear about this. I don't know why the nested class support in the parser >> is a gross hack (the reentering parser bit), probably ease of >> implementation targeting C in the first instance. I think back in about > >> 2000 when SWIG was going through a lot of development, this problem was > >> put on the TODO later list, but it never got attended to. Dave Beazley >> probably knows more, but he isn't involved much these days. >> > > The implementation of nested classes in SWIG has always been > complicated due to two historical factors. First, many of the > target languages supported by SWIG don't actually support nested > classes themselves (or if they do, the support is weak/flaky). > Second, the implementation of SWIG's various language modules didn't > allow class code to be emitted in a manner that was reentrant. That > is, if you were in the middle of emitting code for a class, you > couldn't just suspend it temporarily and enter a nested code generation context. > > The "solution" to both of these problems was to have the parser > forcefully unnest the class definitions and play some hacks with > parsing and typenames. Specifically, if a nested class was > found, the parser would simply capture the whole class definition > body and set it aside. After the enclosing class was fully > processed, the body of the nested class would then be processed as > if it were a separate unnested class definition. A typedef like > "typedef Outer::Inner Outer_Inner" would be made somewhere in the > emitted code and the class wrapped as if it were actually called "Outer_Inner". Yes, a hack. > > A major reason why nested class support has never been implemented > to this day is due to the fact that making it work would probably > require a major refactoring of all of the target language modules to > allow reentrant class wrapping. Given the large number of modules, > it's a major task. I just don't think anyone has been motivated enough to tackle it. > > I'm not sure if this helps, but it's a bit of history about why > there are some really gross hacks in there ;-). > I quite understand why the nested classes were not implemented, I just wondered why it was easier to implement the workaround on the parser level, not as a parse tree transformation. I think that will have to be done now anyway, because as you correctly point out, there are languages w/o nested classes support at all. I think it should be possible to implement the same %feature nestedworkaround in the typepass, or maybe just before it. At least some kind of feature should be introduced, because it is impractical to expect all the language modules be refactored at once. I personally know well enough only C# and perhaps Java. |
From: Vladimir K. <vka...@op...> - 2013-01-09 11:57:39
|
> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >> On 08/01/13 14:20, Vladimir Kalinin wrote: >> The second problem is Unicode string literals. (Strings like L"Str" ) >> In our code all the strings are wchar_t based and such literals occur >> in headers from time to time. I proposed a workaround some time ago: >> http://sourceforge.net/p/swig/bugs/971/, >> but recently I put some work to it and implemented a full solution. >> See the patch attached. The wide literals may occur in default >> arguments and in expressions. In both cases they are correctly parsed. >> Wide char literals are not unescaped while parsing, because they may >> contain escaped characters not fitting into "char" type. If the target >> language wants to use them but cannot handle C escape codes, language >> module itself may do the necessary conversion. >> >> Usage example: >> >> %module(directors="1") MyModule >> %include "wchar.i" >> %feature("director") MyClass; >> %csconst(1); >> >> class MyClass >> { >> public: >> static const wchar_t * const myConst = L"MyWideCharConst\x234"; >> virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); >> virtual ~MyClass(); >> }; >> >> Generated C# wrapper code: >> >> public class MyClass : IDisposable { >> ... >> public const string myConst = "MyWideCharConst\x234"; >> } >> >> Generated C++ proxy code: >> class SwigDirector_MyClass : public MyClass, public Swig::Director { >> public: >> SwigDirector_MyClass(); >> virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); >> ... >> }; >> >> >> These cases may seem marginal to non C# users, but I think we are not unique >> with our requirements and these patches may benefit other users. (Of course we may >> continue using locally forked swig version, >> but it is rather inconvenient, especially for our clients, some of >> which may want to regenerate .net wrappers from sources.) >> > No doubt someone will benefit, so let's aim to get these into the main > SWIG distribution, it is better for all. > > It might be more complicated than you wish though. This support is > already in the gsoc2009-matevz branch: > https://github.com/swig/swig/blob/gsoc2009-matevz/Doc/Manual - See > Cpp0x.html#Cpp0x_New_string_literals. This branch is my highest priority > large integration into the mainline. So could I ask you to use this > branch for this particular problem and provide any additional change in > this area to that branch, or wait until I've merged it into master, > probably by early Feb. > > Aside... anyone know how to provide a direct link so that the above > .html file in github is rendered in a browser in github (it was possible > in svnweb). I checked out that branch and it doesn't pass the test I quoted above. Anyway, if there is someone working on it, perhaps I'd better not interfere. It is unfortunate though, that corresponding bug on SF was not set to 'assigned'. That would have saved me a day's work. Vladimir |
From: David B. <da...@da...> - 2013-01-09 12:40:20
|
On Jan 9, 2013, at 5:36 AM, Vladimir Kalinin wrote: >>> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>>>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>>> Now, the question: >>>> Currently I'm working at the nested classes support for C# module. I >>>> already fixed parser.y, typepass and lang.cxx and working on >>>> the csharp module itself. One thing I don't understand - why was it >>>> disabled at the parser level? I mean why the pain with that weird >>>> "nestedworkaround" approach, reentering the parser (obviously not >>>> designed for reentrance)? I fear that I'm missing something essential. Who is the >>>> right person to discuss this with? (I may commit my current state of >>>> work to some remote repository to review. Not that there is much to look at, >>>> but at least the symtables are generated correctly (or so it seems)) >>>> >>>> (I didn't touch the templates yet, that is where main parsing problems are >>>> supposed to be.) >>>> >>>> >>> Nested class support will be a big step forward for SWIG, so good to >>> hear about this. I don't know why the nested class support in the parser >>> is a gross hack (the reentering parser bit), probably ease of >>> implementation targeting C in the first instance. I think back in about >> >>> 2000 when SWIG was going through a lot of development, this problem was >> >>> put on the TODO later list, but it never got attended to. Dave Beazley >>> probably knows more, but he isn't involved much these days. >>> >> >> The implementation of nested classes in SWIG has always been >> complicated due to two historical factors. First, many of the >> target languages supported by SWIG don't actually support nested >> classes themselves (or if they do, the support is weak/flaky). >> Second, the implementation of SWIG's various language modules didn't >> allow class code to be emitted in a manner that was reentrant. That >> is, if you were in the middle of emitting code for a class, you >> couldn't just suspend it temporarily and enter a nested code generation context. >> >> The "solution" to both of these problems was to have the parser >> forcefully unnest the class definitions and play some hacks with >> parsing and typenames. Specifically, if a nested class was >> found, the parser would simply capture the whole class definition >> body and set it aside. After the enclosing class was fully >> processed, the body of the nested class would then be processed as >> if it were a separate unnested class definition. A typedef like >> "typedef Outer::Inner Outer_Inner" would be made somewhere in the >> emitted code and the class wrapped as if it were actually called "Outer_Inner". Yes, a hack. >> >> A major reason why nested class support has never been implemented >> to this day is due to the fact that making it work would probably >> require a major refactoring of all of the target language modules to >> allow reentrant class wrapping. Given the large number of modules, >> it's a major task. I just don't think anyone has been motivated enough to tackle it. >> >> I'm not sure if this helps, but it's a bit of history about why >> there are some really gross hacks in there ;-). >> > > I quite understand why the nested classes were not implemented, I just > wondered why it was easier to implement the workaround on the parser > level, not as a parse tree transformation. > I think that will have to be done now anyway, because as you correctly > point out, there are languages w/o nested classes support at all. > I think it should be possible to implement the same %feature > nestedworkaround in the typepass, or maybe just before it. At least > some kind of feature should be introduced, because it is impractical > to expect all the language modules be refactored at once. I personally > know well enough only C# and perhaps Java. > Doing it at the parser level is probably a historical artifact. SWIG started out as a 1-pass parser that simply emitted code without ever building an intermediate parse tree. Obviously that's different now, but the original implementation of the parser still carries that baggage. There was probably no motivation to fix it given that nested classes weren't supported in the target languages anyways. Cheers, Dave |
From: <vka...@op...> - 2013-01-09 13:10:09
|
>>>On Jan 9, 2013, at 5:36 AM, Vladimir Kalinin wrote: >>>> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>>>>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>>>> Now, the question: >>>>> Currently I'm working at the nested classes support for C# module. I >>>>> already fixed parser.y, typepass and lang.cxx and working on >>>>> the csharp module itself. One thing I don't understand - why was it >>>>> disabled at the parser level? I mean why the pain with that weird >>>>> "nestedworkaround" approach, reentering the parser (obviously not >>>>> designed for reentrance)? I fear that I'm missing something essential. Who is the >>>>> right person to discuss this with? (I may commit my current state of >>>>> work to some remote repository to review. Not that there is much to look at, >>>>> but at least the symtables are generated correctly (or so it seems)) >>>>> >>>>> (I didn't touch the templates yet, that is where main parsing problems are >>>>> supposed to be.) >>>>> >>>>> >>>> Nested class support will be a big step forward for SWIG, so good to >>>> hear about this. I don't know why the nested class support in the parser >>>> is a gross hack (the reentering parser bit), probably ease of >>>> implementation targeting C in the first instance. I think back in about >>> >>>> 2000 when SWIG was going through a lot of development, this problem was >>> >>>> put on the TODO later list, but it never got attended to. Dave Beazley >>>> probably knows more, but he isn't involved much these days. >>>> >>> >>> The implementation of nested classes in SWIG has always been >>> complicated due to two historical factors. First, many of the >>> target languages supported by SWIG don't actually support nested >>> classes themselves (or if they do, the support is weak/flaky). >>> Second, the implementation of SWIG's various language modules didn't >>> allow class code to be emitted in a manner that was reentrant. That >>> is, if you were in the middle of emitting code for a class, you >>> couldn't just suspend it temporarily and enter a nested code generation context. >>> >>> The "solution" to both of these problems was to have the parser >>> forcefully unnest the class definitions and play some hacks with >>> parsing and typenames. Specifically, if a nested class was >>> found, the parser would simply capture the whole class definition >>> body and set it aside. After the enclosing class was fully >>> processed, the body of the nested class would then be processed as >>> if it were a separate unnested class definition. A typedef like >>> "typedef Outer::Inner Outer_Inner" would be made somewhere in the >>> emitted code and the class wrapped as if it were actually called "Outer_Inner". Yes, a hack. >>> >>> A major reason why nested class support has never been implemented >>> to this day is due to the fact that making it work would probably >>> require a major refactoring of all of the target language modules to >>> allow reentrant class wrapping. Given the large number of modules, >>> it's a major task. I just don't think anyone has been motivated enough to tackle it. >>> >>> I'm not sure if this helps, but it's a bit of history about why >>> there are some really gross hacks in there ;-). >>> >> >> I quite understand why the nested classes were not implemented, I just >> wondered why it was easier to implement the workaround on the parser >> level, not as a parse tree transformation. >> I think that will have to be done now anyway, because as you correctly >> point out, there are languages w/o nested classes support at all. >> I think it should be possible to implement the same %feature >> nestedworkaround in the typepass, or maybe just before it. At least >> some kind of feature should be introduced, because it is impractical >> to expect all the language modules be refactored at once. I personally >> know well enough only C# and perhaps Java. >> > > Doing it at the parser level is probably a historical artifact. > SWIG started out as a 1-pass parser that simply emitted code without > ever building an intermediate parse tree. Obviously that's > different now, but the original implementation of the parser still > carries that baggage. There was probably no motivation to fix it > given that nested classes weren't supported in the target languages anyways. > Oh, that explains it, thanks. |
From: William S F. <ws...@fu...> - 2013-01-10 22:49:46
|
On 09/01/13 11:57, Vladimir Kalinin wrote: >> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>> The second problem is Unicode string literals. (Strings like L"Str" ) >>> In our code all the strings are wchar_t based and such literals occur >>> in headers from time to time. I proposed a workaround some time ago: >>> http://sourceforge.net/p/swig/bugs/971/, >>> but recently I put some work to it and implemented a full solution. >>> See the patch attached. The wide literals may occur in default >>> arguments and in expressions. In both cases they are correctly parsed. >>> Wide char literals are not unescaped while parsing, because they may >>> contain escaped characters not fitting into "char" type. If the target >>> language wants to use them but cannot handle C escape codes, language >>> module itself may do the necessary conversion. >>> >>> Usage example: >>> >>> %module(directors="1") MyModule >>> %include "wchar.i" >>> %feature("director") MyClass; >>> %csconst(1); >>> >>> class MyClass >>> { >>> public: >>> static const wchar_t * const myConst = L"MyWideCharConst\x234"; >>> virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); >>> virtual ~MyClass(); >>> }; >>> >>> Generated C# wrapper code: >>> >>> public class MyClass : IDisposable { >>> ... >>> public const string myConst = "MyWideCharConst\x234"; >>> } >>> >>> Generated C++ proxy code: >>> class SwigDirector_MyClass : public MyClass, public Swig::Director { >>> public: >>> SwigDirector_MyClass(); >>> virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); >>> ... >>> }; >>> >>> >>> These cases may seem marginal to non C# users, but I think we are not unique >>> with our requirements and these patches may benefit other users. (Of course we may >>> continue using locally forked swig version, >>> but it is rather inconvenient, especially for our clients, some of >>> which may want to regenerate .net wrappers from sources.) >>> >> No doubt someone will benefit, so let's aim to get these into the main >> SWIG distribution, it is better for all. >> >> It might be more complicated than you wish though. This support is >> already in the gsoc2009-matevz branch: >> https://github.com/swig/swig/blob/gsoc2009-matevz/Doc/Manual - See >> Cpp0x.html#Cpp0x_New_string_literals. This branch is my highest priority >> large integration into the mainline. So could I ask you to use this >> branch for this particular problem and provide any additional change in >> this area to that branch, or wait until I've merged it into master, >> probably by early Feb. >> >> Aside... anyone know how to provide a direct link so that the above >> .html file in github is rendered in a browser in github (it was possible >> in svnweb). > > > I checked out that branch and it doesn't pass the test I quoted above. > Anyway, if there is someone working on it, perhaps I'd better not > interfere. It is unfortunate though, that corresponding bug on SF was > not set to 'assigned'. That would have saved me a day's work. > Sorry to hear that, there are way too many bugs and patches to keep track of them all. If you don't provide a patch for the gsoc2009-matevz branch, I'll merge it in once I've merged the branch into master. William |
From: William S F. <ws...@fu...> - 2013-01-10 22:54:25
|
On 08/01/13 22:59, William S Fulton wrote: > On 08/01/13 14:20, Vladimir Kalinin wrote: >> >> We are using SWIG for quite a while now, and with some success. Last >> version of our product contains stable version of .NET wrappers >> generated by SWIG. Of course, due to the size of our API (hundreds of >> header files), there are some difficult places that SWIG couldn't >> quite handle. >> We use directors extensively, and there are few issues in directors >> support that were very limiting. The main problem is lacking >> "csdirectorin" "pre:" and "post" code attributes in C# module. Without >> them it is >> not trivial to marshal strings and smart-pointers back and forth >> between user callback code and native code. (especially by reference) >> I propose a patch introducing this support (with test). >> It also fixes 2 minor issues in director code generation that are >> difficult to come by until "csdirectorin" attribute is extended. >> The first is that "ref" types used in directors lead to invalid >> signature generation (the type array used to match methods possibly >> overloaded by user). typeof(ref T) is used instead of >> typeof().MakeByRefType() >> The second is that ignored director methods are not completely ignored >> - if there was a %typemap(imtype, "directorinattributes") it is not >> skipped for ignored method. >> >> The issue was discussed in a mailing list long ago (see e.g. >> http://permalink.gmane.org/gmane.comp.programming.swig/16385) >> I proposed a patch a year and a half ago >> http://sourceforge.net/p/swig/patches/268/ >> Attached to this mail is a git version of the patch, which is easier >> to review. > Thanks very much, I'll review that very shortly. Great patch. All looks good except for some inconsistent spacing in the generated code. I'll commit this tomorrow and fix up the spacing. The only way it could be further improved is with an example in the docs. William |
From: Vladimir K. <vka...@op...> - 2013-01-10 23:21:37
|
> On 08/01/13 22:59, William S Fulton wrote: >> On 08/01/13 14:20, Vladimir Kalinin wrote: > >>> >>> We are using SWIG for quite a while now, and with some success. Last >>> version of our product contains stable version of .NET wrappers >>> generated by SWIG. Of course, due to the size of our API (hundreds of >>> header files), there are some difficult places that SWIG couldn't >>> quite handle. >>> We use directors extensively, and there are few issues in directors >>> support that were very limiting. The main problem is lacking >>> "csdirectorin" "pre:" and "post" code attributes in C# module. Without >>> them it is >>> not trivial to marshal strings and smart-pointers back and forth >>> between user callback code and native code. (especially by reference) >>> I propose a patch introducing this support (with test). >>> It also fixes 2 minor issues in director code generation that are >>> difficult to come by until "csdirectorin" attribute is extended. >>> The first is that "ref" types used in directors lead to invalid >>> signature generation (the type array used to match methods possibly >>> overloaded by user). typeof(ref T) is used instead of >>> typeof().MakeByRefType() >>> The second is that ignored director methods are not completely ignored >>> - if there was a %typemap(imtype, "directorinattributes") it is not >>> skipped for ignored method. >>> >>> The issue was discussed in a mailing list long ago (see e.g. >>> http://permalink.gmane.org/gmane.comp.programming.swig/16385) >>> I proposed a patch a year and a half ago >>> http://sourceforge.net/p/swig/patches/268/ >>> Attached to this mail is a git version of the patch, which is easier >>> to review. >> Thanks very much, I'll review that very shortly. > Great patch. All looks good except for some inconsistent spacing in the > generated code. I'll commit this tomorrow and fix up the spacing. The > only way it could be further improved is with an example in the docs. Oh, right, the documentation, I quite missed that out, I'll compose something. (The spacing is wrong probably because at the time of writing (2 years ago) I didn't realize that the tab size is 8 spaces in SWIG code. I fixed that in the code itself recently, but forgot to check the generated code. Thanks for taking care of that.) Vladimir |
From: Vladimir K. <vka...@op...> - 2013-01-10 23:57:03
|
> On 09/01/13 11:57, Vladimir Kalinin wrote: >>> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>>> The second problem is Unicode string literals. (Strings like L"Str" ) >>>> In our code all the strings are wchar_t based and such literals occur >>>> in headers from time to time. I proposed a workaround some time ago: >>>> http://sourceforge.net/p/swig/bugs/971/, >>>> but recently I put some work to it and implemented a full solution. >>>> See the patch attached. The wide literals may occur in default >>>> arguments and in expressions. In both cases they are correctly parsed. >>>> Wide char literals are not unescaped while parsing, because they may >>>> contain escaped characters not fitting into "char" type. If the target >>>> language wants to use them but cannot handle C escape codes, language >>>> module itself may do the necessary conversion. >>>> >>>> Usage example: >>>> >>>> %module(directors="1") MyModule >>>> %include "wchar.i" >>>> %feature("director") MyClass; >>>> %csconst(1); >>>> >>>> class MyClass >>>> { >>>> public: >>>> static const wchar_t * const myConst = L"MyWideCharConst\x234"; >>>> virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); >>>> virtual ~MyClass(); >>>> }; >>>> >>>> Generated C# wrapper code: >>>> >>>> public class MyClass : IDisposable { >>>> ... >>>> public const string myConst = "MyWideCharConst\x234"; >>>> } >>>> >>>> Generated C++ proxy code: >>>> class SwigDirector_MyClass : public MyClass, public Swig::Director { >>>> public: >>>> SwigDirector_MyClass(); >>>> virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); >>>> ... >>>> }; >>>> >>>> >>>> These cases may seem marginal to non C# users, but I think we are not unique >>>> with our requirements and these patches may benefit other users. (Of course we may >>>> continue using locally forked swig version, >>>> but it is rather inconvenient, especially for our clients, some of >>>> which may want to regenerate .net wrappers from sources.) >>>> >>> No doubt someone will benefit, so let's aim to get these into the main >>> SWIG distribution, it is better for all. >>> >>> It might be more complicated than you wish though. This support is >>> already in the gsoc2009-matevz branch: >>> https://github.com/swig/swig/blob/gsoc2009-matevz/Doc/Manual - See >>> Cpp0x.html#Cpp0x_New_string_literals. This branch is my highest priority >>> large integration into the mainline. So could I ask you to use this >>> branch for this particular problem and provide any additional change in >>> this area to that branch, or wait until I've merged it into master, >>> probably by early Feb. >>> >>> Aside... anyone know how to provide a direct link so that the above >>> .html file in github is rendered in a browser in github (it was possible >>> in svnweb). >> >> >> I checked out that branch and it doesn't pass the test I quoted above. >> Anyway, if there is someone working on it, perhaps I'd better not >> interfere. It is unfortunate though, that corresponding bug on SF was >> not set to 'assigned'. That would have saved me a day's work. >> > Sorry to hear that, there are way too many bugs and patches to keep > track of them all. If you don't provide a patch for the gsoc2009-matevz > branch, I'll merge it in once I've merged the branch into master. What is the status of that branch? I mean - is someone still working on it or it is considered finished? Because, as you understand, it is a lot harder to fix someone's code than to write one's own. If there is an author available, I can show him a failing test case, and he may probably fix what needs to be fixed, or explain why it is not necessary/possible, or just say that he doesn't have the time, and then I'll have to fix it myself (after nested classes). Actually this is the full class I used for debugging wide char literals: struct MyClass { static const wchar_t * const myConst = L"MyWideCharCont\x234"; static const wchar_t myConstI = L'\34'; virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); virtual void myFunc1(const char* myString = "MyCharLiteral\x12"); virtual void myFunc2(wchar_t myChar = L'\x1234'); // virtual void myFunc3(int myInt = L'\x1334'); virtual ~MyClass(); }; Commented out function reveals an unrelated bug in default parameters implementation - we don't retain deducted types of the default values and expressions. E.g. the constant: static const int x = '1'; would produce the wrapper static const int x = 1; Perhaps I should enter a bug about it, but as I understand bug tracking is in flux now, so this (rather insignificant) issue may wait. Vladimir |
From: Vladimir K. <vka...@op...> - 2013-01-11 17:46:54
|
> On 08/01/13 22:59, William S Fulton wrote: >> On 08/01/13 14:20, Vladimir Kalinin wrote: > >>> >>> We are using SWIG for quite a while now, and with some success. Last >>> version of our product contains stable version of .NET wrappers >>> generated by SWIG. Of course, due to the size of our API (hundreds of >>> header files), there are some difficult places that SWIG couldn't >>> quite handle. >>> We use directors extensively, and there are few issues in directors >>> support that were very limiting. The main problem is lacking >>> "csdirectorin" "pre:" and "post" code attributes in C# module. Without >>> them it is >>> not trivial to marshal strings and smart-pointers back and forth >>> between user callback code and native code. (especially by reference) >>> I propose a patch introducing this support (with test). >>> It also fixes 2 minor issues in director code generation that are >>> difficult to come by until "csdirectorin" attribute is extended. >>> The first is that "ref" types used in directors lead to invalid >>> signature generation (the type array used to match methods possibly >>> overloaded by user). typeof(ref T) is used instead of >>> typeof().MakeByRefType() >>> The second is that ignored director methods are not completely ignored >>> - if there was a %typemap(imtype, "directorinattributes") it is not >>> skipped for ignored method. >>> >>> The issue was discussed in a mailing list long ago (see e.g. >>> http://permalink.gmane.org/gmane.comp.programming.swig/16385) >>> I proposed a patch a year and a half ago >>> http://sourceforge.net/p/swig/patches/268/ >>> Attached to this mail is a git version of the patch, which is easier >>> to review. >> Thanks very much, I'll review that very shortly. > Great patch. All looks good except for some inconsistent spacing in the > generated code. I'll commit this tomorrow and fix up the spacing. The > only way it could be further improved is with an example in the docs. Attached is a proposed documentation patch for csdirectorin changes. Vladimir |
From: William S F. <ws...@fu...> - 2013-01-11 22:24:40
|
On 11/01/13 17:46, Vladimir Kalinin wrote: >> On 08/01/13 22:59, William S Fulton wrote: >>> On 08/01/13 14:20, Vladimir Kalinin wrote: >> >>>> >>>> We are using SWIG for quite a while now, and with some success. Last >>>> version of our product contains stable version of .NET wrappers >>>> generated by SWIG. Of course, due to the size of our API (hundreds of >>>> header files), there are some difficult places that SWIG couldn't >>>> quite handle. >>>> We use directors extensively, and there are few issues in directors >>>> support that were very limiting. The main problem is lacking >>>> "csdirectorin" "pre:" and "post" code attributes in C# module. Without >>>> them it is >>>> not trivial to marshal strings and smart-pointers back and forth >>>> between user callback code and native code. (especially by reference) >>>> I propose a patch introducing this support (with test). >>>> It also fixes 2 minor issues in director code generation that are >>>> difficult to come by until "csdirectorin" attribute is extended. >>>> The first is that "ref" types used in directors lead to invalid >>>> signature generation (the type array used to match methods possibly >>>> overloaded by user). typeof(ref T) is used instead of >>>> typeof().MakeByRefType() >>>> The second is that ignored director methods are not completely ignored >>>> - if there was a %typemap(imtype, "directorinattributes") it is not >>>> skipped for ignored method. >>>> >>>> The issue was discussed in a mailing list long ago (see e.g. >>>> http://permalink.gmane.org/gmane.comp.programming.swig/16385) >>>> I proposed a patch a year and a half ago >>>> http://sourceforge.net/p/swig/patches/268/ >>>> Attached to this mail is a git version of the patch, which is easier >>>> to review. >>> Thanks very much, I'll review that very shortly. >> Great patch. All looks good except for some inconsistent spacing in the >> generated code. I'll commit this tomorrow and fix up the spacing. The >> only way it could be further improved is with an example in the docs. > > Attached is a proposed documentation patch for csdirectorin changes. > Superb thanks. I've applied and tweaked a bit and also pushed your code patch to master. William |
From: William S F. <ws...@fu...> - 2013-01-12 23:15:05
|
On 10/01/13 23:56, Vladimir Kalinin wrote: >> On 09/01/13 11:57, Vladimir Kalinin wrote: >>>> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>>>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>>>> The second problem is Unicode string literals. (Strings like L"Str" ) >>>>> In our code all the strings are wchar_t based and such literals occur >>>>> in headers from time to time. I proposed a workaround some time ago: >>>>> http://sourceforge.net/p/swig/bugs/971/, >>>>> but recently I put some work to it and implemented a full solution. >>>>> See the patch attached. The wide literals may occur in default >>>>> arguments and in expressions. In both cases they are correctly parsed. >>>>> Wide char literals are not unescaped while parsing, because they may >>>>> contain escaped characters not fitting into "char" type. If the target >>>>> language wants to use them but cannot handle C escape codes, language >>>>> module itself may do the necessary conversion. >>>>> >>>>> Usage example: >>>>> >>>>> %module(directors="1") MyModule >>>>> %include "wchar.i" >>>>> %feature("director") MyClass; >>>>> %csconst(1); >>>>> >>>>> class MyClass >>>>> { >>>>> public: >>>>> static const wchar_t * const myConst = L"MyWideCharConst\x234"; >>>>> virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); >>>>> virtual ~MyClass(); >>>>> }; >>>>> >>>>> Generated C# wrapper code: >>>>> >>>>> public class MyClass : IDisposable { >>>>> ... >>>>> public const string myConst = "MyWideCharConst\x234"; >>>>> } >>>>> >>>>> Generated C++ proxy code: >>>>> class SwigDirector_MyClass : public MyClass, public Swig::Director { >>>>> public: >>>>> SwigDirector_MyClass(); >>>>> virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); >>>>> ... >>>>> }; >>>>> >>>>> >>>>> These cases may seem marginal to non C# users, but I think we are not unique >>>>> with our requirements and these patches may benefit other users. (Of course we may >>>>> continue using locally forked swig version, >>>>> but it is rather inconvenient, especially for our clients, some of >>>>> which may want to regenerate .net wrappers from sources.) >>>>> >>>> No doubt someone will benefit, so let's aim to get these into the main >>>> SWIG distribution, it is better for all. >>>> >>>> It might be more complicated than you wish though. This support is >>>> already in the gsoc2009-matevz branch: >>>> https://github.com/swig/swig/blob/gsoc2009-matevz/Doc/Manual - See >>>> Cpp0x.html#Cpp0x_New_string_literals. This branch is my highest priority >>>> large integration into the mainline. So could I ask you to use this >>>> branch for this particular problem and provide any additional change in >>>> this area to that branch, or wait until I've merged it into master, >>>> probably by early Feb. >>>> >>>> Aside... anyone know how to provide a direct link so that the above >>>> .html file in github is rendered in a browser in github (it was possible >>>> in svnweb). >>> >>> >>> I checked out that branch and it doesn't pass the test I quoted above. >>> Anyway, if there is someone working on it, perhaps I'd better not >>> interfere. It is unfortunate though, that corresponding bug on SF was >>> not set to 'assigned'. That would have saved me a day's work. >>> >> Sorry to hear that, there are way too many bugs and patches to keep >> track of them all. If you don't provide a patch for the gsoc2009-matevz >> branch, I'll merge it in once I've merged the branch into master. > > What is the status of that branch? I mean - is someone still working > on it or it is considered finished? A large bulk of the work was done by a summer of code student who hasn't stayed around. I mentored him so have a fair idea of what is going on and have since improved the work. I don't recall much in that particular area, but it really is one of my top priorities to get this work merged into master. There are a lot of new functionality in this branch and it does require some polishing off though before ready for release. > Because, as you understand, it is a lot harder to fix someone's > code than to write one's own. If there is an author available, I can > show him a failing test case, and he may probably fix what needs to be > fixed, or explain why it is not necessary/possible, or just say that he > doesn't have the time, and then I'll have to fix it myself (after > nested classes). > > Actually this is the full class I used for debugging wide char > literals: > > struct MyClass > { > static const wchar_t * const myConst = L"MyWideCharCont\x234"; > static const wchar_t myConstI = L'\34'; > virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); > virtual void myFunc1(const char* myString = "MyCharLiteral\x12"); > virtual void myFunc2(wchar_t myChar = L'\x1234'); > // virtual void myFunc3(int myInt = L'\x1334'); > virtual ~MyClass(); > }; > > Commented out function reveals an unrelated bug in default parameters > implementation - we don't retain deducted types of the default values > and expressions. E.g. the constant: > > static const int x = '1'; > > would produce the wrapper > > static const int x = 1; > > Perhaps I should enter a bug about it, but as I understand > bug tracking is in flux now, so this (rather insignificant) issue may > wait. You can still use SF while the bug tracking is sorted out, and best that you do else this will get forgotten about. William |
From: Vladimir K. <vka...@op...> - 2013-01-15 21:54:05
|
>>>>> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>>>>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>>>>> The second problem is Unicode string literals. (Strings like L"Str" ) >>>>>> In our code all the strings are wchar_t based and such literals occur >>>>>> in headers from time to time. I proposed a workaround some time ago: >>>>>> http://sourceforge.net/p/swig/bugs/971/, >>>>>> but recently I put some work to it and implemented a full solution. >>>>>> See the patch attached. The wide literals may occur in default >>>>>> arguments and in expressions. In both cases they are correctly parsed. >>>>>> Wide char literals are not unescaped while parsing, because they may >>>>>> contain escaped characters not fitting into "char" type. If the target >>>>>> language wants to use them but cannot handle C escape codes, language >>>>>> module itself may do the necessary conversion. >>>>>> >>>>>> Usage example: >>>>>> >>>>>> %module(directors="1") MyModule >>>>>> %include "wchar.i" >>>>>> %feature("director") MyClass; >>>>>> %csconst(1); >>>>>> >>>>>> class MyClass >>>>>> { >>>>>> public: >>>>>> static const wchar_t * const myConst = L"MyWideCharConst\x234"; >>>>>> virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); >>>>>> virtual ~MyClass(); >>>>>> }; >>>>>> >>>>>> Generated C# wrapper code: >>>>>> >>>>>> public class MyClass : IDisposable { >>>>>> ... >>>>>> public const string myConst = "MyWideCharConst\x234"; >>>>>> } >>>>>> >>>>>> Generated C++ proxy code: >>>>>> class SwigDirector_MyClass : public MyClass, public Swig::Director { >>>>>> public: >>>>>> SwigDirector_MyClass(); >>>>>> virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); >>>>>> ... >>>>>> }; >>>>>> >>>>>> >>>>>> These cases may seem marginal to non C# users, but I think we are not unique >>>>>> with our requirements and these patches may benefit other users. (Of course we may >>>>>> continue using locally forked swig version, >>>>>> but it is rather inconvenient, especially for our clients, some of >>>>>> which may want to regenerate .net wrappers from sources.) >>>>>> >>>>> No doubt someone will benefit, so let's aim to get these into the main >>>>> SWIG distribution, it is better for all. >>>>> >>>>> It might be more complicated than you wish though. This support is >>>>> already in the gsoc2009-matevz branch: >>>>> https://github.com/swig/swig/blob/gsoc2009-matevz/Doc/Manual - See >>>>> Cpp0x.html#Cpp0x_New_string_literals. This branch is my highest priority >>>>> large integration into the mainline. So could I ask you to use this >>>>> branch for this particular problem and provide any additional change in >>>>> this area to that branch, or wait until I've merged it into master, >>>>> probably by early Feb. >>>>> >>>>> Aside... anyone know how to provide a direct link so that the above >>>>> .html file in github is rendered in a browser in github (it was possible >>>>> in svnweb). >>>> >>>> >>>> I checked out that branch and it doesn't pass the test I quoted above. >>>> Anyway, if there is someone working on it, perhaps I'd better not >>>> interfere. It is unfortunate though, that corresponding bug on SF was >>>> not set to 'assigned'. That would have saved me a day's work. >>>> >>> Sorry to hear that, there are way too many bugs and patches to keep >>> track of them all. If you don't provide a patch for the gsoc2009-matevz >>> branch, I'll merge it in once I've merged the branch into master. >> >> What is the status of that branch? I mean - is someone still working >> on it or it is considered finished? > A large bulk of the work was done by a summer of code student who hasn't > stayed around. I mentored him so have a fair idea of what is going on > and have since improved the work. I don't recall much in that particular > area, but it really is one of my top priorities to get this work merged > into master. There are a lot of new functionality in this branch and it > does require some polishing off though before ready for release. I reviewed the code changes related to Unicode strings support, and found out that in gsoc2009-matevz there was added support for custom delimiters for string literals and wide char literal is regarded as a particular case. Custom delimited string is converted to normal string while parsing. This approach does not work with Unicode literals, unless they are converted to UTF-8, and even in this case we will lose deducted literal type (it will become char* instead of wchar_t*). Unicode string is Unicode not in the name only - it really can contain wide characters (as escape sequences), and they will not easily fit into char*. I suggest to leave custom delimited strings implementation as is, and merge my implementation of Unicode (L"") strings to this branch, treating Unicode strings separately. I can do the merging if you like. Vladimir |
From: William S F. <ws...@fu...> - 2013-01-15 23:34:03
|
On 15/01/13 21:53, Vladimir Kalinin wrote: >>>>>> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>>>>>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>>>>>> The second problem is Unicode string literals. (Strings like L"Str" ) >>>>>>> In our code all the strings are wchar_t based and such literals occur >>>>>>> in headers from time to time. I proposed a workaround some time ago: >>>>>>> http://sourceforge.net/p/swig/bugs/971/, >>>>>>> but recently I put some work to it and implemented a full solution. >>>>>>> See the patch attached. The wide literals may occur in default >>>>>>> arguments and in expressions. In both cases they are correctly parsed. >>>>>>> Wide char literals are not unescaped while parsing, because they may >>>>>>> contain escaped characters not fitting into "char" type. If the target >>>>>>> language wants to use them but cannot handle C escape codes, language >>>>>>> module itself may do the necessary conversion. >>>>>>> >>>>>>> Usage example: >>>>>>> >>>>>>> %module(directors="1") MyModule >>>>>>> %include "wchar.i" >>>>>>> %feature("director") MyClass; >>>>>>> %csconst(1); >>>>>>> >>>>>>> class MyClass >>>>>>> { >>>>>>> public: >>>>>>> static const wchar_t * const myConst = L"MyWideCharConst\x234"; >>>>>>> virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); >>>>>>> virtual ~MyClass(); >>>>>>> }; >>>>>>> >>>>>>> Generated C# wrapper code: >>>>>>> >>>>>>> public class MyClass : IDisposable { >>>>>>> ... >>>>>>> public const string myConst = "MyWideCharConst\x234"; >>>>>>> } >>>>>>> >>>>>>> Generated C++ proxy code: >>>>>>> class SwigDirector_MyClass : public MyClass, public Swig::Director { >>>>>>> public: >>>>>>> SwigDirector_MyClass(); >>>>>>> virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); >>>>>>> ... >>>>>>> }; >>>>>>> >>>>>>> >>>>>>> These cases may seem marginal to non C# users, but I think we are not unique >>>>>>> with our requirements and these patches may benefit other users. (Of course we may >>>>>>> continue using locally forked swig version, >>>>>>> but it is rather inconvenient, especially for our clients, some of >>>>>>> which may want to regenerate .net wrappers from sources.) >>>>>>> >>>>>> No doubt someone will benefit, so let's aim to get these into the main >>>>>> SWIG distribution, it is better for all. >>>>>> >>>>>> It might be more complicated than you wish though. This support is >>>>>> already in the gsoc2009-matevz branch: >>>>>> https://github.com/swig/swig/blob/gsoc2009-matevz/Doc/Manual - See >>>>>> Cpp0x.html#Cpp0x_New_string_literals. This branch is my highest priority >>>>>> large integration into the mainline. So could I ask you to use this >>>>>> branch for this particular problem and provide any additional change in >>>>>> this area to that branch, or wait until I've merged it into master, >>>>>> probably by early Feb. >>>>>> >>>>>> Aside... anyone know how to provide a direct link so that the above >>>>>> .html file in github is rendered in a browser in github (it was possible >>>>>> in svnweb). >>>>> >>>>> >>>>> I checked out that branch and it doesn't pass the test I quoted above. >>>>> Anyway, if there is someone working on it, perhaps I'd better not >>>>> interfere. It is unfortunate though, that corresponding bug on SF was >>>>> not set to 'assigned'. That would have saved me a day's work. >>>>> >>>> Sorry to hear that, there are way too many bugs and patches to keep >>>> track of them all. If you don't provide a patch for the gsoc2009-matevz >>>> branch, I'll merge it in once I've merged the branch into master. >>> >>> What is the status of that branch? I mean - is someone still working >>> on it or it is considered finished? >> A large bulk of the work was done by a summer of code student who hasn't >> stayed around. I mentored him so have a fair idea of what is going on >> and have since improved the work. I don't recall much in that particular >> area, but it really is one of my top priorities to get this work merged >> into master. There are a lot of new functionality in this branch and it >> does require some polishing off though before ready for release. > > I reviewed the code changes related to Unicode strings support, and > found out that in gsoc2009-matevz there was added support for custom > delimiters for string literals and wide char literal is regarded as a > particular case. Custom delimited string is converted to normal string > while parsing. > > This approach does not work with Unicode literals, unless they are > converted to UTF-8, and even in this case we will lose deducted > literal type (it will become char* instead of wchar_t*). > Unicode string is Unicode not in the name only - it really can contain > wide characters (as escape sequences), and they will not easily fit > into char*. > > I suggest to leave custom delimited strings implementation as is, and > merge my implementation of Unicode (L"") strings to this branch, > treating Unicode strings separately. > Yes good points. > I can do the merging if you like. I would very much appreciate that. My main problem with that branch is the rvalue references, but I think I have a plan for that now. Once that is implemented and after a bit of tidy up, it'll be ready for merge to master. Probably I will do a new release just before the merge and then one one with the branch. Hopefully over the next month. William |
From: Vladimir K. <vka...@op...> - 2013-01-17 18:32:36
Attachments:
0001-Unicode-literals.patch
|
> On 15/01/13 21:53, Vladimir Kalinin wrote: >>>>>>> On Jan 8, 2013, at 4:59 PM, William S Fulton wrote: >>>>>>>> On 08/01/13 14:20, Vladimir Kalinin wrote: >>>>>>>> The second problem is Unicode string literals. (Strings like L"Str" ) >>>>>>>> In our code all the strings are wchar_t based and such literals occur >>>>>>>> in headers from time to time. I proposed a workaround some time ago: >>>>>>>> http://sourceforge.net/p/swig/bugs/971/, >>>>>>>> but recently I put some work to it and implemented a full solution. >>>>>>>> See the patch attached. The wide literals may occur in default >>>>>>>> arguments and in expressions. In both cases they are correctly parsed. >>>>>>>> Wide char literals are not unescaped while parsing, because they may >>>>>>>> contain escaped characters not fitting into "char" type. If the target >>>>>>>> language wants to use them but cannot handle C escape codes, language >>>>>>>> module itself may do the necessary conversion. >>>>>>>> >>>>>>>> Usage example: >>>>>>>> >>>>>>>> %module(directors="1") MyModule >>>>>>>> %include "wchar.i" >>>>>>>> %feature("director") MyClass; >>>>>>>> %csconst(1); >>>>>>>> >>>>>>>> class MyClass >>>>>>>> { >>>>>>>> public: >>>>>>>> static const wchar_t * const myConst = L"MyWideCharConst\x234"; >>>>>>>> virtual void myFunc(const wchar_t* myString = L"MyWideChar\x1234Literal\555"); >>>>>>>> virtual ~MyClass(); >>>>>>>> }; >>>>>>>> >>>>>>>> Generated C# wrapper code: >>>>>>>> >>>>>>>> public class MyClass : IDisposable { >>>>>>>> ... >>>>>>>> public const string myConst = "MyWideCharConst\x234"; >>>>>>>> } >>>>>>>> >>>>>>>> Generated C++ proxy code: >>>>>>>> class SwigDirector_MyClass : public MyClass, public Swig::Director { >>>>>>>> public: >>>>>>>> SwigDirector_MyClass(); >>>>>>>> virtual void myFunc(wchar_t const *myString = L"MyWideChar\x1234Literal\555"); >>>>>>>> ... >>>>>>>> }; >>>>>>>> >>>>>>>> >>>>>>>> These cases may seem marginal to non C# users, but I think we are not unique >>>>>>>> with our requirements and these patches may benefit other users. (Of course we may >>>>>>>> continue using locally forked swig version, >>>>>>>> but it is rather inconvenient, especially for our clients, some of >>>>>>>> which may want to regenerate .net wrappers from sources.) >>>>>>>> >>>>>>> No doubt someone will benefit, so let's aim to get these into the main >>>>>>> SWIG distribution, it is better for all. >>>>>>> >>>>>>> It might be more complicated than you wish though. This support is >>>>>>> already in the gsoc2009-matevz branch: >>>>>>> https://github.com/swig/swig/blob/gsoc2009-matevz/Doc/Manual - See >>>>>>> Cpp0x.html#Cpp0x_New_string_literals. This branch is my highest priority >>>>>>> large integration into the mainline. So could I ask you to use this >>>>>>> branch for this particular problem and provide any additional change in >>>>>>> this area to that branch, or wait until I've merged it into master, >>>>>>> probably by early Feb. >>>>>>> >>>>>>> Aside... anyone know how to provide a direct link so that the above >>>>>>> .html file in github is rendered in a browser in github (it was possible >>>>>>> in svnweb). >>>>>> >>>>>> >>>>>> I checked out that branch and it doesn't pass the test I quoted above. >>>>>> Anyway, if there is someone working on it, perhaps I'd better not >>>>>> interfere. It is unfortunate though, that corresponding bug on SF was >>>>>> not set to 'assigned'. That would have saved me a day's work. >>>>>> >>>>> Sorry to hear that, there are way too many bugs and patches to keep >>>>> track of them all. If you don't provide a patch for the gsoc2009-matevz >>>>> branch, I'll merge it in once I've merged the branch into master. >>>> >>>> What is the status of that branch? I mean - is someone still working >>>> on it or it is considered finished? >>> A large bulk of the work was done by a summer of code student who hasn't >>> stayed around. I mentored him so have a fair idea of what is going on >>> and have since improved the work. I don't recall much in that particular >>> area, but it really is one of my top priorities to get this work merged >>> into master. There are a lot of new functionality in this branch and it >>> does require some polishing off though before ready for release. >> >> I reviewed the code changes related to Unicode strings support, and >> found out that in gsoc2009-matevz there was added support for custom >> delimiters for string literals and wide char literal is regarded as a >> particular case. Custom delimited string is converted to normal string >> while parsing. >> >> This approach does not work with Unicode literals, unless they are >> converted to UTF-8, and even in this case we will lose deducted >> literal type (it will become char* instead of wchar_t*). >> Unicode string is Unicode not in the name only - it really can contain >> wide characters (as escape sequences), and they will not easily fit >> into char*. >> >> I suggest to leave custom delimited strings implementation as is, and >> merge my implementation of Unicode (L"") strings to this branch, >> treating Unicode strings separately. >> > Yes good points. > >> I can do the merging if you like. > I would very much appreciate that. My main problem with that branch is > the rvalue references, but I think I have a plan for that now. Once that > is implemented and after a bit of tidy up, it'll be ready for merge to > master. Probably I will do a new release just before the merge and then > one one with the branch. Hopefully over the next month. The patch for the "gsoc2009-matevz" branch is attached. BTW, wchar_t* typemaps in Lib/csharp/wchar.i appear to be unfinished: SWIG_csharp_wstring_callback is declared but never used in actual typemaps. (I used my own typemaps for patch testing) Vladimir |
From: William S F. <ws...@fu...> - 2013-01-21 19:37:06
|
> >> I can do the merging if you like. > > I would very much appreciate that. My main problem with that branch is > > the rvalue references, but I think I have a plan for that now. Once that > > is implemented and after a bit of tidy up, it'll be ready for merge to > > master. Probably I will do a new release just before the merge and then > > one one with the branch. Hopefully over the next month. > > The patch for the "gsoc2009-matevz" branch is attached. > > BTW, wchar_t* typemaps in Lib/csharp/wchar.i appear to be unfinished: > SWIG_csharp_wstring_callback is declared but never used in actual > typemaps. (I used my own typemaps for patch testing) > > Vladimir > Hi Vladimir I'm just looking at this patch and it added %expect in parser.y +%expect 6 and apparently from this yacc man page: http://invisible-island.net/byacc/manpage/yacc.html it is broken in some versions of bison. Also it won't be supported in some older versions of yacc, so we'll have to check for min versions of these programs. Can you get this information as to the min versions required, otherwise, I'll drop this part of the patch as the last thing we want is it not work on some peoples machines without useful errors, preferably at configure time. Meanwhile, thanks and I'm trying out the rest of the patch... William |
From: Vladimir K. <vka...@op...> - 2013-01-21 20:43:53
|
>> >> I can do the merging if you like. >> > I would very much appreciate that. My main problem with that branch is >> > the rvalue references, but I think I have a plan for that now. Once that >> > is implemented and after a bit of tidy up, it'll be ready for merge to >> > master. Probably I will do a new release just before the merge and then >> > one one with the branch. Hopefully over the next month. >> >> The patch for the "gsoc2009-matevz" branch is attached. >> >> BTW, wchar_t* typemaps in Lib/csharp/wchar.i appear to be unfinished: >> SWIG_csharp_wstring_callback is declared but never used in actual >> typemaps. (I used my own typemaps for patch testing) >> >> Vladimir >> > > Hi Vladimir > > I'm just looking at this patch and it added %expect in parser.y > > +%expect 6 > > and apparently from this yacc man page: > http://invisible-island.net/byacc/manpage/yacc.html it is broken in some > versions of bison. Also it won't be supported in some older versions of > yacc, so we'll have to check for min versions of these programs. Can you > get this information as to the min versions required, otherwise, I'll > drop this part of the patch as the last thing we want is it not work on > some peoples machines without useful errors, preferably at configure time. This is obviously a cosmetic change, feel free to drop it. |
From: William S F. <ws...@fu...> - 2013-01-21 20:14:14
|
On 21/01/13 19:36, William S Fulton wrote: > >> >> I can do the merging if you like. >> > I would very much appreciate that. My main problem with that branch is >> > the rvalue references, but I think I have a plan for that now. Once >> that >> > is implemented and after a bit of tidy up, it'll be ready for merge to >> > master. Probably I will do a new release just before the merge and then >> > one one with the branch. Hopefully over the next month. >> >> The patch for the "gsoc2009-matevz" branch is attached. >> >> BTW, wchar_t* typemaps in Lib/csharp/wchar.i appear to be unfinished: >> SWIG_csharp_wstring_callback is declared but never used in actual >> typemaps. (I used my own typemaps for patch testing) >> Did you avoid the callback approach? If you have improvements to the typemaps, please submit them as another patch. Thanks. >> Vladimir >> > > Hi Vladimir > > I'm just looking at this patch and it added %expect in parser.y > > +%expect 6 > > and apparently from this yacc man page: > http://invisible-island.net/byacc/manpage/yacc.html it is broken in some > versions of bison. Also it won't be supported in some older versions of > yacc, so we'll have to check for min versions of these programs. Can you > get this information as to the min versions required, otherwise, I'll > drop this part of the patch as the last thing we want is it not work on > some peoples machines without useful errors, preferably at configure time. > > Meanwhile, thanks and I'm trying out the rest of the patch... I've applied the patch and some minor changes to the gsoc2009-matevz branch now. Sorry about the convoluted way for this to eventually get to master! William |
From: Vladimir K. <vka...@op...> - 2013-01-21 20:49:23
|
> On 21/01/13 19:36, William S Fulton wrote: >> >>> >> I can do the merging if you like. >>> > I would very much appreciate that. My main problem with that branch is >>> > the rvalue references, but I think I have a plan for that now. Once >>> that >>> > is implemented and after a bit of tidy up, it'll be ready for merge to >>> > master. Probably I will do a new release just before the merge and then >>> > one one with the branch. Hopefully over the next month. >>> >>> The patch for the "gsoc2009-matevz" branch is attached. >>> >>> BTW, wchar_t* typemaps in Lib/csharp/wchar.i appear to be unfinished: >>> SWIG_csharp_wstring_callback is declared but never used in actual >>> typemaps. (I used my own typemaps for patch testing) >>> > Did you avoid the callback approach? If you have improvements to the > typemaps, please submit them as another patch. Thanks. I just used built-in marshaling, like this: %typemap(imtype, inattributes="[MarshalAs(UnmanagedType.LPWStr)]", outattributes="[return: MarshalAs(UnmanagedType.LPWStr)]", directorinattributes="[MarshalAs(UnmanagedType.LPWStr)]", directoroutattributes="[return: MarshalAs(UnmanagedType.LPWStr)]" ) const wchar_t*, const wchar_t*&, const wchar_t* const & "String" I'm not ready to submit it because I'm kind of busy with nested classes (that is, when I have time to work on SWIG at all), and these typemaps require some debugging due to possible memory leaks. We use our own strings in our project and memory issues are taken care of, but the universal solution should be, well, universal. Vladimir. |
From: Vladimir K. <vka...@op...> - 2013-01-21 20:52:22
|
>> I'm just looking at this patch and it added %expect in parser.y >> >> +%expect 6 >> >> and apparently from this yacc man page: >> http://invisible-island.net/byacc/manpage/yacc.html it is broken in some >> versions of bison. Also it won't be supported in some older versions of >> yacc, so we'll have to check for min versions of these programs. Can you >> get this information as to the min versions required, otherwise, I'll >> drop this part of the patch as the last thing we want is it not work on >> some peoples machines without useful errors, preferably at configure time. >> >> Meanwhile, thanks and I'm trying out the rest of the patch... > > I've applied the patch and some minor changes to the gsoc2009-matevz > branch now. Sorry about the convoluted way for this to eventually get to > master! > Thank you. Glad that it's finally there. |