Re: [Swig-devel] Templates and recursive macro expansion of typemaps.

[William S F. <ws...@fu...>]

Guillaume Yziquel wrote:
> Hello.
>
> Sorry for the replying delay. I've been busy elsewhere for a while.
>
Sorry for big delay too, I'm overloaded on SWIG work.
> William S Fulton a écrit :
> >
> > The UTL languages that make heavy use of %fragment are scripting
> > languages and they solve the STL wrapping in a way that doesn't deal
> > with the needs of statically typed languages. You really ought to be
> > modelling your approach on other static languages, so take a look at
> > how Java wraps the STL. Does this approach not suite Ocaml?
>
> I indeed noticed that there was a different approach for wrapping the
> STL in python than in Java. Didn't really know why.
>
> However, I'm quite lost reading the java.cxx file. Since you're
> obviously much more familiar with it than I am, could you please answer
> the following question:
>
> Where in java.cxx is the wrapper function code emitted for a, say,
> constructor of a C++ class?
constructorHandler()

>
> >>> I'm not entirely clear what you are asking given I don't know Ocaml,
> >>> nor do I know what your new typemaps do. If ocamlout, is like
> >>> javaout or csout, then it should contain Ocaml code to marshal from
> >>> the C/C++  type to the Ocaml type.
>
> In a nutshell, this typemap is not C/C++ code, it contains only the
> string of an OCaml type, in the OCaml language.
>
> >>> But I don't think you have followed this convention. I guess
> >>> ocamlout is like the jstype or cstype typemap so it contains the  
> >>> Ocaml type for the given C/C++ type. Correct?
> >>
> >> As far as I understand you, yes.
>
> So I should rename it to ocamlstype? Or to ocamltype?
I suggest ocamltype.
>
> What is the difference between the jstype typemap and the jtype typemap?
>
jstype contains the pure Java type used in the proxy class and module
class. The jtype typemap contains the intermediary type used in the JNI
layer on the Java side. The jni typemap is the JNI equivalent to the
jtype on the C/C++ side.
> > Do you get to study the C#/Java typemaps yet? They are a model which
> > has worked for statically typed languages. If you get to grips with
> > how these work, you'll see that there are many solved problems done
> > over many years which you will just have to re-invent, or is there
> > something fundamentally wrong with the approach taken by these 2
> > language modules? Modula3 is another statically typed language based
> > on Java/C#, but I am not sure of its current state.
>
> I've had a look at Java typemaps, but I'm afraid I do not really
> understand them yet. For instance, looking at std_string.i, I do see
> some jstype and jtype typemaps. But looking at std_vector.i, I do not
> see such typemaps. Why is that so?
>
This works using the typemap pattern matching rules which are explained
in Typemaps.html. Because there is no jstype or jtype or jni typemap for
std::vector, it defaults to the SWIGTYPE typemap which is in java.swg.
This typemap contains the special variable $javaclassname, which gets
expanded to the proxy classname in this case. (Sometimes $javaclassname
gets expanded to the intermediary classname - when used for intermediary
classes, ie unknown types). In C# he equivalent is $csclassname, you
probably should use $ocamlclassname.
> >>> If so, you are asking how to obtain the Ocaml type when wrapping
> >>> std::vector<int>?
>
> Yes. In my case, it would be std::list<int>, but yes.
>
> > Assuming you will be using %template, as that is fundamental to
> > wrapping templates, the Ocaml type for std::vector<int> will be as
> > specified by %template and that needs to be a valid symbol, ie no
> > spaces, so "list int" won't work. Can you not give the proxy class for
> > vector<int> a name like VectorInt?
>
> I believe that I really need an "int list" type.
>
> Take into account that in OCaml, there is no such thing as 'type
> casting'. So if I export std::list<int> to, say, intlist, I'm stuck with
> this type. There is no way to convert it to an 'int list'. And functions
> that work on lists will simply not compile when trying to feed them an
> intlist. Since an intlist is not a list, whereas an int list is indeed a
> list.
>
> OCaml's typing is extremely strong, and I have to comply with it.
>
> OK. You could do some type casting, using Obj.magic. But then you're an
> OCaml heretic. I'd like the OCaml module i'm writing to be as orthodox
> as possible.
>
> So I guess I do need an 'int list', with a space in it, as that is the
> proper OCaml type.
>
This is going to be a problem as SWIG symbols used on the target
language side do not support having spaces in their names. Modifications
to the core will be needed to deal with this and change the space into a
valid C symbol name where this is needed.

Alternatively, we'd have to think of some other way to do vector<int>.

> >>> Ignoring templates for now, how have you solved getting the Ocaml
> >>> type for a non-templated class, eg class X {}; ?
> >>
> >> I'm holding the OCaml type in the ocamlout typemap.
> >>
> >> So, for a C++ class X {} declaration, I would have to define
> >> explicitely a %typemap in the .i file.
> >>
> >> For now, this is not done this way, but this is what I want to do.
> >>
> >> For now, ocamlout manages to convert std::string to OCaml type
> >> 'string'. For classes, I use a suboptimal solution of putting the C++
> >> class name directly in the OCaml code, without going through a
> >> typemap. This is not good, but that can easily be changed in
> >> ocaml.cxx, and is not that much of a problem (I guess).
> >
> > That is rather tedious for every type and not recommended. Take a look
> > at the C# $csclassname for this to be calculated automatically for
> > complex types. It basically replaces the C++ type, eg
> > SomeNamespace::SomeClass with the target language symbol, SomeClass or
> > in the event of %rename, the renamed symbol, ie what is in the
> > "sym:name" node attribute. For primitive types and strings, the
> > $csclassname is not used, but specified directly in the cstype typemap
> > much like you have already done for string.
>
> Thank you very much for this information. I do not yet understand every
> bit of it, but I'm sure it'll be very helpful.
>
> Nevertheless, for now, I would rather focus on templates than on this
> issue, since this is what I perceive as a bottleneck in writing an OCaml
> module.
>
> >>> What would the Ocaml type be here and how do you declare this in
> >>> your Ocaml type typemap, which I think is ocamlout?
> >>
> >> For instance, for std::string I have in std::string.i:
> >>
> >>> namespace std {
> >>>
> >>>   class string;
> >>>
> >>>   %typemap(ocamlin) string "string"
> >>>   %typemap(in) string
> >>>   %{$1.assign(String_val($input), caml_string_length($input));%}
> >>>
> >>>   %typemap(ocamlin) const string & "string"
> >>>   %typemap(in) const string &
> >>>   {
> >>>     std::string $1_str(String_val($input), 
> caml_string_length($input));
> >>>     $1 = &$1_str;
> >>>   }
> >>>
> >>> }
> >>
> >> OK. I did not need the ocamlout typemap for now. But it would be
> >> pretty much the same as the ocamlin typemap:
> >>
> >> %typemap(ocamlout) string "string"
> >
> > What is the planned difference between ocamlout and ocamlin?
>
> A potential one...
>
I suggest changing this to follow the rest of the SWIG conventions. Note
that in these cases in C# for asymmetric type marshalling there is an
optional 'out' typemap attribute which specifies the return type, so 
you'd have

%typemap(cstype, out="String") std::string "string";

see the C# docs for more details.

> >>> You might also want to give an example of the desired output for the
> >>> complete Ocaml proxy class for std::vector<int> to aid what you are
> >>> trying to achieve.
> >>
<big snip>

> >> Currently I can manage to get this 'ocaml_typeparm[word] list' type
> >> from an awkward macro based implementation.
> >>
> >> For example, if I have a std::list<std::list<int>>, I can now get a
> >>
> >>     ocaml_typeparm[std::list<int>] list
> >>
> >> and the idea I have is to recursively expand it, using code *in*
> >> ocaml.cxx, to
> >>
> >>     ( ocaml_typeparm[int] list ) list
> >>
> >> and then to
> >>
> >>     ( int list ) list
> >>
> >> which would be the corresponding, valid, OCaml type. (A shorter
> >> version of this OCaml type would be 'int list list', but I would
> >> perfectly be satisfied with the above.
> >>
> > I'm wondering if your approach works with a more general case of
> > templates. How would this design work with other templates that are
> > not a list, say just some arbitrary template class Foo<Bar>? Would you
> > recursively expand it to
> >   ( Bar Foo ) Foo
> > and does that make sense? What about something more complex, like
> > Foo<Foo<Bar> > ?
>
> I do not really see a problem when it comes to OCaml.
>
> Types such as (string, ((int * bool) list)) Hashtbl.t would be the type
> of a hash table whose keys are strings and whose values are lists of
> couples of integers and booleans.
>
> This kind of typing is perfectly legitimate and usual in OCaml.
>

So maybe the solution is to use the ocamltype typemap for looking up the 
appropriate types. for example:

%typemap(ocamltype) int "int";
%typemap(ocamltype) std::vector "list $subtype";
namespace std {
   template<typename T> std::vector { ... };
}
%template(VectorInt) std::vector<int>;

then given in ocaml.cxx you'd get "list $subtype" and then you'd lookup 
the ocamltype for T, which would give you "int", resulting in "list 
int". Not sure what else to suggest, other than improving the typemap 
system in the core such that the following:

%typemap(ocamltype) std::vector<typename T> "list $typemap(ocamltype, T)"

would give rise to "list int" for std::vector<int> as 
"$typemap(ocamltype, T") would look up the ocamltype typemap for type T, 
where T=int in this case. Currently T is not expanded to the appropriate 
type, so it doesn't know to look up the "int" ocamltype typemap. This 
would be a useful general improvement in SWIG.

William