Re: [Swig-devel] best way to get from type to feature text?

[David F. <fr...@tu...>]

Hi William,

>>>>> "WSF" == William S Fulton <William> writes:

WSF> When you say you want to associate Perl code with a type, do you mean 
WSF> just class or any type, such as a struct/union/class/primitive type, as 
WSF> well as pointers/reference pointer references etc etc?

A class was my intent, but for a more general solution, one would
probably want to consider any/every type.


WSF> Assuming yes, there is no such type node that I can think
WSF> of. Features don't attach to types, they attach to
WSF> functions/methods and classes/structs/unions and enums, so you
WSF> can't use a feature.

Well... I did use a feature, actually, and it seems to work without a
hitch, at least for what I needed.  I did have to do a bit of a
brute-force walk of the parse tree in order to find the feature data,
but this seems to work.

(I was quite tempted, as I was doing this, to simply import the data I
needed into swig without involving the .i files, the parse tree and
DOH.  Walking the parse tree is not nearly as simple as I'd like, and
DOH ... well ... let's just say that I'm not a fan of DOH.  I thought
about adding support for sqlite [http://www.sqlite.org] but I simply
hate messing with autoconf, so I opted to not do that.  But, had
sqlite been available, the choice of using sqlite over DOH would have
been a no-brainer.)


WSF> Of course typemaps are associated with types and you are probably
WSF> looking at inventing a new typemap. This is usually to be
WSF> avoided, so I think it would be best to give an detailed example
WSF> of what you are trying to do.

Agreed: I really don't want another typemap.  Certainly not a static
typemap.  A feature is kinda, sorta less than a typemap, at least in
my brain, and it was sufficient for what I needed.  Since swig already
supports them, that's what I chose to use.

Ok, here's a description of what I've got and what I'm trying to
achieve:

	- We use SWIG for our system.  We support perl, tcl and
	  python, and all are simultaneously active.

	- For our system, we have a class called bomValue.  bomValue
	  objects are basically a discriminated union.  Eg, there's a
	  type indicator and a union of bool, ints of various flavors,
	  floats of various flavors, a string and a pointer.  Think of
	  this as how you'd implement a perl (or python or tcl or ...) 
	  variable and you'd not be too far off.  This sort of thing
	  is also commonly used for yacc parsers.

	- The bomValue class is not exposed to the extension
	  languages.  Instead, the data within a bomValue object is
	  unpacked and exported into the extension language (or vice
	  versa).  We have typemaps that deal with the un/packing of a
	  bomValue object, and the conversion goes in both directions.

	- The issue I was trying to solve is, what's the best way to
	  deal with the pointer when sending this to perl.  I need to
	  convert this into a blessed reference on the perl side, but
	  this happens selectively because the bomValue may store an
	  int or float or ...  We also take the pointer's type into
	  account so that we can dynamically determine the package to
	  use when blessing the pointer/ref.

	- Before, we had a "trampoline" function that handled all of
	  this, but it turns out that this wasn't reliable (owing to
	  quirks in perl) and we had a couple of significant memory
	  leaks.

	- The change I made has eliminated these leaks.  It works by
	  avoiding the trampoline function and handling the pointer
	  directly, with some help from the perl code that is
	  expressed with the feature directive.

Given what our typemap does in C++ land, the perl code expressed
within the feature is pretty simple:

    #if defined(SWIGPERL)

    %feature("perlafter") bomValue
    %{
	return undef if (!defined($result));
	return $result if (! ref($result));
	my %resulthash;
	tie %resulthash, ref($result), $result;
	return bless \%resulthash, ref($result);
    %}

    #endif

Yes, this looks like what swig already generates, except that this is
for a type that isn't directly exposed to the extension languages.

The modified perl5.cxx code that handles this feature is shown below.
This is for the functionWrapper() function; similar modifications were
made to the memberfunctionHandler() function.

Note: we're using an older version of SWIG (a '36 vintage) so this
won't match up with the latest '40 version you've got.

		...

	if (blessed &&
	    !member_func)
	  {
	    bool bWroteInterestingStubCode = false;

	    String *func = NewString("");

	    //-------------------------------------------------------
	    // We'll make a stub since we may need it anyways
	    //-------------------------------------------------------
	    Printv(func, "sub ", iname, " {\n",
		   tab4, "my @args = @_;\n",
		   NIL);

	    //-------------------------------------------------------
	    // Get user-supplied 'before' code, if any.
	    //-------------------------------------------------------
	    String* pBeforeCode = 0;
	    String* pAfterCode = 0;
	    GetBeforeAfterText(pReturnType, pBeforeCode, pAfterCode);

	    if (pBeforeCode != 0)
	      {
		Printf(func, "%s", pBeforeCode);
		bWroteInterestingStubCode = true;
	      }

	    //-------------------------------------------------------
	    // Call the internal function.
	    //-------------------------------------------------------
	    Printv(func, tab4, "my $result = ", cmodule, "::", iname, "(@args);\n", NIL);

	    //-------------------------------------------------------
	    // Get user-supplied 'after' code, if any.  If there
	    // is 'after' code, this is all that will be written.
	    //-------------------------------------------------------
	    if (pAfterCode != 0)
	      {
		Printf(func, "%s}\n", pAfterCode);
		bWroteInterestingStubCode = true;
	      }

			...

Yes, I've completely ignored the indentation and commenting style used
in the rest of swig.  I wasn't planning on releasing this in the wild
so, please, no flames.

In order to get this to work, I needed some extra code to find the
feature data in the parse tree.  Quite a bit of extra code, actually.
I've included this code at the end of the file.  This is a brute-force
search and not very elegant.  Perhaps there are better ways to search
the parse tree in swig, and I'd be happy to have a tutorial on this.
(Id' be far happier if I could get the parse tree shoved into a sqlite
database.  Then searching would be much simpler.)


WSF> Incidentally, I don't know of any scripting language typemaps
WSF> that generate scripting language code, they are used for c/c++
WSF> code, but perhaps someone else knows of some. Java and C#, on the
WSF> other hand generate all the Java/C# code using typemaps, and they
WSF> take quite a different approach to this part of the code
WSF> generation. I'd suggest that if typemaps are to start generating
WSF> Perl code, that they go the whole way like Java/C#.

Well, I've said this for years: instead of implementing yet another
static typemap, I'd be inclined to make the system support 'active'
functions instead.  That is, I'd be pleased (read: thrilled) if I
could register a function with the system and then have swig invoke
that function at certain points as the parse tree is scanned.  A
function is much preferred over static text in a typemap.

Thanks for your help.  If you see flaws in the code I'd like to hear
about them.

--
David Fletcher				Tuscany Design Automation, Inc.
dav...@tu...		3030 S. College Ave.
					Ft. Collins, CO 80525 USA

PS If you ever want to explore adding sqlite to swig, I'd be eager to
discuss this and quite willing to help.

	- sqlite is public domain, so licensing shouldn't be an issue.
	- The quality of the code is quite high, with testing that
	  results in well over 95% branch coverage.
	- There is an active community for this DB, and it has been
	  ported to a wide variety of systems.  Linux, Mac, Windows,
	  iPhone, etc.  It's part of Firefox.  Etc, etc.
	- There are interfaces to sqlite from all of the major
	  extension languages (perl, tcl, python, lua, ...).
	- There are only 3 files --- two header files and one C file
	  --- that need to be added to the mix within Swig.
	- sqlite is written in C.  I'd be happy to help develop a C++
	  interface for use within SWIG.

I believe that the use of sqlite would make certain oeprations
substantially easier.  Eg, the brute-force code below, could be
achieved in a dozen lines of code.  Probably less.

Here's the brute-force scan of the parse tree:

/*----------------------------------------------------------------------------
 * static bool
 * GetBeforeAfterTextHelper(...)
 *
 *//*!
 *
 * Given a node, find a child named "include", or one named 'classforward'.
 * The 'classforward' nodes are the ones we're after, but only if this node
 * matches the name of the desired type, and if there are 'perlbefore' and/or
 * 'perlafter' key attributes on the 'classforward' node.
 *
 *//*-----------------------------------------------------------------------*/
static bool
GetBeforeAfterTextHelper(String* pDesiredTypeName,
			 Node* pOrigNode,
			 String*& rpBeforeCode,
			 String*& rpAfterCode,
			 int iLevel)
{
  //-------------------------------------------------------
  // Given a node, examine this node and all of its siblings.
  // In other words, examine one level of a tree.
  //-------------------------------------------------------
  for (Node* pNode = pOrigNode; pNode != 0; pNode = nextSibling(pNode))
    {
      //-------------------------------------------------------
      // Get the 'nodeType' for the node.  We want this to
      // match 'include' or 'classforward'.
      //
      // We'll handle the 'include' case first.  Basically,
      // all we do is examine the immediate children of these
      // nodes.
      //-------------------------------------------------------
      String* pNodeType = nodeType(pNode);
      if (pNodeType == 0)
	continue;

      if (Cmp(pNodeType, "include") == 0)
	{
	  bool bResult = GetBeforeAfterTextHelper(pDesiredTypeName,
						  firstChild(pNode),
						  rpBeforeCode,
						  rpAfterCode,
						  iLevel + 1);
	  if (bResult == true)
	    return true;
	}

      //-------------------------------------------------------
      // The next case we need to consider is the 'classforward'
      // node.  We'll find the 'sym:name' attribute on this
      // node and this should, hopefully, match the name of the
      // desired type.
      //-------------------------------------------------------
      if (Cmp(pNodeType, "classforward") != 0)
	continue;

      String* pSymName = Getattr(pNode, "sym:name");
      if (pSymName == 0)
	continue;

      if (Cmp(pSymName, pDesiredTypeName) != 0)
	continue;

      //-------------------------------------------------------
      // Ok, we found a candidate node.  Now, we need to see
      // if we can find the 'perlbefore' and/or 'perlafter'
      // attributes... except these are key attributes.  What's
      // the difference?  Who knows?  I don't.  Anyway, we're
      // looking for perlbefore/perlafter and, should we be
      // fortunate enough to find these, we're done.
      //-------------------------------------------------------
      String* pKey;

      for (Iterator ki = First(pNode);
	   (pKey = ki.key) != 0;
	       ki = Next(ki))
	{
	  if (! DohIsString(Getattr(pNode, pKey)))
	    continue;

	  //-------------------------------------------------------
	  // Before...
	  //-------------------------------------------------------
	  if (rpBeforeCode == 0)
	    {
	      if (Cmp(pKey, "feature:perlbefore") == 0)
		{
		  rpBeforeCode = Getattr(pNode, pKey);
		  continue;
		}
	    }

	  //-------------------------------------------------------
	  // After...
	  //-------------------------------------------------------
	  if (rpAfterCode == 0)
	    {
	      if (Cmp(pKey, "feature:perlafter") == 0)
		{
		  rpAfterCode = Getattr(pNode, pKey);
		  continue;
		}
	    }

	  //-------------------------------------------------------
	  // Success?  If we happened to find both the 'before'
	  // and 'after' values, we're done.  Otherwise, we'll
	  // keep scanning until we have both (and the first
	  // definition wins, by the way).
	  //-------------------------------------------------------
	  if (rpBeforeCode != 0 ||
	      rpAfterCode != 0)
	    return true;
	}
    }

  //-------------------------------------------------------
  // If, after the scan, we have EITHER the 'before' or
  // 'after', we'll call this a success.  Otherwise, we'll
  // return false to indicate that we couldn't find this
  // information.
  //-------------------------------------------------------
  if (rpBeforeCode != 0 ||
      rpAfterCode != 0)
    return true;

  return false;
}

/*----------------------------------------------------------------------------
 * static bool
 * GetBeforeAfterText(SwigType* pType, String*& rpBeforeCode, String*& rpAfterCode)
 *
 *//*!
 *
 * Now, I will admit that there may be a better way to go about all of
 * this, because this function is searching what I'm presuming to be the
 * raw parse tree.  I would guess that there's a "compiled" form of this
 * tree .... but since I don't see anything like this....
 *
 * What we have to do is navigate from a "desired" type, which is probably
 * the return type for some function.  Then, we jump through some hoops
 * to get the name of the type in a form we can use.  Then, we'll do a
 * semi-brute-force scan of the raw parse tree to find the information
 * we're after.
 *
 *//*-----------------------------------------------------------------------*/
static bool
GetBeforeAfterText(SwigType* pDesiredType,
		   String*& rpBeforeCode,
		   String*& rpAfterCode)
{
  //-------------------------------------------------------
  // Initialize rpBeforeCode and rpAfterCode.  These
  // will hold the strings of perl code we'd like to insert
  // into the output.
  //-------------------------------------------------------
  rpBeforeCode = 0;
  rpAfterCode = 0;

  if (pDesiredType == 0)
    return false;

  //-------------------------------------------------------
  // Given the desired type, find the node that represents
  // this type.  If we can't find this, there's a problem,
  // so return right away.
  //-------------------------------------------------------
  Node* pDesiredTypeNode = Swig_symbol_clookup(pDesiredType, 0);

  if (pDesiredTypeNode == 0)
    return false;

  //-------------------------------------------------------
  // Now, we need the string equivalent for this Node.
  // sym:name is probably what we're after, as this should
  // be the simple, unadorned type name.  But, we'll extend
  // the search just a bit in order to cover all of the bases.
  //-------------------------------------------------------
  String* pDesiredTypeNodeName = Getattr(pDesiredTypeNode, "sym:name");

  if (pDesiredTypeNodeName == 0)
    pDesiredTypeNodeName = Getattr(pDesiredTypeNode, "name");

  if (pDesiredTypeNodeName == 0)
    pDesiredTypeNodeName = nodeType(pDesiredTypeNode);

  if (pDesiredTypeNodeName == 0)
    {
      Printf(stdout, "-E- Could not find pDesiredTypeNodeName\n");
      return false;
    }

  //-------------------------------------------------------
  // First, we'll walk up from the node to the top of
  // the tree.  Then, we'll examine the top node's immediate
  // children.
  //-------------------------------------------------------
  Node* pTop = pDesiredTypeNode;

  while (1)
    {
      Node* pParent = parentNode(pTop);
      if (pParent == 0)
	break;

      pTop = pParent;
    }

  //-------------------------------------------------------
  // Now, we'll need to find the node in the raw parse
  // tree that matches this type name.  We'll do this by
  // walking down a level or two to look for this node
  // that happens to have the perlbefore and/or perlafter
  // attributes.
  //-------------------------------------------------------
  bool bResult = GetBeforeAfterTextHelper(pDesiredTypeNodeName,
					  firstChild(pTop),
					  rpBeforeCode,
					  rpAfterCode, 1);

  return bResult;
}