orbit-python-list

[Orbit-python-list] Updates

[Jason T. <ta...@li...>]

The latest batch of commits represent a pretty significant code overhaul.
There's lots of old code still to remove, and some bugs/leaks to fix
still.  Here is the ChangeLog excerpt:

   * Completely rewrote the way IDL objects are handled internally to
     more closely comply with the spec.  (IDL modules are Python modules,
     interfaces are classes, etc.)
   * inheritance is now supported, as well as delegation.  Inheritance is
     preferred since the spec (while vague) seems to require it, whereas
     delegation is optional.
   * Complies with the mapping spec on mappings for module names.  import
     Module for client side, import Module__POA for server side.
   * servants support implicit activation through _this()
   * deprecated POA::the_POAManager in favor of POA::_get_the_POAManager()
   * Uses _GlobalIDL and _GlobalIDL__POA modules for the global IDL scope,
     which is compatible with omniORBpy and Fnorb.
   * Implemented preprocessing of IDL files at runtime to determine which
     IDL files offer which modules.  Implications on performance are yet
     to be determined.  This approach may not scale very well at all for
     hundreds of IDL files.  It also could be broken in some cases; the
     preprocessor is not a complete IDL parser, so it may not work 100%.
     Time shall tell.  Uses the IDLPATH environment variable to get a
     list of paths to scan, otherwise defaults to the current directory.
   * hook __import__ to automagically load IDL files discovered by the
     processor that are required for the requested module
   * Pass -D__ORBIT_IDL__ -D__BONOBO_COMPILATION to libIDL so that we can
     load Bonobo IDLs
   * introduced more memory leaks, and probably more bugs.

The IDL directory preprocessor is a particularly interesting addition, and
it remains to be seen if this approach won't completely flop.  For me, at
least, it works pretty well, but my setup is somewhat limited (basically
just GNOME with Bonobo).  On my system it takes about 0.30 seconds to
preprocess 3 directories with about 50 IDL files, and 0.04 seconds once
cached.

Please report bugs as I'm not aware of any.  The leaks should be plugged 
in the next few days, however.

Jason.

Re: [Orbit-python-list] Updates

[Roland M. <ma...@ec...>]

Jason Tackaberry (2000-08-21 23:05:33 -0400) :

>    * Implemented preprocessing of IDL files at runtime to determine which
>      IDL files offer which modules.  Implications on performance are yet
>      to be determined.  This approach may not scale very well at all for
>      hundreds of IDL files.  It also could be broken in some cases; the
>      preprocessor is not a complete IDL parser, so it may not work 100%.
>      Time shall tell.  Uses the IDLPATH environment variable to get a
>      list of paths to scan, otherwise defaults to the current
>      directory.
>    * hook __import__ to automagically load IDL files discovered by the
>      processor that are required for the requested module
[...]
> The IDL directory preprocessor is a particularly interesting addition, and
> it remains to be seen if this approach won't completely flop.  For me, at
> least, it works pretty well, but my setup is somewhat limited (basically
> just GNOME with Bonobo).  On my system it takes about 0.30 seconds to
> preprocess 3 directories with about 50 IDL files, and 0.04 seconds once
> cached.

I'm not sure I think it's always a good idea to do it automagically.
I mean, yes, it can be handy, but I feel it would be preferable to
continue to specify the IDL file(s) to load.  I'm not sure about that,
but I think it's less risky to "load_idl (blah)" and then "import
mymodule".  Supposing there are several IDL files describing a
similarly named module...  Supposing there are a great many IDL files
and we only need one...  Supposing many things, I think that can lead
to breakage.  I don't really like the idea of ORBit-Python doing
things in my back.  On the other hand, I'm totally in favour of an
automatisation (does that word exist in English?) of things.  As I
said, I'm all for the "load_idl (blah)" then "import blahmod".  That
could also be a "preprocess_idl_dirs (blah)" then "import blahmod".

  Of course, you're the boss :-)

  On another point, I just scanned through src/process_idl.c, and it
appears you parse the IDL files by hand.  Well, why not, but you said
it yourself, that's not a complete IDL parser.  Why not use the libIDL
like in src/idl.c, with the appropriate tree_pre_func and
tree_post_func?

>    * Pass -D__ORBIT_IDL__ -D__BONOBO_COMPILATION to libIDL so that we can
>      load Bonobo IDLs

  Here again, I feel that could (should?) be optional arguments to
load_idl.  On the other hand, I don't know what these arguments do to
IDL_parse_filename, so they might be completely innocent.

>    * introduced more memory leaks, and probably more bugs.

  No memory leak seen from here for the first tests.  I haven't tried
the big one yet, since the module mappings have changed and I'll have
to check and fix where it broke.

  Ah, and I fixed the patch I had uploaded.  Now it works (well,
should work anyway).

  Have a nice day.

Roland.
-- 
Roland Mas

Qu'est-ce qui est jaune, qui pèse deux cents kilos et qui chante ?
Un sumotori qui a bu trop de saké.

Re: [Orbit-python-list] Updates

[Jason T. <ta...@li...>]

> continue to specify the IDL file(s) to load.  I'm not sure about that,
> but I think it's less risky to "load_idl (blah)" and then "import

Sure, but load_idl() is an orbit-python specific extension.  I plan on
having some way to disable the automagic-loading, though.  So those who
want complete control and don't mind writing orbit-python specific code can
do that.

I think the preprocessing performance will be acceptable.  It doesn't scale
linearly mind you, but there's some optimizations I can do to help that.  I
plan on doing some benchmarks to guage the performance hit.  0.03 seconds
for 50 cached reasonably complex IDL files is pretty good, I think.  I'm
more concerned with making the auto IDL loading feature smarter, but even
right now it's pretty intelligent.

If you do find instances where the auto-IDL loader doesn't do what it
should, I'll definitely want to hear about it.

> mymodule".  Supposing there are several IDL files describing a
> similarly named module...  Supposing there are a great many IDL files
> and we only need one...  Supposing many things, I think that can lead

In the case where there are several IDL files describing one module, it will
load all those IDL files.  It's also smart enough to trace through includes.
For example, Bonobo consists of several IDL files, while Bonobo.idl just
includes all those IDL files.  If you import Bonobo, orbit-python will load
only Bonobo.idl because it includes all the files that would be needed anyway.

In the case where you need only one, you can (or will be able to) do:

   from GNOME import Gnumeric

say, which will load only the Gnumeric IDL file.  (It doesn't do this yet,
but it will.)

> automatisation (does that word exist in English?) of things.  As I
> said, I'm all for the "load_idl (blah)" then "import blahmod".  That
> could also be a "preprocess_idl_dirs (blah)" then "import blahmod".

Probably something like:

   import CORBA
   CORBA._orbit_auto_import = 0

>   On another point, I just scanned through src/process_idl.c, and it
> appears you parse the IDL files by hand.  Well, why not, but you said
> it yourself, that's not a complete IDL parser.  Why not use the libIDL
> like in src/idl.c, with the appropriate tree_pre_func and
> tree_post_func?

Because I don't want the overhead of libIDL.  It's quick, sure, but I need
the preprocessing to be as fast as possible.  And I think a crude parser
is all that I need.  After all, I only need to discover modules in the global
and first-level scopes.

>   Here again, I feel that could (should?) be optional arguments to
> load_idl.  On the other hand, I don't know what these arguments do to
> IDL_parse_filename, so they might be completely innocent.

Yes, this will be optional, and yes, these options are completely innocent.
Right now I'm focusing on making things compliant with the spec.  After that,
I can provide options so that the users can obtain whatever level of control
they want.

>   No memory leak seen from here for the first tests.  I haven't tried
> the big one yet, since the module mappings have changed and I'll have
> to check and fix where it broke.

Well, there are definitely leaks. :)  But I also have a lot of the code still
in there from the old way of doing things, and so once I remove that, some
or all of the leaks might vanish.

>   Ah, and I fixed the patch I had uploaded.  Now it works (well,
> should work anyway).

Great.  I'll have a look at that tonight.

Jason.