orbit-python-list

Re: [Orbit-python-list] Porting to Python 2.0b1

[Roland M. <ma...@ec...>]

Jason Tackaberry (2000-09-07 12:47:36 -0400) :

> >    * added a #define and some #ifndef's to make it easy to temporarily
> >      comment out the (unfortunately buggy) thread-safe-ication code.
> >      That code will probably have to go when the bug is fixed.
> 
> Are you referring to the changes I added?  What bugs have you observed
> from the code?

Well, I as I stated in another message (September the 5th), I get an
error (PyThreadState_Get: no current thread) every time a server tries
to execute a request via CORBA if the request is to be executed in the
same process.

> >    * renamed init_exceptions() to ORBit_Python_init_exceptions() to avoid
> >      namespace conflict with a similarly named function appearing in
> >      Python 2.0.  Works fine with 2.0b1.
> 
> That's fine; we might as well rename all the other init_ functions to
> use this convention as well.

  Okay, I'll do that part.  Well, my Emacs will, at any rate :-)

Roland.
-- 
Roland Mas

La menace de la baffe pèse plus lourd que la baffe elle-même.
  -- in Sri Raoul le petit yogi (Gaudelette)

Re: [Orbit-python-list] Porting to Python 2.0b1

[Jason T. <ta...@li...>]

>    * added a #define and some #ifndef's to make it easy to temporarily
>      comment out the (unfortunately buggy) thread-safe-ication code.
>      That code will probably have to go when the bug is fixed.

Are you referring to the changes I added?  What bugs have you observed
from the code?

>    * renamed init_exceptions() to ORBit_Python_init_exceptions() to avoid
>      namespace conflict with a similarly named function appearing in
>      Python 2.0.  Works fine with 2.0b1.

That's fine; we might as well rename all the other init_ functions to
use this convention as well.

I've unfortunately been a little quiet lately.  I'm working on another
project for work right now that's occupying most of my time.
Fortunately this is stuff I get to release back to the community, so
getting paid to work on it is a nice bonus. :)

Jason.

Re: [Orbit-python-list] Porting to Python 2.0b1

[Roland M. <ma...@ec...>]

Roland Mas (2000-09-06 18:54:34 +0200) :

>   I therefore suggest renaming at least some of ORBit-Python's
> functions to something with a prefix (I used "OPY_" as a quick hack,
> but I don't intend to keep it).  I'll grep through the sources of
> ORBit-Python, then nm through libpython2.0.a, see the intersection of
> the function names thusly obtained, and rename them.  Expect a series
> of CVS commits soon :-)

Not there yet, but here is an intermediate hack, committed to CVS.
From the ChangeLog:

   * added a #define and some #ifndef's to make it easy to temporarily
     comment out the (unfortunately buggy) thread-safe-ication code.
     That code will probably have to go when the bug is fixed.
   * renamed init_exceptions() to ORBit_Python_init_exceptions() to avoid
     namespace conflict with a similarly named function appearing in
     Python 2.0.  Works fine with 2.0b1.

>   BTW: how do I add a conditional to ./configure so that I can keep
> the threading support out?  (Or alternatively, when will it be
> fixed? :-)

  Probably not worth adding a conditional.  I added #ifndef's, so you
can type 'make CFLAGS="-DORBIT_PYTHON_NOT_THREADED"', and anyway that
code is supposed to go away at some point.

Roland.
-- 
Roland Mas

Sauvez les castors, plantez des arbres.

[Orbit-python-list] Porting to Python 2.0b1

[Roland M. <ma...@ec...>]

I got the following message in my server under Python 2.0b1:

** WARNING **: Exception must be derived from CORBA.UserException

  Using the current CVS code slightly patched: I just commented out
the offending code causing errors with threading (reported in another
message).  This bug happens when I use Python 2.0b1 but not when I use
Python 1.5.2.

  After a bit of searching, it appears that the function
init_exceptions() (in src/except.c) is not called.  After another bit
of searching, it appears that init_exceptions() is in fact called, but
unfortunately it's not ours.  I don't know what Python 2.0b1's does,
but I'm pretty confident ;-) that it does not create the
CORBA.UserException and CORBA.SystemException exceptions.

  I therefore suggest renaming at least some of ORBit-Python's
functions to something with a prefix (I used "OPY_" as a quick hack,
but I don't intend to keep it).  I'll grep through the sources of
ORBit-Python, then nm through libpython2.0.a, see the intersection of
the function names thusly obtained, and rename them.  Expect a series
of CVS commits soon :-)

  BTW: how do I add a conditional to ./configure so that I can keep
the threading support out?  (Or alternatively, when will it be
fixed? :-)

  (BTW² for Jason: I'll also have a look at the memory leaks, since I
still encounter them.  You have any ideas where to look?)

  Have fun,

Roland.
-- 
Roland Mas

Mou ichido !  Hayaku !  Ookii koede !
  -- Atsuko Sasaki

Re: [Orbit-python-list] Threading support

[Roland M. <ma...@ec...>]

Jason Tackaberry (2000-08-26 14:35:06 -0400) :

> I tested it briefly and it seems to work, but there still may be
> problems.  It only took me a couple minutes, and I'm skeptical to
> think anything is that simple. :) Let me know if you run into any
> bugs.

I do run into a bug:

Fatal Python error: PyThreadState_Get: no current thread
Aborted

  This happens in a single-threaded CORBA server that just sent a
request to itself (by accessing an attribute of an object that it
happens to be hosting).  I'm not sure, but it seems that it happens on
the first time this situation (server calling itself through CORBA)
occurs.  The bug is entirely reproducible in my case (same thing
happens every time I run my thing).  I'll try and track it down a bit
in a few days when I get some time.  Meanwhile, consider this a bug
report asking for attention :-)

Roland.
-- 
Roland Mas

At a train station, the trains stop.  At a bus station, the buses stop.
I have a workstation on my desk...

[Orbit-python-list] Last updates

[Roland M. <ma...@ec...>]

Two batches of updates.  ChangeLog excerpt:

   * src/marshal.c, src/demarshal.c: added *experimental* support for
     wstring type, mapped to a Python Unicode object.  Only works with
     Python >= 2.0 (e.g. from Sourceforge CVS).  Very likely *not* to be
     interoperable *at all* with other ORBs.
   * src/server.c: added code to allow the server program to overload
     the _get_* and _set_* operations (with e.g. side effects).
   * src/server.c: fixed a compilation bug for current Python from CVS
     (the soon-to-be Python 2.0).

  The wide string stuff works between an ORBit-Python client and an
ORBit-Python server.  Nothing has been done to make it work between
ORBit-Python and any other ORB or binding.  I had a hard time finding
ORBs that handle wstrings, though...  Only found ILU and TAO (and
ORBit, obviously, though not that much), and I have no clue how to use
them yet.  I'll do it soon, as well as the wchar type.

Roland.
-- 
Roland Mas

Qu'est-ce qui est jaune, qui pèse deux cents kilos et qui chante ?
Un sumotori qui a bu trop de saké.

Re: [Orbit-python-list] Threading support

[Joseph T. B. <jb...@tj...>]

cool.  Like I said, I was probably just being paranoid.

jb

Jason Tackaberry writes:
 > > My concern is that if someone unsuspectingly calls orb.run from
 > > different threads within the same process, is it a possibility that 2
 > > threads could be inside the the corba module C code at the same time
 > 
 > Why would you want or need to execute orb.run() in more than one thread?
 > Assuming you did, ORBit might not like that too much.   (Although I just
 > tested it and it didn't seem to cause any problems, it just seemed to
 > ignore one of the threads.)
 > 
 > Anyway, the global interpreter lock saves us from the possibility of
 > 2 threads being inside the CORBA module at once.  The only exception is
 > calling orb.run() in more than one thread, which shouldn't be necessary
 > anyway.  
 > 
 > Jason.

-- 
------------------+
  Joseph T. Bore  |
 jb...@TJ... |
------------------+

Re: [Orbit-python-list] Threading support

[Jason T. <ta...@li...>]

> My concern is that if someone unsuspectingly calls orb.run from
> different threads within the same process, is it a possibility that 2
> threads could be inside the the corba module C code at the same time

Why would you want or need to execute orb.run() in more than one thread?
Assuming you did, ORBit might not like that too much.   (Although I just
tested it and it didn't seem to cause any problems, it just seemed to
ignore one of the threads.)

Anyway, the global interpreter lock saves us from the possibility of
2 threads being inside the CORBA module at once.  The only exception is
calling orb.run() in more than one thread, which shouldn't be necessary
anyway.  

Jason.

Re: [Orbit-python-list] Threading support

[Joseph T. B. <jb...@tj...>]

My concern is that if someone unsuspectingly calls orb.run from
different threads within the same process, is it a possibility that 2
threads could be inside the the corba module C code at the same time
and step on each other?  I imagine that if there is any I/O one thread
could block, and then another thread would be able to scribble on the
other's buffer.  Or am I just being paraoid?  8)

jb

Jason Tackaberry writes:
 > > Essentially they only hold the global interpreter lock or the tk lock
 > > exclusive (except for special cases) of each other.  I imagine this
 > > may still be a problem?  Or do you think this is a non issue?
 > 
 > I'm tempted to say this isn't a problem, but I admit I don't completely
 > follow you.  Can you elaborate on the potential problem?
 > 
 > Jason.

-- 
------------------+
  Joseph T. Bore  |
 jb...@TJ... |
------------------+

Re: [Orbit-python-list] Threading support

[Jason T. <ta...@li...>]

> Essentially they only hold the global interpreter lock or the tk lock
> exclusive (except for special cases) of each other.  I imagine this
> may still be a problem?  Or do you think this is a non issue?

I'm tempted to say this isn't a problem, but I admit I don't completely
follow you.  Can you elaborate on the potential problem?

Jason.

[Orbit-python-list] Threading support

[Joseph T. B. <jb...@tj...>]

rats, and I was determined to figure it out myself.  Between corba,
orbit, and extending the python interpreter, plus the threads code, it
was taking me longer to wade throught it all than I thought.

I had noticed that the tcl/tk module had essentially the same conditions
for locking.  So I was using it as a model.

One piece of wisdom that I picked up from that work (_tkinter) was the
fact that they have another lock that only lets one thread run inside
the tcl interpreter at a time.  I was concerned about this for the
orbit-python module.

Essentially they only hold the global interpreter lock or the tk lock
exclusive (except for special cases) of each other.  I imagine this
may still be a problem?  Or do you think this is a non issue?

jb

Jason Tackaberry writes:
 > I added the appropriate locks/unlocks to the code to support threading in
 > Python.  I tested it briefly and it seems to work, but there still may be
 > problems.  It only took me a couple minutes, and I'm skeptical to think
 > anything is that simple. :)  Let me know if you run into any bugs.
 > 
 > >From the ChangeLog:
 > 
 >    * server.c, client.c, CORBA_ORB.c: added necessary interpreter unlocking
 >      to support Python threads properly.  (Not very tested, but seems to
 >      work.)
 > 
 > Jason.

-- 
------------------+
  Joseph T. Bore  |
 jb...@TJ... |
------------------+

Re: [Orbit-python-list] Wchar/wstring + Python 2

[Roland M. <ma...@ec...>]

Jason Tackaberry (2000-08-27 11:40:25 -0400) :

> > not compile with Python 2.  There's a PyMem_XDEL somewhere
> > (src/server.c maybe), and apparently that macro or function is no
> > more.  Replacing it by a PyMem_DEL did not explode at me right away,
> > but I wouldn't recommend that anyone do the same replacement without
> 
> Replacing it with PyMem_DEL is safe.  The XDEL was something I added
> as a test and forgot to revert it.

Okay.  Whichever of us happens to commit next lot of patches to CVS...

> Was this the only change required to make it compile with CVS
> Python?

  Yes.  Otherwise, everything runs the same.  Same speed, same memory
leaks.  Initial size of the process is bigger, but I suppose it's more
related to Python than to ORBit-Python.

Roland.
-- 
Roland Mas

You can tune a piano, but you can't tuna fish.
  -- in the tunefs manual page.

Re: [Orbit-python-list] Wchar/wstring + Python 2

[Jason T. <ta...@li...>]

> started to look into that, so I'm not sure.  In any case, don't expect
> anything before a few days / weeks (maybe around the Python 2.0b1
> release if I'm lucky).

Great news.  Keep me posted; I'm eager to see how this progresses.

> not compile with Python 2.  There's a PyMem_XDEL somewhere
> (src/server.c maybe), and apparently that macro or function is no
> more.  Replacing it by a PyMem_DEL did not explode at me right away,
> but I wouldn't recommend that anyone do the same replacement without

Replacing it with PyMem_DEL is safe.  The XDEL was something I added
as a test and forgot to revert it.

Was this the only change required to make it compile with CVS Python?

Jason.

[Orbit-python-list] Wchar/wstring + Python 2

[Roland M. <ma...@ec...>]

  Hi all,

  Just to avoid duplicating efforts: work is in progress here on the
wide char / wide string support in ORBit-Python.  Nothing has been
written yet, but I'm experimenting on the Python Unicode objects and
it shouldn't be that much of a hassle to handle wchar and wstring in
the marshalling/demarshalling.  Could be a bit trickier as far as
wchar/wstring constants are specified in the IDL, but I haven't
started to look into that, so I'm not sure.  In any case, don't expect
anything before a few days / weeks (maybe around the Python 2.0b1
release if I'm lucky).

  Work is based on current Python developement branch (as found on the
Sourceforge CVS), and a preliminary remark is this: ORBit-Python does
not compile with Python 2.  There's a PyMem_XDEL somewhere
(src/server.c maybe), and apparently that macro or function is no
more.  Replacing it by a PyMem_DEL did not explode at me right away,
but I wouldn't recommend that anyone do the same replacement without
further testing.

  Stay tuned...

Roland.
-- 
Roland Mas

Sauvez les castors, plantez des arbres.

[Orbit-python-list] Threading support

[Jason T. <ta...@li...>]

I added the appropriate locks/unlocks to the code to support threading in
Python.  I tested it briefly and it seems to work, but there still may be
problems.  It only took me a couple minutes, and I'm skeptical to think
anything is that simple. :)  Let me know if you run into any bugs.

From the ChangeLog:

   * server.c, client.c, CORBA_ORB.c: added necessary interpreter unlocking
     to support Python threads properly.  (Not very tested, but seems to
     work.)

Jason.

Re: [Orbit-python-list] more naivete

[Jason T. <ta...@li...>]

> Call me green, but i've never done this before.  How do I make the cvs
> tree look like the dist so I can run configure and make?  I ran
> autogen.sh, but it leaves me with a mish mosh of files that apparently
> are broken.  I'm on redhat-6.2 in case it matters.

Apply the patches and install the .m4 and .am files in python-am-changes/
in the cvs tree.  There is a README explaining how to do this.

Have fun,
Jason.

[Orbit-python-list] more naivete

[Joseph T. B. <jb...@tj...>]

Call me green, but i've never done this before.  How do I make the cvs
tree look like the dist so I can run configure and make?  I ran
autogen.sh, but it leaves me with a mish mosh of files that apparently
are broken.  I'm on redhat-6.2 in case it matters.

jb


-- 
------------------+
  Joseph T. Bore  |
 jb...@TJ... |
------------------+

Re: [Orbit-python-list] orb.run with threads

[Jason T. <ta...@li...>]

* Jason Tackaberry (ta...@li...) [000825 12:18]:
> CORBA_ORB_run().  Then in operation_skel, reclaim the lock and release
> it after exiting.  On the client side, you'd release the lock before
     ^^^^^

Of course I meant _before_ exiting.  Releasing a lock after exiting would
surely be an impressive accomplishment! :)

Sheepishly,
Jason.

Re: [Orbit-python-list] orb.run with threads

[Jason T. <ta...@li...>]

> Now here is where my naivete comes in about extending the python
> interpreter.  Is it possible to have pyorbit release the lock right

I don't know about pyorbit, but I'm sure it's possible with ORBit-Python. :)
(PyOrbit is a different project.)

> before it blocks and then after it unblocks, re-aquire the lock?  It
> just cant be that simple I'm sure.

As I understand it, this is pretty much how it must work.  However, with
the sort of arrangement in ORBit-Python it might not be so clear cut.  

On the server side, you'll need to release the lock before calling
CORBA_ORB_run().  Then in operation_skel, reclaim the lock and release
it after exiting.  On the client side, you'd release the lock before
invoking any of the GIOP calls (which can potentially block for a
long time if over a slow connection), and then reacquire it after.

Feel free to give this a whirl and let me know how it goes.

> I'd be willing to help out with this, but I'm going to probably need
> some help figuring out some of the internals that I dont know already.

If you have any questions, by all means ask on the list.  I'm sure either
myself or Roland (or someone else in the know) will get back to you.

> if all goes well i'd help out there too.

Glad to hear it!

Cheers,
Jason.

Re: [Orbit-python-list] orb.run with threads

[Roland M. <ma...@ec...>]

Joseph T. Bore (2000-08-25 11:04:56 -0400) :

> Does this have something to do with the "global interpreter lock"
> that I've heard so much about?

I'm not enough of a CORBA guru to be able to answer that one :-)

> As I look at this further i've begun to wonder how you can write a
> gtk/corba(ORBit) application when I need to call gtk_mainloop and
> orb.run but yet neither of them returns until the program wants to
> go away.

  Yep.  This is probably not the answer you're expecting, but another
question instead: do you really need your Gtk application to be the
CORBA server?  What I personally do is that my CORBA server is a sort
of daemon, with very little user interaction (except for the command
line), and the monitoring application is another one, functioning as a
CORBA client.

> It seems that you would need to be multithreaded, but yet all of the
> evidence i've seen so fars indicates that python-orbit, and perhaps
> even orbit itself are not thread safe.

  Judging by the fact that there's a project to make ORBit
thread-safe, I think it currently isn't :-/

  What you *could* do, though, is the following.  Assuming you can have
a CORBA client in a thread (which might be a long reach, but only
testing will show me right or wrong), you could have a single-threaded
server storing the objects, and your multi-threaded thing would be a
CORBA client, changing the objects on the server so that other clients
could use them.  It's ugly, I know, yeah.  But the other way is to
contribute to the multithread-safe'ing of ORBit, so take your pick.

  In any case, I suggest you ask on the ORBit mailing list.  See the
page at <URL:http://mail.gnome.org/mailman/listinfo/orbit-list> for
more info.

Roland.
-- 
Roland Mas

Just a little bit of you every day will surely keep the doctors away.
  -- Just a little bit of you (The Jackson Five)

Re: [Orbit-python-list] orb.run with threads

[Joseph T. B. <jb...@tj...>]

Jason Tackaberry writes:
 > > As I look at this further i've begun to wonder how you can write a
 > > gtk/corba(ORBit) application when I need to call gtk_mainloop and
 > > orb.run but yet neither of them returns until the program wants to go
 > > of the evidence i've seen so fars indicates that python-orbit, and
 > > perhaps even orbit itself are not thread safe.
 > 
 > This is true.  ORBit-Python definitely isn't reentrant, and I don't think
 > ORBit is either.  (Rumor has it there is or was some work done in this area,
 > but I don't know what the status is.)

actually the more I think about it, orbit doesnt really worry me that
much since it's in a different process space.

 > What you really want is Gnorba.  Basically, Gnorba (among other things)
 > provides a new main loop that handles both gtk+ and ORBit main loops.  This
 > is something I always wanted to do (and was partly my motivation for writing
 > ORBit-Python), but I never did get around to it.

This interests me, but unforunately I was only musing about the
difficulty of using gtk and orbit.

For now I just need to use python-orbit and threads.  gtk is the next
step, so you can bet I'm going to be back... 8)

Now here is where my naivete comes in about extending the python
interpreter.  Is it possible to have pyorbit release the lock right
before it blocks and then after it unblocks, re-aquire the lock?  It
just cant be that simple I'm sure.

I'd be willing to help out with this, but I'm going to probably need
some help figuring out some of the internals that I dont know already.

 > I would certainly be pleased to see some work done in this area, and I'd
 > be happy to work on that (or help work on that) once I stabilize
 > ORBit-Python itself.  Check out:
 > http://developer.gnome.org/arch/component/gnorba.html

if all goes well i'd help out there too.

jb

-- 
------------------+---------------------------------------------------------
  Joseph T. Bore  | When you have tremendous conviction on a trade, you have
 jb...@TJ... | to go for the jugular.  It takes courage to be a pig.
------------------+---------------------------------------------------------

Re: [Orbit-python-list] orb.run with threads

[Jason T. <ta...@li...>]

> As I look at this further i've begun to wonder how you can write a
> gtk/corba(ORBit) application when I need to call gtk_mainloop and
> orb.run but yet neither of them returns until the program wants to go
> of the evidence i've seen so fars indicates that python-orbit, and
> perhaps even orbit itself are not thread safe.

This is true.  ORBit-Python definitely isn't reentrant, and I don't think
ORBit is either.  (Rumor has it there is or was some work done in this area,
but I don't know what the status is.)

What you really want is Gnorba.  Basically, Gnorba (among other things)
provides a new main loop that handles both gtk+ and ORBit main loops.  This
is something I always wanted to do (and was partly my motivation for writing
ORBit-Python), but I never did get around to it.

I would certainly be pleased to see some work done in this area, and I'd
be happy to work on that (or help work on that) once I stabilize ORBit-Python
itself.  Check out: http://developer.gnome.org/arch/component/gnorba.html

Cheers,
Jason.

[Orbit-python-list] orb.run with threads

[Joseph T. B. <jb...@tj...>]

Hey folks, I'm trying to use the orbit orb to make several objects in
a system that I'm writing availalbe to other processes.

Problem is this: I have one process with several threads (around 5)
that runs fine.  But I want to be able to control it at times remotely
from a client program.  But wheneverer I incorporate corba into the
server and eventually run orb.run(), even in a separate thread,
everything else comes to a screeching halt.

Does this have something to do with the "global interpreter lock" that
I've heard so much about?

As I look at this further i've begun to wonder how you can write a
gtk/corba(ORBit) application when I need to call gtk_mainloop and
orb.run but yet neither of them returns until the program wants to go
away.  It seems that you would need to be multithreaded, but yet all
of the evidence i've seen so fars indicates that python-orbit, and
perhaps even orbit itself are not thread safe.

as you can probably guess, I'm a little confused and I'm probably
missing a few pieces of the puzzle.  any help greatly appreciated!

jb

-- 
------------------+
  Joseph T. Bore  |
 jb...@TJ... |
------------------+

Re: [Orbit-python-list] Updates

[Jason T. <ta...@li...>]

> continue to specify the IDL file(s) to load.  I'm not sure about that,
> but I think it's less risky to "load_idl (blah)" and then "import

Sure, but load_idl() is an orbit-python specific extension.  I plan on
having some way to disable the automagic-loading, though.  So those who
want complete control and don't mind writing orbit-python specific code can
do that.

I think the preprocessing performance will be acceptable.  It doesn't scale
linearly mind you, but there's some optimizations I can do to help that.  I
plan on doing some benchmarks to guage the performance hit.  0.03 seconds
for 50 cached reasonably complex IDL files is pretty good, I think.  I'm
more concerned with making the auto IDL loading feature smarter, but even
right now it's pretty intelligent.

If you do find instances where the auto-IDL loader doesn't do what it
should, I'll definitely want to hear about it.

> mymodule".  Supposing there are several IDL files describing a
> similarly named module...  Supposing there are a great many IDL files
> and we only need one...  Supposing many things, I think that can lead

In the case where there are several IDL files describing one module, it will
load all those IDL files.  It's also smart enough to trace through includes.
For example, Bonobo consists of several IDL files, while Bonobo.idl just
includes all those IDL files.  If you import Bonobo, orbit-python will load
only Bonobo.idl because it includes all the files that would be needed anyway.

In the case where you need only one, you can (or will be able to) do:

   from GNOME import Gnumeric

say, which will load only the Gnumeric IDL file.  (It doesn't do this yet,
but it will.)

> automatisation (does that word exist in English?) of things.  As I
> said, I'm all for the "load_idl (blah)" then "import blahmod".  That
> could also be a "preprocess_idl_dirs (blah)" then "import blahmod".

Probably something like:

   import CORBA
   CORBA._orbit_auto_import = 0

>   On another point, I just scanned through src/process_idl.c, and it
> appears you parse the IDL files by hand.  Well, why not, but you said
> it yourself, that's not a complete IDL parser.  Why not use the libIDL
> like in src/idl.c, with the appropriate tree_pre_func and
> tree_post_func?

Because I don't want the overhead of libIDL.  It's quick, sure, but I need
the preprocessing to be as fast as possible.  And I think a crude parser
is all that I need.  After all, I only need to discover modules in the global
and first-level scopes.

>   Here again, I feel that could (should?) be optional arguments to
> load_idl.  On the other hand, I don't know what these arguments do to
> IDL_parse_filename, so they might be completely innocent.

Yes, this will be optional, and yes, these options are completely innocent.
Right now I'm focusing on making things compliant with the spec.  After that,
I can provide options so that the users can obtain whatever level of control
they want.

>   No memory leak seen from here for the first tests.  I haven't tried
> the big one yet, since the module mappings have changed and I'll have
> to check and fix where it broke.

Well, there are definitely leaks. :)  But I also have a lot of the code still
in there from the old way of doing things, and so once I remove that, some
or all of the leaks might vanish.

>   Ah, and I fixed the patch I had uploaded.  Now it works (well,
> should work anyway).

Great.  I'll have a look at that tonight.

Jason.

Re: [Orbit-python-list] Updates

[Roland M. <ma...@ec...>]

Jason Tackaberry (2000-08-21 23:05:33 -0400) :

>    * Implemented preprocessing of IDL files at runtime to determine which
>      IDL files offer which modules.  Implications on performance are yet
>      to be determined.  This approach may not scale very well at all for
>      hundreds of IDL files.  It also could be broken in some cases; the
>      preprocessor is not a complete IDL parser, so it may not work 100%.
>      Time shall tell.  Uses the IDLPATH environment variable to get a
>      list of paths to scan, otherwise defaults to the current
>      directory.
>    * hook __import__ to automagically load IDL files discovered by the
>      processor that are required for the requested module
[...]
> The IDL directory preprocessor is a particularly interesting addition, and
> it remains to be seen if this approach won't completely flop.  For me, at
> least, it works pretty well, but my setup is somewhat limited (basically
> just GNOME with Bonobo).  On my system it takes about 0.30 seconds to
> preprocess 3 directories with about 50 IDL files, and 0.04 seconds once
> cached.

I'm not sure I think it's always a good idea to do it automagically.
I mean, yes, it can be handy, but I feel it would be preferable to
continue to specify the IDL file(s) to load.  I'm not sure about that,
but I think it's less risky to "load_idl (blah)" and then "import
mymodule".  Supposing there are several IDL files describing a
similarly named module...  Supposing there are a great many IDL files
and we only need one...  Supposing many things, I think that can lead
to breakage.  I don't really like the idea of ORBit-Python doing
things in my back.  On the other hand, I'm totally in favour of an
automatisation (does that word exist in English?) of things.  As I
said, I'm all for the "load_idl (blah)" then "import blahmod".  That
could also be a "preprocess_idl_dirs (blah)" then "import blahmod".

  Of course, you're the boss :-)

  On another point, I just scanned through src/process_idl.c, and it
appears you parse the IDL files by hand.  Well, why not, but you said
it yourself, that's not a complete IDL parser.  Why not use the libIDL
like in src/idl.c, with the appropriate tree_pre_func and
tree_post_func?

>    * Pass -D__ORBIT_IDL__ -D__BONOBO_COMPILATION to libIDL so that we can
>      load Bonobo IDLs

  Here again, I feel that could (should?) be optional arguments to
load_idl.  On the other hand, I don't know what these arguments do to
IDL_parse_filename, so they might be completely innocent.

>    * introduced more memory leaks, and probably more bugs.

  No memory leak seen from here for the first tests.  I haven't tried
the big one yet, since the module mappings have changed and I'll have
to check and fix where it broke.

  Ah, and I fixed the patch I had uploaded.  Now it works (well,
should work anyway).

  Have a nice day.

Roland.
-- 
Roland Mas

Qu'est-ce qui est jaune, qui pèse deux cents kilos et qui chante ?
Un sumotori qui a bu trop de saké.