freespeech-overflow — List for the Overflow project

Re: [Overflow] Re: DL and PL integration

[Jean-Marc V. <jea...@he...>]

First, for Jarl, I would suggest that you subscribe to the Overflow mailing list
by going to 
http://lists.sourceforge.net/mailman/listinfo/freespeech-overflow
It's fairly low-trafic, since there's only Brad and me so far!

> Jean-Marc wrote:
> >  I still think linking the PL with the BL is the right thing to do...
> >  instead of using CORBA, we could just use simple callbacks, which
> would be
> >  much simpler and faster when exchanging large data.
> 
> The thing I really want is to be able to write nodes that work in Piper
> in languages other than C++. For instance, the AlBeanBox and ISYS nodes
> are written in java -- I think it would be really nice if we could plug
> these into the same node framework. In general, it would be nice if
> people had a number of language choices for writing node extensions in.

I think there are other ways to achieve Overflow 'language bindings'. Note that
using CORBA for BL-PL communication doesn't help in that respect. What's
important is that Overflow C++ nodes can communicate with nodes written in other
languages. I think that the best way to achieve that is to write a special node
for every language we want to support. That node would just be an empty node
that would call the 'foreign' node, either with a function call, as a process,
with CORBA, ... that is left to the one implementing the language binding. As
the the PL itself, it could still be linked in the BL. I really see the BL as an
executable that runs on each machine and that links with the Overflow libraries
to do the local processing.

> BTW, would it be helpful if we had an IRC session to talk about this and
> try to iron some things out? I believe Jarl is 6 hours behind Jean-Marc
> and I, so if we want to try this, maybe we should try to for sometime
> between 4pm and 6pm (Jean-Marc and my time) someday next week. What do
> you guys think? I'm pretty open during these times next week (although
> Wednesday is the worst, but I can rearrange), so it anyone wants to
> propose a time to do talk, I would be very up for it. I just think it
> might help us hash through some stuff a little faster and is easier than
> e-mail sometimes...

So far, monday and tuesday are fine with me.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

Re: [Overflow] Re: DL and PL integration

[Brad C. <cha...@ar...>]

Jean-Marc wrote:
>  I still think linking the PL with the BL is the right thing to do...
>  instead of using CORBA, we could just use simple callbacks, which
would be
>  much simpler and faster when exchanging large data.

The thing I really want is to be able to write nodes that work in Piper
in languages other than C++. For instance, the AlBeanBox and ISYS nodes
are written in java -- I think it would be really nice if we could plug
these into the same node framework. In general, it would be nice if
people had a number of language choices for writing node extensions in.

I will freely admit that I don't really understand how this will work,
but I really want to work towards it. I see having a CORBA layer for
these callbacks will at least be a small step towards this goal.

What do you guys think? Am I being crazy here as well? Is there any way
we are going to be able to do this?

BTW, would it be helpful if we had an IRC session to talk about this and
try to iron some things out? I believe Jarl is 6 hours behind Jean-Marc
and I, so if we want to try this, maybe we should try to for sometime
between 4pm and 6pm (Jean-Marc and my time) someday next week. What do
you guys think? I'm pretty open during these times next week (although
Wednesday is the worst, but I can rearrange), so it anyone wants to
propose a time to do talk, I would be very up for it. I just think it
might help us hash through some stuff a little faster and is easier than
e-mail sometimes...

Brad

[Overflow] Re: DL and PL integration

[Brad C. <cha...@ar...>]

jarl wrote:
>  This will change the old plan of linking the PL and BL to one binairy.
>  There's no real prob as long as we setup a good authentication
meganism
>   between the 2 layers.

Well, basically I guess I was just thinking about the CORBA layer only
being sort of between node communication from the PL to the BL. The
reason I think this is a good idea, is so that nodes can be written in
languages other than C++. I am really into this, since I think it will
help us use nodes from other projects (like ISYS and ALBeanBox) although
I'm not sure how yet :-).


[having id numbers that the BL would have to keep track of]
>  This we already have, the Universal Resource Id (URI), look @ the
>  DL-BL idl.

Right, although I'm not sure I understand how this will hook in with the
way Overflow uses unique 'names' as id for nodes. Have you already coded
this in the BL? Sorry, I am waaaaay behind on where the BL code is at
:-<.
 
I wrote:
>  > 2. How can the stream class return the stop or continue messages
> >  back to the node? I'm not sure how we can achieve this kind of
callback
> >  with the stream setup, but this is probably just me missing
something.

Jarl wrote:
>  I dont quite see what you're pointing at, can you explain a bit more?

Well, what I was thinking of is this:

-> The node registers some kind of error. It pauses the run because of
the error and then calls the error stream.

errorStream << 1 << 'some kind of error';

-> The error stream recieves the error. It creates a PlErrorCallback and
calls the BL with an error message.

-> The BL recieves the error message, passing it on to the DL and then
the UI, where the user decides to continue the run.

-> The BL gets this message and calls runContinue() on the
PlErrorCallback.

-> Now my big question mark -- how does this info get back to the node so
it knows to keep processing getOutputs()? Am I misunderstanding the
design here?

Brad

Does this make sense?

Re: [Overflow] Re: DL and PL integration

[Jean-Marc V. <jea...@he...>]

> | 2. Write the CORBA interface and supporting code so that the stream class
> | can communicate with the BL.
> 
> This will change the old plan of linking the PL and BL to one binairy. There's no
> real prob as long as we setup a good authentication meganism between the 2 layers.

I still think linking the PL with the BL is the right thing to do... instead of
using CORBA, we could just use simple callbacks, which would be much simpler and
faster when exchanging large data.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

[Overflow] Re: DL and PL integration

[jarl v. k. <ja...@ca...>]

>

Hi All,

| 2. Write the CORBA interface and supporting code so that the stream class
| can communicate with the BL.

This will change the old plan of linking the PL and BL to one binairy. There's no
real prob as long as we setup a good authentication meganism between the 2 layers.

>
> The basic idea is that the BL
> implements an interface through which the PL can return errors and
> values. The PL implements a callback interface for whenever it returns an
> error, so that the BL can either stop or continue a run when an error
> occurs.

Sound logical.. and I cant thing of any to complain about, so you got my vote :)

>
> b. Have an id number assigned to each submitted process that would be
> given to a generic BlReporter (or we could have a general BL interface
> that gave specific BlReporters based on id numbers). In this case the PL
> and BL would have to track these id numbers.

This we already have, the Universal Resource Id (URI), look @ the DL-BL idl.

>
> 2. How can the stream class return the stop or continue messages back to
> the node? I'm not sure how we can achieve this kind of callback with the
> stream setup, but this is probably just me missing something.

I dont quite see what you're pointing at, can you explain a bit more?

Tnnx 4 the Bl2Pl IDL Brad..  I'll think about it, you'll hear my comments on it soon.

bye,
jarl

[Overflow] DL and PL integration

[Brad C. <cha...@ar...>]

Hey guys;
	As I mentioned to Jarl yesterday, Jean-Marc and I had been
talking a bit about starting to more closely integrate the BL and PL,
especially with respect to returning errors and calculated values in a
node to be displayed. We kind of came up with a two step plan for
starting on this:

1. Write a stream class for the PL which derives from the C++ iostream
class. This class will be responsible for communicating directly with the
BL through a CORBA interface. The idea is that when nodes are running
they can call this stream class like:

returnValueStream << 'This is a node name' << 'This is some value info';

This kind of abstracts the actual communication to this stream class, and
allows you to turn streams on and off.

2. Write the CORBA interface and supporting code so that the stream class
can communicate with the BL. 

I started working towards this second goal and wrote an initial IDL file
for the communication, which I attached. The basic idea is that the BL
implements an interface through which the PL can return errors and
values. The PL implements a callback interface for whenever it returns an
error, so that the BL can either stop or continue a run when an error
occurs.

This is just a first start to work with, and I already have a number of
questions about the design:

1. How will we distinguish between different BL processes submitted to
the PL? There are two ways I could think of to do it:

a. Have a different BlReporter submitted for each run. The PL would then
somehow have to keep track of which BlReporter object is which.

b. Have an id number assigned to each submitted process that would be
given to a generic BlReporter (or we could have a general BL interface
that gave specific BlReporters based on id numbers). In this case the PL
and BL would have to track these id numbers.

2. How can the stream class return the stop or continue messages back to
the node? I'm not sure how we can achieve this kind of callback with the
stream setup, but this is probably just me missing something.

Anyways, this is just something to get started with. I'm excited to see
this start getting hashed out and seeing the parts be more integrated!
Talk with you all soon.

Brad