freespeech-overflow — List for the Overflow project

[Overflow] Overflow 0.6.1 released

[Jean-Marc V. <jea...@he...>]

Hi,

We're pleased to announce the release of Overflow 0.6.1. This=20
release has mostly focused on bug fixing and improving the=20
core library. This release is compatible at the XML level=20
with 0.6.0 but not at the source/binary level. Those who have=20
nodes coded in C++ should 's/specificInitialize/initialize/'=20
their source file. As usual, download from:
http://sourceforge.net/project/showfiles.php?group_id=3D590

Changes are:

**Method renamed: specificInitialize() has been renamed to=20
initialize(). This affects those who write their own nodes.
- Cancelling a program with the "stop" button has now less=20
chances fo crashing Overflow.
- Node CAllPole was renamed to IIR, making it more clear=20
what the node is doing.
- Various GUI improvements
- Many bug fixes (possibly new ones)
- Real-Time Clock (/dev/rtc) support for timing/waiting
- Added user-defined struct-like types throught the use of=20
the MakeComposite and GetComposite nodes

	Jean-Marc

--=20
Jean-Marc Valin, M.Sc.A.
LABORIUS (http://www.gel.usherb.ca/laborius)
Universit=E9 de Sherbrooke, Qu=E9bec, Canada

Re: [Overflow] Feedback problem

[Jean-Marc V. <va...@ge...>]

> I got this output when I run my network.
> 
> Do you have any ideas about what can cause this?
> (I use 2 feedback in my network. For the 3 first iteration, the result 
> is correct but after I got the output)

It's hard to tell from the output, could you send a simple .n that
reproduces the problem. That's the best way to solve these problems. 

> Yannick
> --------------
> Running network...
> PC6Buffer error: trying to read non-existing element.
> Element 0
> Buffer is:
> <Buffer
> < 2 <Bool 0 >< 3 <Bool 0 >< 4 <Bool 0 > >
> BufferedNode.cc line 123: Node node_OR_1 (type 2OR) Exception caught in 
> BufferedNode::getOutput
> Add.cc line 51: Node node_Add_1 (type 3Add) Exception caught in 
> BufferedNode::getOutput
> Collector.cc line 77: Node ALL_NETWORK_OUTPUTS (type 9Collector) 
> Exception caught in Collector::getOutput
> Collector.cc line 77: Node ALL_NETWORK_OUTPUTS (type 9Collector) 
> Exception caught in Collector::getOutput
> Iterator.cc line 115: Node node_LOOP0_1 (type 8Iterator) Error in 
> Iterator::getOutput
> Collector.cc line 77: Node ALL_NETWORK_OUTPUTS (type 9Collector) 
> Exception caught in Collector::getOutput
> 
> 
> _______________________________________________
> FreeSpeech-Overflow mailing list
> Fre...@li...
> https://lists.sourceforge.net/lists/listinfo/freespeech-overflow
> 

[Overflow] Feedback problem

[Yannick B. <yb...@sh...>]

Hi,

I got this output when I run my network.

Do you have any ideas about what can cause this?
(I use 2 feedback in my network. For the 3 first iteration, the result 
is correct but after I got the output)

Yannick
--------------
Running network...
PC6Buffer error: trying to read non-existing element.
Element 0
Buffer is:
<Buffer
< 2 <Bool 0 >< 3 <Bool 0 >< 4 <Bool 0 > >
BufferedNode.cc line 123: Node node_OR_1 (type 2OR) Exception caught in 
BufferedNode::getOutput
Add.cc line 51: Node node_Add_1 (type 3Add) Exception caught in 
BufferedNode::getOutput
Collector.cc line 77: Node ALL_NETWORK_OUTPUTS (type 9Collector) 
Exception caught in Collector::getOutput
Collector.cc line 77: Node ALL_NETWORK_OUTPUTS (type 9Collector) 
Exception caught in Collector::getOutput
Iterator.cc line 115: Node node_LOOP0_1 (type 8Iterator) Error in 
Iterator::getOutput
Collector.cc line 77: Node ALL_NETWORK_OUTPUTS (type 9Collector) 
Exception caught in Collector::getOutput

[Overflow] Overflow sur debian

[Yannick B. <yb...@sh...>]

Bonjour,

Overflow compile sans problème sous debian (woody) avec gcc 2.95.4.

Par contre, il y a une chose de bizarre. J'ai exécuté configure sans 
paramètre:
le répertoire bin s'est retrouvé à /usr/local/bin
la librairie libflow s'est retrouvé à /usr/local/bin
mais les autres librairies ont été installer dans 
/usr/local/share/overflow/lib

Dans ces conditions, vflow démarre normalement mais il n'y a pas de 
noeud dans la liste.
J'ai joué avec les variables VFLOW_HOME et LD_LIBRARY_PATH et il n'y a 
aucun moyen d'avoir les noeuds sauf
si je copie manuellement les librairies dans /usr/local/lib et que je 
mets VFLOW_HOME=/usr/local.

Au plaisir,

Yannick

[Overflow] Overflow 0.6.0 released

[Jean-Marc V. <va...@ge...>]

We're pleased to announce the release version 0.6.0 of Overflow 
(http://freespeech.sourceforge.net/overflow.html), aka Open Mind Speech base, 
aka Piper PL. You can download it from sourceforge at: 
http://sourceforge.net/project/showfiles.php?group_id=590

This release brings a number of important improvements:

- Many, many improvements to the gnome GUI (copy/paste, tooltips, bug fix, ...)
- C++ code generator (generated code can now be compiled on Win32)
- Basic networking support
- Improved debugging
- Speed improvements (also, auto-detection of 3DNow! and SSE)
- Overflow "shell scripts" (#! files)
- Lots of fixed bugs
- Big code cleanup and simplification (warning, this breaks some minor things)

There are also good news for the project. Overflow has been adopted by InfoSpace 
Speech Solutions for implementing part of a speech recognition engine and should 
ship in a future version of the product. Note, I (Jean-Marc) work for InfoSpace. 
Overflow is now also used in the mobile robotics lab at University of Sherbrooke 
and a robot control toolbox is soon to be released!

There is now an on-line survey on SourceForge. Tell us what you think about 
Overflow at: http://sourceforge.net/survey/survey.php?group_id=590&survey_id=11967

	Jean-Marc Valin

[Overflow] Node documentation

[Bill K. <bi...@ii...>]

Hi,
is there any documentation on nodes such as db, the HP and LP filters
and fft.  I need some idea of settings and values (and value types) for
these and other standard nodes.

Also, is there a repository of more nodes than what comes with the basic
package?

BillK

[Overflow] Bug found in speaker verification code (GMMDiag class)

[Jean-Marc V. <va...@ge...>]

Hi,

I found a very bad bug in the GMM code, which is used in the speaker
verification package. The error (uninitialized value) could cause incorrect
result and even NaN values. The only file affected is HMM/src/DiagGMM.cc and
it's fixed in CVS. You can get a snapshot of the CVS at
http://freespeech.sourceforge.net/FreeSpeech/html/Overflow/download.html

So if you're using the SV code, please update.

	Jean-Marc

[Fwd: [Overflow] I'm stuck again]

[Thomas R. <t....@in...>]

Dominic asked me to forward this, so here you are:

Dominic Letourneau wrote:
> =

> Hi Thomas, I am glad that you sucessfully compiled Overflow.
> Please read below!
> --- Thomas Rast <t....@in...> wrote:
> > Hi again
> >
> > Now I got Overflow to compile, I'm stuck with my first experiment.
> > I'm
> > trying to do a simple addition like this:
> >
> > Constant
> >          \
> >             Add  --  SaveAs
> >          /
> > Constant
> >
> > It's quite similiar to the hello.n example. I set the constants to
> > two
> > arbitrary ints and the output file to /dev/stdout (like in the
> > example)
> > but it errors with
> >
> > UINetwork.cc line 564: No output defined for network
> >
> > I can't get an OUTPUT tag on the right side of SaveAs like in the
> > example :(
> =

> To specify input and output tags simply do :
> =

> 1) Hold the shift key
> 2) While holding the shift key click on the output or input (black
> bullet)
> 3) Enter a name fot he output.
> =

> So if you click on an input you will be asked for an input name, if you=

> click on an output, you will be asked for an output name...
> =

> By the way, if you want to specify conditions (for iterators) you just
> =

> 1) hold the CTRL key and
> 2) Click on the output bullet
> 3) Specify the name of the condition
> =

> >
> > Thomas
> >
> > PS: It seems I've found a bug or two: I can increase the "Outputs"
> > count
> > in the node properties but not decrease it again. It's possible to
> > decrease the *counter* for "Inputs" but it adds an input instead of
> > removing one.
> =

> Input and Output increase doesn't work, and is not useful right now. We=

> should fix that really soon. The idea was to be able to add inputs
> (instead of having a Add with only two inputs, it could be useful to
> have a Add with three or more inputs for instance) for special nodes
> like AND, ADD, etc. We are still working on it! If you need more than
> two inputs right now, please use 2 nodes (cascade) and it should work
> fine.
> =

> I hope it will help!
> =

> Dominic
> =

> =3D=3D=3D=3D=3D
> Dominic L=E9tourneau
> do...@ya...
> dl...@he...
> G=E9nie Informatique
> Universit=E9 de Sherbrooke
> =

> __________________________________________________
> Do You Yahoo!?
> Get personalized email addresses from Yahoo! Mail
> http://personal.mail.yahoo.com/

Thomas

-- =

Thomas Rast
t....@in...
ICQ# 103670088

Re: [Overflow] I'm stuck again

[Dominic L. <do...@ya...>]

Hi Thomas,

Please read the documentation, it explains how to use vflow (The
overflow visual editor).

http://freespeech.sourceforge.net/FreeSpeech/html/Overflow/doc.html

Thomas, can you forward the last message that I sent you to the
Overflow mailing list, so people know how to create output and input
nodes! I think Jean-Marc also answered, so maybe his message too should
be forwarded to the mailing list.

Regards, 

Dominic


PS. The documentation is not perfect, but it still can help when you
are stuck!



--- Thomas Rast <t....@in...> wrote:
> Hi again
> 
> Now I got Overflow to compile, I'm stuck with my first experiment.
> I'm
> trying to do a simple addition like this:
> 
> Constant
>          \
>             Add  --  SaveAs
>          /
> Constant
> 
> It's quite similiar to the hello.n example. I set the constants to
> two
> arbitrary ints and the output file to /dev/stdout (like in the
> example)
> but it errors with
> 
> UINetwork.cc line 564: No output defined for network
> 
> I can't get an OUTPUT tag on the right side of SaveAs like in the
> example :(
> 
> Thomas
> 
> PS: It seems I've found a bug or two: I can increase the "Outputs"
> count
> in the node properties but not decrease it again. It's possible to
> decrease the *counter* for "Inputs" but it adds an input instead of
> removing one.
> 
> -- 
> Thomas Rast
> t....@in...
> ICQ# 103670088
> 
> _______________________________________________
> FreeSpeech-Overflow mailing list
> Fre...@li...
> http://lists.sourceforge.net/lists/listinfo/freespeech-overflow


=====
Dominic Létourneau
do...@ya...
dl...@he...
Génie Informatique
Université de Sherbrooke

__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail
http://personal.mail.yahoo.com/

[Overflow] I'm stuck again

[Thomas R. <t....@in...>]

Hi again

Now I got Overflow to compile, I'm stuck with my first experiment. I'm
trying to do a simple addition like this:

Constant
         \
            Add  --  SaveAs
         /
Constant

It's quite similiar to the hello.n example. I set the constants to two
arbitrary ints and the output file to /dev/stdout (like in the example)
but it errors with

UINetwork.cc line 564: No output defined for network

I can't get an OUTPUT tag on the right side of SaveAs like in the
example :(

Thomas

PS: It seems I've found a bug or two: I can increase the "Outputs" count
in the node properties but not decrease it again. It's possible to
decrease the *counter* for "Inputs" but it adds an input instead of
removing one.

-- 
Thomas Rast
t....@in...
ICQ# 103670088

[Overflow] Can't compile NNet in 0.5.0

[Thomas R. <t....@in...>]

Hi all

I can't get Overflow 0.5.0 to compile. When trying to compile NNet, it
always errors with

/bin/sh ../libtool --mode=compile c++ -DLINUX=1 -DHAVE_FLOAT_H=1
-DHAVE_HASH_MAP=1 -DHAVE_LIBM=1 -D_REENTRANT=1  -I. -I.  -I../include
-I../../data-flow/include        [l_INCLUDEDIR    -g -O2 -c FFLayer.cc
c++ -DLINUX=1 -DHAVE_FLOAT_H=1 -DHAVE_HASH_MAP=1 -DHAVE_LIBM=1
-D_REENTRANT=1 -I. -I. -I../include -I../../data-flow/include
"[l_INCLUDEDIR" -g -O2 -Wp,-MD,.deps/FFLayer.pp -c FFLayer.cc  -fPIC
-DPIC -o FFLayer.lo
c++: cannot specify -o with -c or -S and multiple compilations
make: *** [FFLayer.lo] Error 1

I hope you can fix this soon, Overflow seems to be an interesting
approach to programming.

Thanks in advance

Thomas

System:        i586 architecture
Distribution:  Suse Linux 6.4
Kernel:        2.2.16
gcc:           2.95.2-98
GNOME libs:    1.2.4-10
GNOME core:    1.2.1-24

-- 
Thomas Rast
t....@in...
ICQ# 103670088

[Overflow] Overflow 0.5.0 released

[Jean-Marc V. <va...@ge...>]

We're pleased to announce the release of Overflow 0.5.0. Overflow
(http://freespeech.sourceforge.net/overflow.html) is a free (GPL/LGPL)
"data flow oriented" development environment. It can be use to build
complex applications by combining small, reusable building blocks. In
some way, it has similarities with Simulink and LabView, although it
is not designed (and far) to be a "clone" of any of them.

Overflow features a GUI that allows rapid application development and
features a visual debugger.  Although Overflow can be used as a rapid
prototyping tool, it can still be used for building real-time
applications, such as audio effects processing. Since overflow is not
really an "interpreted language", it can be quite fast.


This new release brings a number of improvements over version
0.4.3. These improvements include:

- Binary I/O (serialization/unserialization) of objects
- Major improvements to object ASCII I/O, mostly with the Vector type
- Default FFT implementation when FFTW is not available
- Possibility of passing any object type as a parameter from the GUI
- gcc 2.96-RH support
- Performance improvements
- Fixes for Piper (http://bioinformatics.org/piper/) compatibility 
- Several new nodes

Issues:

- It is not clear whether Overflow compiles with gcc 3.0 installed in 
     the standard (/usr) path

[Overflow] Overflow 0.4.3 released

[Jean-Marc V. <va...@ge...>]

Just letting you know that version 0.4.3 of Overflow
(http://freespeech.sourceforge.net/overflow.html) has been released. Download
location is as usual: http://sourceforge.net/project/showfiles.php?group_id=590

The main point in this release is that it allows to use the speaker verification
prototype I just made available at:
ftp://freespeech.sourceforge.net/pub/freespeech/speaker_verification-0.0.1.tgz

Otherwise, it shouldn't break source compatibility with previous releases.

Have fun!

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

[Overflow] Overflow 0.4.2 released (2nd try)

[<jea...@he...>]

Hi,

Just to let you know that I just released Overflow 0.4.2. This release improves
Overflow portability. It should now compile without change on Linux, Solaris
(x86 at least) and FreeBSD with any GNU compiler between egcs 1.1.2 and the gcc3
pre-releases.

Also, for developers, a new config tool: overflow-config, that works in the same
way as gnome-config/gtk-config. You can download Overflow 0.4.1 at:
http://sourceforge.net/project/showfiles.php?group_id=590
The web page is always at:
http://freespeech.sourceforge.net/overflow.html

Enjoy,

        Jean-Marc

[Overflow] Overflow 0.4.1 released

[Jean-Marc V. <va...@ge...>]

Hello to all Overflow (Piper PL) fans!

Just to let you know that version 0.4.1 has been released. It's mostly a bugfix
release, and it should be more stable than 0.4.0.

One new (experimental) feature is the ability to have multiple threads in an
Overflow program. This is done using the new "SerialThread", "ParallelThread"
and "ThreadJoin" nodes, as illustrated in examples/mixedthread.n

Have fun!

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

[Overflow] Re: [gst-devel] Possible collaboration with Overflow

[Jean-Marc V. <va...@ge...>]

> I'm compiling Overflow right now, and like what I see.  It sounds like the
> perfect tool to work on one of my other pet projects, which is codec
> development (new code, not new implementations), among other things.

Don't know whether you mean audio coding or video coding (or both), but Overflow
already has lots of stuff for audio coding (I did my master in a speech coding
group). There's some image processing stuff, but it's a bit outdated. 

Also, tell me if you have any problem compiling... I'm having a hard time
dealing with all the different C++ compilers. 

> At GUADEC I was approached by a guy working on another somewhat related
> project (graph of decision processing, that he compiles to C, used for
> things like deciding whether to grant a loan or not) that has a *MEGA*
> canvas-based graph editting system called Seashore.  He says it'll be out
> (LGPL'd) in a month or so, if there is enough interest in collab between
> various graph-based projects like GStreamer, Overflow, aRts, etc., I can
> probably convince him to release a copy to us earlier, for feedback and
> such.

I'd like to see that... I think the GUI is the weakest point in Overflow.

> Seashore is based on the AA canvas, and as such has all sorts of neat
> interaction stuff such as alpha-fading links and such.  More importantly,
> it has massive graph-structuring code, with spline links and all that.
> From just 5 seconds looking at it, I was already drooling.
> 
> I think that Seashore would provide the perfect basis for all these
> projects, and would implicitly allow editting of one graph inside the
> other, which would be a *killer* feature.  The one flaw (and the reason I
> want to get ahold of code now rather than later) is that he didn't design
> it with arbitrary embedding in mind, as the GStreamer editor was.  In
> gsteditor, as in GStreamer itself, a container *is* an object.  You can
> place a container in the graph and it acts like any other object, except
> it's got another graph inside of it.  This is the GUI widget model, and
> it's very very powerful.

In Overflow too, the "Network" class (container) derives from the "Node" class
(toplevel processing object class). However, I'm not sure that we really want to
have the GUI object for a Network "embedded" in another network. The reason for
that is that it has to be possible to include two instances of the same network.
Just like in C (or any other language) you can make two (or more) calls to the
same function. In Overflow, I have a tab for each network I define and I'm free
to use it as a Node any number of times I want. (Does this paragraph make any
sense?)

> Another thing that happened at GUADEC was serious discussion between
> myself and Stefan Westerfeld, the aRts author.  We agreed to build plugins
> for each others systems that embed each other.  I've hit a roadblock with
> the aRts plugin for GStreamer, but it's just because I don't understand
> aRts enough.
> 
> The same thing could be done for Overflow, and even the third leg of
> Overflow<>aRts plugins.  Then with graph-builder integration we could do
> some seriously cool stuff.

That'd be cool... I think Overflow fits well for handling the lower level
operations, because it's quite fast at handling small elements (like audio
frames, ...), because it's limited to pull-only. For the higher-level stuff,
gstreamer might be better though (sorry, didn't have time to try GStreamer yet).

> > * Common base multimedia libraries: We could probably share the low-level
> > audio/image/... processing routines.
> This is very much related to my codec interests.  On codecs.org I have
> several packages, including libcodec, which is specifically designed to do
> just this.  Someone on IRC pointed out that libcodec is potetially too
> narrow a name, since the same specialization architecture (each function
> can have arbitrary, run-time switchable implementations optimized for
> different architectures) can be used in all sorts of projects.

That could make interesting Overflow Nodes... 

So, what do you think the next step should be?

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

[Overflow] Overflow 0.4 released

[Jean-Marc V. <va...@ge...>]

Hello,

We're pleased to announce version 0.4.0 of Overflow (aka the Open Mind Speech
environment, aka Piper PL). This new release brings a number of major
improvements over version 0.3.0. Some of the changes are:

* The VFLOW_PATH environment variable is no longer required (it's determined
from the --prefix configure option) if you don't add your own nodes.

* The FFTW library is no longer required to build Overflow. If you don't have
it, the build will continue without FFT support.

* It's now possible to have Feedback loops in a flow. This is done by using the
"Feedback" node and a demo is available in the examples directory.

* Overflow also supports throwing and catching exceptions within the
  flow using the "Throw" and "Catch" node. This is still undocumented.

* Support for AMD 3D Now! instructions in vector primitives (use -D_USE_3DNOW)

* Rewrite of the Neural network toolbox to make it faster

* More stuff in the "examples" directory.

* Updated html documentation

* Better layout the links on the GUI (links can be broken lines)

* As usual, a whole bunch of new node


Overflow is a visual development environment, using a GNOME UI, that allows easy
development of applications using toolboxes. Current toolboxes include signal
processing, image processing, neural networks, vector quantization and more...

Download, compile, enjoy, report bugs!

[Overflow] Getting close to 0.4

[Jean-Marc V. <jm...@lo...>]

Hi,

Overflow 0.4 will be released soon. This will be a major release, with
lots of new stuff going in. If you're having any problems
compiling/running the CVS version, please let me know. The changes since
0.3 include

- The VFLOW_PATH environment variable is no longer required (it's
determined from the --prefix configure option) if you don't add your own
nodes.

- What was in libflowui is now in the same library than the one called
libnetwork. The new library is now called libflow and is all that needs
to be linked with Piper.

- The FFTW library is no longer required to build Overflow. If you don't
have it, the build will continue without FFT support.

- It's now possible to have Feedback loops in a flow. This is done by
using the "Feedback" node and a demo is available in the examples
directory.

- Overflow also supports throwing and catching exceptions within the
flow using the "Throw" and "Catch" node. This is still undocumented.

- Support for AMD 3D Now! instructions in vector primitives

- Rewrite of the Neural network toolbox to make it faster

- More stuff in the "example" directory.

- Updated html documentation

- Ability to better layout the links on the GUI

Comments? Feature requests? Something annoys you? e-mail me.

	Jean-Marc

[Overflow] Overflow 0.3.0 released

[Jean-Marc V. <va...@ge...>]

Hello everybody,

I just released version 0.3.0 of Overflow (aka Open Mind Speech). There has been
lots of (architectural) changes recently, but things are now stabilizing again.
There has been lots of changes since version 0.2 and you are all encouraged to
upgrade. One of the most noticeable changes is that it should now compile with
egcs 1.1.2.

	Jean-Marc

RELEASE NOTES:

This new release includes a number of bug fixes, as well as optimizations that
reduce the global overhead of using the system (vs. using plain C/C++). Overflow
should now be more portable and ANSI C++ compilant. Last but not least, an
important architectural change occured for all frame-based operations.


-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

[Overflow] Re: IRC

[Jean-Marc V. <jea...@he...>]

> >  I'd also something about the BL part I'd like to see between the DL
> >  and the
> >  Mother-BL. I'd like to see the BL take care of all communication
> >  between different machines. Since some of the PL nodes (viewers,
> debugging
> >  nodes, ...) will have to interact with the UI, the only way to achieve
> that is
> >  through a middle-BL (what I used to call "master") that takes care of
> the
> >  communication between the DL and all the mother-BL's. Does that make
> >  sense?
> 
> Can we continue the whole "family" analogy and call the middle-BL the
> "grandma BL" (GBL) if we choose to have such a thing? This makes it seem
> clearer in my mind.

Agreed for grandma-BL!

> I agree that we will need to have some kind of central BL that the DL
> communicates with. If there will be multiple MBLs (which it seemed like
> the way the discussion was heading) then I agree that we need something
> central to which the DL always communicates. I hope I am understanding
> the discussion so far :-)

Well, it was always clear that there would be multiple BL's (since there are
multiple machine). Having a GBL will make things much simpler when it comes to
interactions between the DL and the PL. I would also like this grandma-BL to
perform the splitting, and that's the point on which Jarl and I disagree and
that would need to be discussed further.

	Jean-Marc

[Overflow] Re: IRC

[Brad C. <cha...@ar...>]

> I'm probably going to be late tomorrow (around 4:30). Also, I really
>  think this decision about how the splitting has to be done is
important enough
>  to be discussed with all the Piper developers. What do you think?

Agreed. I sent the log out along with an invitation for everyone to join
in. Sorry it came out so late, bioinformatics.org was down last night I
guess.

>  I'd also something about the BL part I'd like to see between the DL
>  and the
>  Mother-BL. I'd like to see the BL take care of all communication
>  between different machines. Since some of the PL nodes (viewers,
debugging
>  nodes, ...) will have to interact with the UI, the only way to achieve
that is
>  through a middle-BL (what I used to call "master") that takes care of
the
>  communication between the DL and all the mother-BL's. Does that make
>  sense?

Can we continue the whole "family" analogy and call the middle-BL the
"grandma BL" (GBL) if we choose to have such a thing? This makes it seem
clearer in my mind.

I agree that we will need to have some kind of central BL that the DL
communicates with. If there will be multiple MBLs (which it seemed like
the way the discussion was heading) then I agree that we need something
central to which the DL always communicates. I hope I am understanding
the discussion so far :-)

Brad

[Overflow] Re: IRC

[Jean-Marc V. <jea...@he...>]

I'm probably going to be late tomorrow (around 4:30). Also, I really think this
decision about how the splitting has to be done is important enough to be
discussed with all the Piper developers. What do you think?

I'd also something about the BL part I'd like to see between the DL and the
Mother-BL. I'd like to see the BL take care of all communication between
different machines. Since some of the PL nodes (viewers, debugging nodes, ...)
will have to interact with the UI, the only way to achieve that is through a
middle-BL (what I used to call "master") that takes care of the communication
between the DL and all the mother-BL's. Does that make sense?

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

[Overflow] Re: BL thoughts

[Jean-Marc V. <jea...@he...>]

> Hmm... not really :) I was thinking about the node that will be the glue between
> instances of BL's. Building the glue between BL and PL will be a quite similair task.
> I'm currently stuck with porting the C Corba code to C++. I asked Brad to have a look
> and I ordered a book about this subject.

I don't understand what kind of "communication node" you're talking about. Is an
IRC chat at the Brad suggested OK for you (4pm EST, 10pm for you?)?

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

[Overflow] Re: BL thoughts

[Jean-Marc V. <jea...@he...>]

> Jean-Marc, this looks really good! You and Jarl have definately convinced
> me that we shouldn't worry too much about the "other langage" plugins
> right now, and just focus on getting the PL/BL communication down. I
> really like this plan you've proposed. Can we IRC about it later today at
> 4ish our time (10ish Jarl's time)? If we can agree on doing things this
> way, I would really like to hash out what we need to do to start coding
> on it!

4pm is OK for me.

> >  I'm assuming the BL will be split in a "master" process, running where
> >  the UI is running, and a bunch of "slave" processes on remote (and
> local)
> >  machines.
> 
> Makes sense.
> 
> >  I think the master could easiliy be linked with the DL, so
> >  communications would be
> >  easy (function calls).
> 
> Right now Jarl and I have set up this link via CORBA.

From my conversation with Jarl:
Jarl:
> > Sounds like you understand the BL pretty good. Only one thing: the BL wont have
> > master or slave processes, every machine will have 1 BL process running.

Me:
> Well, what I meant by the BL "master" was the part that splits the networks and
> sends it to all the machines... but maybe it's part of the DL. One way or
> another, the code is at the same place and does the same thing, so it just a
> matter of what you call what.


> Okay, so we are going to have the master BL be able to split a network
> description up arbitratily in sub-networks (as you should in your
> A->B->C->D example) into the slave BL processes. Then these slave BLs can
> be responsible for using the Overflow getOutput() pull system to execute
> a sub-network. Am I understanding your ideas correctly?

Yes.


> >  Now let's say that in our precious example node C was a viewer. When C
> >  gets the result from B, it will tell (with a function call) its BL to
> display
> >  some data.
> >  Its BL slave will send that to its BL "master", which will send it
> >  (function call) to the DL, which will know what to do with it: either
> send it to
> >  the GUI or log it in a file for a batch processing (or even discard it
> when
> >  the user doesn't want to see it).
> >
> >  What do you think of that?
> 
> Very nice! That helps rid us of the whole callback-inside-a-node thing,
> since the BL can be responsible for splitting up a process so it
> automagically can retrieve the results it needs to return to the DL. This
> should reduce a level of callbacks and hopefully make things cleaner. My
> biggest question is how are we going to implement this so that the BL
> knows how to break things up? Jarl definately knows better then I how the
> BL can handle this and I guess this is something we can chat about
> tonight.

I'm not sure what you mean by "callback-inside-a-node thing". As for how the BL
will split the network, it can happen in many different ways. The user might
want to select from the GUI what runs where, then the BL can look if all the
nodes are available from all the machines... at some point in the future, the BL
may be "smart" enough to know how to split the processing without any action
from the user, but I don't see that happening in the short term.

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...

[Overflow] BL thoughts

[Jean-Marc V. <jea...@he...>]

Hi,

I've been thinking about how to integrate the BL with the PL and here are my
thoughts on that for now.

I'm assuming the BL will be split in a "master" process, running where the UI is
running, and a bunch of "slave" processes on remote (and local) machines. I
think the master could easiliy be linked with the DL, so communications would be
easy (function calls). 

As for the "slaves", I see them linking with Overflow. That way, the BL slave
would call getOutput() on the PL sub-network, get the result and send that to
another BL process. Since the BL would link with the overflow (PL) libraries,
communications would be made by functions/callbacks. For instance, the first
node of a distributed section would call a function in the BL and ask for its
incoming data. In the same way, for probes and viewers, we would have a node
that would call a BL function (either directly or through a method from a new
iostream). Here's with an example how I see the whole thing.

We have the following processing:
A->B->C->D   (each letter is a Node performing a certain operation)
Suppose the user (or the system) decides to break-up the processing in three
parts:
A -> (B->C) -> D

The BL will spawn 3 processes (we don't care where they run for now). Now, let's
consider the second process doing B->C, the BL will construct an Overflow
network:

N->B->C , where N is a special BL communication node.
When the second BL is asked to compute its result (remember that we're going
backwards, using a pull), it will call getOutput() on node C. In order to
perform its calculation, node C will call getOutput() on node B, which will
getOutput() on node N. Node N will then call a BL function that will ask for its
input. The BL function will ask (CORBA/IPC/socket/...) the first BL and return
the answer it got. Node N will then return the result to node B, which will
calculate something and return to node C, and at the end, the second BL will
have the answer. It will send that answer to the third BL in the same way that
the first BL sent it to the second. (is it clear or have I lost everybody?). If
we suppose that each of the "slave" BL was running on a different machine, we
have that each machine only needs one process (or thread) running. The only
inter-process communication will happen between different BL parts, so both the
PL and DL will not have to care about that. Does that make sense? 

Now let's say that in our precious example node C was a viewer. When C gets the
result from B, it will tell (with a function call) its BL to display some data.
Its BL slave will send that to its BL "master", which will send it (function
call) to the DL, which will know what to do with it: either send it to the GUI
or log it in a file for a batch processing (or even discard it when the user
doesn't want to see it).

What do you think of that?

	Jean-Marc

-- 
Jean-Marc Valin
Universite de Sherbrooke - Genie Electrique
va...@ge...