beepcore-ruby-discuss

[beepcore-ruby-discuss] My thoughts on implementation

[Nathan D. <cu...@us...>]

Hi Beepers,

Just thought I would dabble out some notes, just to get the ball rolling 
again.

*Message I/O*

First off I thought that a message should contain a MessageOutput (for 
sending messages) or a MessageInput (for receiving messages). For 
instance a MessageOutput would have the following interface:

    * #write(s)
          o Send the given bytes into the message buffer (flush buffer
            and send a message segment if buffer limit reached)
          o s - some bytes of a message to send
    * #close()
          o Close the message output and send remaining bytes

Equally MessageInput would have methods #read(n) and #close(). Of course 
this idea might just stem from the fact that I code Java for my day job 
and am used to having InputStream and OutputStream. I was looking at 
extending the Ruby IO class for these classes, but it seems impossible 
to implement a clean custom version since it depends on file descriptors.

*Message Encapsulations*

Another thought I had was to apply a message encapsulation to a message 
stream, thus wrapping the data sent in a chosen format. As we all know, 
BEEP enforces no message format inside the payload, but often uses a 
simple MIME format for control messages. The few encapsulations I had in 
mind were:

    * Verbatim (default, do not add any format to the payload)
    * MIME
    * DIME (you might be going 'hmmm' here, but someone might want to
      use this)

The added value would be that the programmer would not have to parse the 
message payload just to get the headers. Rather they could use 
OutgoingMessage#set_property(name,value) and 
IncomingMessage#get_property(name) for access to these. Also the message 
format could be switched without any change in code, while retaining the 
same support for message headers.

For it all to make sense, the default message encapsulation of a channel 
should be a property of the profile.

*Transport*

Maybe this goes without saying, but I thought to mention it, just in 
case. Of course, the TCP/IP mapping will be the choice of 
implementation, but I suggest that the BEEP transport code is abstracted 
such that other transport mechanisms can be used, i.e. a different 
transport mapping can be plugged into a session.

If nobody thinks that anything other than TCP will be useful...; well, 
fair enough, but I can easilly imagine so. One of my own ideas center 
around IPC communication via BEEP and the transport being shared memory.

*Threads*

What are peoples thoughts on threading and thread utilization? 
Personally, I feel that the Ruby BEEP Core should function in both 
modes, single threaded and multi. The session should not need any 
disparate configuration, although it would be necessary to have the 
profile implementation decide whether it wants to pull messages or run 
in asynchronous mode. Thus, the (a)synchronicity of communication would 
be a by-channel property.

Also, does anyone know of any material discussing Good(tm) profile 
implementation? Seems to me, that the spec does not explicitly state, 
that profiles may not block eachother's channels in the application (the 
BEEP implementation, of course, may not) but what are the issues 
regarding deadlock? Do we have any means of supporting this for the 
developer, making it easier to avoid stupidity? Do we choose not to care?

*Interoperability*

Which implementation do we plan to use for testing against? Is there an 
approved test profile that is commonly used to test interop between 
implementations? If possible, I think it would be a good idea to have 
contact with at least one other BEEP project to communicate on the issue 
of making implementations that function with eachother.

*APEX*

Well, for now I know that we are just going to try to actually get some 
BEEP code running but what are our plans for the future? How far does 
the Ruby BEEP Core go? /The Application Exchange Core/ is a cool place 
to go when you have BEEP and maybe an APEX framework would a logical 
next step?

Well, just my thoughts so far.
Thoughts, comments?

Curne.
--
/BEEP - Adding another layer to the OSI model/
http://sourceforge.net/projects/beepcore-ruby/

Re: [beepcore-ruby-discuss] My thoughts on implementation

[Dan M. <dn...@po...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

At 06:26 AM 5/27/2003 +0200, Nathan Droft wrote:

>Hi Beepers,
>
>Just thought I would dabble out some notes, just to get the ball rolling 
>again.

Thanks for your notes Nathan. Sorry it's taken so long for me to get back 
to you on this.

[snip]

>Transport
>
>Maybe this goes without saying, but I thought to mention it, just in case.
> Of course, the TCP/IP mapping will be the choice of implementation, but I
> suggest that the BEEP transport code is abstracted such that other 
>transport mechanisms can be used, i.e. a different transport mapping can 
>be plugged into a session.
>
>If nobody thinks that anything other than TCP will be useful...; well, 
>fair enough, but I can easilly imagine so. One of my own ideas center 
>around IPC communication via BEEP and the transport being shared memory.

This may be something to look into in a future release; that is, beyond the
first released reference implementation of a Ruby BEEP Core. I'm 
comfortable on top of TCP for now, and while I would in general prefer 
cleanly separated code, this is an area I don't think I'd be likely to 
spend a great deal of time abstracting out should it prove to be difficult 
to do so.

>Threads
>
>What are peoples thoughts on threading and thread utilization? Personally,
> I feel that the Ruby BEEP Core should function in both modes, single 
>threaded and multi. The session should not need any disparate 
>configuration, although it would be necessary to have the profile 
>implementation decide whether it wants to pull messages or run in 
>asynchronous mode. Thus, the (a)synchronicity of communication would be a 
>by-channel property.
>
>Also, does anyone know of any material discussing Good(tm) profile 
>implementation? Seems to me, that the spec does not explicitly state, that
> profiles may not block eachother's channels in the application (the BEEP 
>implementation, of course, may not) but what are the issues regarding 
>deadlock? Do we have any means of supporting this for the developer, 
>making it easier to avoid stupidity? Do we choose not to care?

I can see concurrency and Ruby threads being tar pits of complexity early 
on, so my inclination is to "do the simplest thing that could possibly 
work" in the short term to get the proof-of-concept code out, and then the 
reference implementation, and then work on making beepcore-ruby more robust
for real world use. I have a massive problem with getting code out early 
and often, so changing that behavior is my main goal.

>Interoperability
>
>Which implementation do we plan to use for testing against? Is there an 
>approved test profile that is commonly used to test interop between 
>implementations? If possible, I think it would be a good idea to have 
>contact with at least one other BEEP project to communicate on the issue 
>of making implementations that function with eachother.

Gabe Wachob, who started the PyBEEP project a while back, created 
BEEPBuilders <http://beepbuiders.sf.net/> as an interop and conformance 
testing group/project. I think we should get involved in that and push 
other implementers to do likewise. It sounds odd for this project which has
a pretty shoddy track record so far of keeping up on it's goals to spur 
another to the same, but it can't possibly hurt.

>APEX
>
>Well, for now I know that we are just going to try to actually get some 
>BEEP code running but what are our plans for the future? How far does the 
>Ruby BEEP Core go? The Application Exchange Core is a cool place to go 
>when you have BEEP and maybe an APEX framework would a logical next step?

APEX is not part of the BEEPcore, but is something I'm interested in. 
Eventually, I'd like beepcore-ruby to be able to support any use of BEEP, 
including APEX, as well as HSTP, reliable syslog, etc. We have lots of work
to do. ;]


- -- 
Dan Moniz <dn...@po...> [http://www.pobox.com/~dnm/]

-----BEGIN PGP SIGNATURE-----
Version: PGP 8.0
Comment: Please see <http://www.pobox.com/~dnm/keys.html>

iQA/AwUBPyeR9VmE1hyKYjtREQLGygCfYnyVKgNQ7RM+wDm6zZrSD3UqphUAoKYz
kKK8ZmC5uox+DCCIRDnmRB3q
=EHU/
-----END PGP SIGNATURE-----

Re: [beepcore-ruby-discuss] My thoughts on implementation

[Nathan D. <cu...@us...>]

Hi List,

Sorry it's taken a few days to reply but I am kind of tied down at work 
these days.

Anyway, comments below...

Dan Moniz wrote:

>  
>
>>Transport
>>
>>Maybe this goes without saying, but I thought to mention it, just in case.
>>Of course, the TCP/IP mapping will be the choice of implementation, but I
>>suggest that the BEEP transport code is abstracted such that other 
>>transport mechanisms can be used, i.e. a different transport mapping can 
>>be plugged into a session.
>>
>>If nobody thinks that anything other than TCP will be useful...; well, 
>>fair enough, but I can easilly imagine so. One of my own ideas center 
>>around IPC communication via BEEP and the transport being shared memory.
>>    
>>
>
>This may be something to look into in a future release; that is, beyond the
>first released reference implementation of a Ruby BEEP Core. I'm 
>comfortable on top of TCP for now, and while I would in general prefer 
>cleanly separated code, this is an area I don't think I'd be likely to 
>spend a great deal of time abstracting out should it prove to be difficult 
>to do so.
>

I expect the later abstraction of the transport mechanism is not a 
horrendous job if it becomes relevant at all. I certainly accept the 
vote, that TCP will be the main use for now. However, if (read: when) 
BEEP reaches industry standard level, I imagine it on top of SCTP.

>>Threads
>>
>>What are peoples thoughts on threading and thread utilization? Personally,
>>I feel that the Ruby BEEP Core should function in both modes, single 
>>threaded and multi. The session should not need any disparate 
>>configuration, although it would be necessary to have the profile 
>>implementation decide whether it wants to pull messages or run in 
>>asynchronous mode. Thus, the (a)synchronicity of communication would be a 
>>by-channel property.
>>
>>Also, does anyone know of any material discussing Good(tm) profile 
>>implementation? Seems to me, that the spec does not explicitly state, that
>>profiles may not block eachother's channels in the application (the BEEP 
>>implementation, of course, may not) but what are the issues regarding 
>>deadlock? Do we have any means of supporting this for the developer, 
>>making it easier to avoid stupidity? Do we choose not to care?
>>    
>>
>
>I can see concurrency and Ruby threads being tar pits of complexity early 
>on, so my inclination is to "do the simplest thing that could possibly 
>work" in the short term to get the proof-of-concept code out, and then the 
>reference implementation, and then work on making beepcore-ruby more robust
>for real world use. I have a massive problem with getting code out early 
>and often, so changing that behavior is my main goal.
>  
>
Well, in my experience, the simplest thing when designing an application 
with multiple connections (in this case, multiple channels aswell) is to 
just to start a thread for each and mutex of the critical regions. 
However, this unfailingly turns into a heap of threads sitting around 
doing nothing most of the time.

Making a solution that supports both a threaded and non-threaded mode 
allows the developer to employ synchronous and asynchronous channels, 
both of which are relevant use cases.

I do, however agree that the main goal of this project is to get it 
working as soon as possible. I view development-oriented discussion as 
something parallel to coding. I have seen too many projects go belly up 
from endless design discussions. As a general rule, something that works 
is worth more than something that looks good on paper.

>>Interoperability
>>
>>Which implementation do we plan to use for testing against? Is there an 
>>approved test profile that is commonly used to test interop between 
>>implementations? If possible, I think it would be a good idea to have 
>>contact with at least one other BEEP project to communicate on the issue 
>>of making implementations that function with eachother.
>>    
>>
>
>Gabe Wachob, who started the PyBEEP project a while back, created 
>BEEPBuilders <http://beepbuiders.sf.net/> as an interop and conformance 
>testing group/project. I think we should get involved in that and push 
>other implementers to do likewise. It sounds odd for this project which has
>a pretty shoddy track record so far of keeping up on it's goals to spur 
>another to the same, but it can't possibly hurt.
>  
>
This sounds fantastic. That was exactly the type of thing I had in mind.

Michael, I can see you are also a member of that group. Can you comment 
on the progress of the code and/or specs coming out of that project?

>>APEX
>>
>>Well, for now I know that we are just going to try to actually get some 
>>BEEP code running but what are our plans for the future? How far does the 
>>Ruby BEEP Core go? The Application Exchange Core is a cool place to go 
>>when you have BEEP and maybe an APEX framework would a logical next step?
>>    
>>
>
>APEX is not part of the BEEPcore, but is something I'm interested in. 
>Eventually, I'd like beepcore-ruby to be able to support any use of BEEP, 
>including APEX, as well as HSTP, reliable syslog, etc. We have lots of work
>to do. ;]
>  
>
Yes, isn't it great :-)

I agree with your general point, start small, grow big.
Let's get this cool project going!

Curne
--
/BEEP - Adding another layer to the OSI model/
http://sourceforge.net/projects/beepcore-ruby/