Re: [opendds-devel] Integration of Event Handling for CORBA AMI and OpenDDS

[M. H. <ha...@co...>]

Scott,

Thanks for your response.  For the time being, it sounds like I'll just
setup my main thread to service my CORBA AMI client using the normal ACE/TAO
reactor and the separate transport/listener thread(s) (only expect to have
one for a simple client) will handle DDS subscriptions.  Any shared objects
or context between the two threads will simply have to be locked,
time-multiplexed via protocol agreement, or utilize some other standard
concurrent access mechanism.

The desire for the single threaded model is mostly to help sell this
particular comprehensive combination of technology (CORBA AMI & DDS) to some
of the technical folks where I work, by claiming it is easy to use -- when
that feature is important (like when inexperienced programmers need to build
an application component using a boilerplate approach).  We have many
software components that are of easy to moderate complexity, so that's a
nice selling point.  On the other hand, it's admirable that you've tackled
the multi-threaded model already in order to support more complex and
scalable publishers & subscribers.  Multi-threaded models are obviously key
to performance gains as CPUs begin to implement more cores in the future and
cell processors become more prevalent.

I understand and appreciate OCI's business model.  It is not out of the
realm of possibility that you may be contacted to help move development of
at least this one desirable feature along.

Thanks again,
M. Hayman


-----Original Message-----
From: Scott Harris [mailto:har...@oc...] 
Sent: Monday, September 10, 2007 8:07 PM
To: Mark Hayman
Cc: ope...@li...
Subject: Re: [opendds-devel] Integration of Event Handling for CORBA AMI and
OpenDDS

Hi Mark,

I am Scott Harris from OCI.  I was on the original design & 
implementation team for the OpenDDS implementation of DDS.

See my comments below.

Thanks,
  Scott

Mark Hayman wrote:
>
> I originally posted this question to the comp.soft-sys.ace usenet 
> group a couple of days ago since it's related to foundational event 
> handling, and was referred to this mailing list.  Despite the somewhat 
> long rambling post, which is replicated below, I guess the fundamental 
> question has to do with OpenDDS.
>
>  
>
> M. Hayman
>
> ----------------------------------
>
>  
>
> Background: I've been doing a lot of research over the past few weeks 
> into the latest _performant_ middleware approaches to support a 
> variety of programs/projects where I work, ranging from HPC embedded 
> systems to LAN-based distributed computing systems (standard 
> requirements - multiple platforms/OS's, multiple languages, location 
> independence, etc.).  I'm not concerned with enterprise class problems 
> at this point, e.g. multi-site, WAN-based, Internet type applications 
> -- just those in the "single-site" domain for which underlying 
> broadcast & multicast transport protocols are an option.
>
>  
>
> I need to support building both services, per the SOA paradigm and the 
> tightly-coupled request/reply information exchange pattern, as well as 
> high performance (low latency, high throughput) streaming data/event 
> delivery capabilities utilizing the more loosely-coupled 
> publish/subscribe information exchange pattern -- inherent in EDA and 
> DOA architecture paradigms.  IMHO the combination of both information 
> exchange patterns meets the full range of needs for the types of 
> systems we build.
>
>  
>
> Perhaps it's due to my background in building both real-time systems 
> and GUIs, but I have always preferred an event-driven application 
> component development model based upon registering event 
> handlers/callbacks with an event dispatcher/loop (e.g. like the 
> ACE_Reactor encapsulation of select() & Windows equivalent).  Junior 
> software developers sometimes take a while to get used to this 
> asynchronous programming approach, e.g. having to save state between 
> events to decide how to handle the next unordered/unexpected event, 
> however I've found in the long run that it produces the most 
> performant and least complex/buggy code -- often avoiding the need to 
> escalate design complexity of a particular component to support 
> multi-threading and the inevitable plethora of debug problems 
> associated with lots of mutexes, shared objects, concurrency issues, 
> unintended tight-coupling/colocation dependencies, etc. that come 
> along with it, particularly for inexperienced programmers.  In 
> addition, I've come to detest long-running event handlers and any 
> system level call that has the potential to block for long time 
> periods (e.g. standard synchronous CORBA), since this inevitably 
> causes single-threaded applications to "hang" and become unresponsive 
> to other input events.  Same for using semaphores for IPC - which I 
> also hate.  Sure, you can spin off potentially long-blocking 
> synchronous calls into separate threads, but for many relatively 
> simple application components this is complication I'd just as soon 
> avoid (e.g. multithreaded apps) for the reasons I just stated.
>
I agree that keeping to a single thread reduces the complexity of the 
code in many cases.  Our original intent was to provide multiple 
threading models (single threaded, thread pools, and 
thread-per-connection).   Supporting multiple models could reduces the 
complexity of user code (using single threaded) and improve on 
performance in some scenarios (using threading pools for a large number 
of connections).

>  
>
> Having followed the DDS standard evolution over the past couple years, 
> it has led me to recently rediscovering the ACE/TAO stuff, in 
> conjunction with OpenDDS, after having dismissed it for a long time 
> (CORBA is "dead", right?) due to a forced foray into the wonderful 
> (non-performant) world of J2EE/.Net and web services.
>
> ----------------------
>
>  
>
> So, my current leaning based upon all of this and a lot more I won't 
> go into is toward utilization of the CORBA AMI ("new" since the last 
> time I looked at CORBA :) callback model for 
> SOA/services/request-reply, in combination with the DDS standard for 
> DOA/EDA/publish-subscribe.  Decoupling of the RPC reply event from a 
> service request invocation is desirable, as is the inherent 
> event-driven nature of DDS.  I like the fact that CORBA AMI allows you 
> to avoid spinning off a separate thread to handle a synchronous call 
> so that your main thread can remain event-responsive.  However, based 
> upon my initial cursory evaluation, it appears that the current 
> OpenDDS implementation for a subscriber _forces_ you into building a 
> multi-threaded application.  I'm off to look at RTI DDS and OpenSPLICE 
> DDS as well, but this is my current understanding for OpenDDS - which 
> I'd prefer to use for development/experimentation.
>
>  
>
> Every example of OpenDDS I have seen has a main() ending with a spin 
> loop containing a sleep() call.  I have always considered this "less 
> than optimal" design -- more like polling than event driven, usually 
> resulting in unnecessary context switches and built-in minimum event 
> response latency defined by the arbitrary sleep interval.
>
 I agree that this pattern of sleeps in the application code is ugly and 
could delay the timeliness of the shutdown of an application but these 
loops have a very minor impact on performance of the data communication 
because coming out of the loop is not required to perform work and the 
only overhead is a context switch at the period of the switch.

 We may want to introduce a method in the OpenDDS implementation that 
says the main is done (and the call would block until the DCPS 
implementation was told to shutdown).  This would reduce ugliness and 
the context switches. 

Of course, when we implement a single threaded model for OpenDSS the 
main would not end in a sleep loop but with the running of the Reactor.  
Some of the current examples and tests may also need to move some code 
in the main into callbacks.
>
>  Is there no way to register the event associated with a DDS data read 
> with an ACE_Reactor-based (e.g. orb->run()) event handler in a single 
> thread?
>
With the current implementation, there is no way to have a single 
threaded OpenDDS application.  There is at least one thread for the 
main, one for the ORB (to communicate with the InfoRepo) and one thread 
per transport (to handle payload communication (distribution of samples)).

>  
>
> If so (not at all obvious), an even more desirable capability would be 
> to allow OpenDDS subscriber/datareader event handlers (registered 
> callbacks) to be managed in a _single_ event loop that also handles 
> CORBA AMI events (e.g. orb->run() or a lower level ACE call).  Is this 
> possible? 
>
As I stated earlier, our original intent was to provide multiple 
threading models (single threaded, thread pools, and 
thread-per-connection).  But we ran out of time before the 
single-threaded and threading pool models could be implemented.

Yes, it would be possible to change the implementation of OpenDDS to 
support a threading model that uses one reactor and one thread.

OCI would be interested in modifying the OpenDDS implementation to 
support a single threaded model, with support from your organization.
Since OpenDDS is open source and free of licensing fees, we do not 
derive any revenue other than through our services and support.  So, 
unfortunately, we are unable to provide free support.  Please contact 
sa...@oc... for information on opening a support contract.

> The only way I can see doing this right now (without a timer-based 
> polling approach - yuk) is to have the separate OpenDDS subscriber 
> threads raise a signal or write to a pipe when a new data sample 
> arrives, and then register an event handler for the pipe FD with the 
> TAO event handler in the main thread.  But, this is starting to get 
> really ugly, since it's exactly the kind of stuff you want 
> framework/middleware code to handle for you.
>
I agree - that sounds ugly.

>  
>
> What am I missing or not understanding?  I really expected that since 
> TAO and OpenDDS were put out by more-or-less the same group that it 
> would be easy to integrate them in the way I'm describing, e.g. a 
> single calling thread.  For many apps that we build, it would be very 
> nice for a [simple, single-threaded] client to be able to function 
> both as a CORBA AMI client as well as subscribe to a DDS topic or two 
> at the same time.
>
>  
>
>
> No virus found in this outgoing message.
> Checked by AVG Free Edition.
> Version: 7.5.485 / Virus Database: 269.13.12/997 - Release Date: 
> 9/9/2007 10:17 AM
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> ------------------------------------------------------------------------
>
> _______________________________________________
> opendds-devel mailing list
> ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opendds-devel
>