Re: [Flightgear-devel] Designing a thread-safe property tree API (was Re: A FGPythonSys implementation: ...)

[Torsten D. <to...@t3...>]

Edward,

I'd like to understand why you would want a python subsystem to be tied to
the rendering frame.
For aircraft specific functions and for some special cases, it might be
necessary to run in sync with the FDM iteration. But almost every system
should be able to run completely independently of the frame rate. What of
course would be mandatory is to sync properties at well defined time
stamps, this is what the RTI takes care of.

My major concern with your Python integration is that it introduces an
unpredictable (w.r.t. timing) step in the main loop getting us on step
further away from constant frame rates.
Ahrgh - sorry, that should have gone into the other thread :-/

Torsten


Edward d'Auvergne <tru...@gm...> schrieb am Di., 26. Jan. 2016 um
11:10 Uhr:

> On 26 January 2016 at 10:09, Stuart Buchanan <stu...@gm...> wrote:
> > Hi All,
> >
> > I'm not going to comment on the general point of integrating Python, or
> on the
> > general architecture discussion, as I haven't read or thought enough
> > to determine
> > whether I have a view and/or can contribute usefully to the discussion!
> >
> > However, there is something I can comment on immediately:
> >
> > On Mon, Jan 25, 2016 at 10:00 AM, Edward d'Auvergne wrote:
> >> I'm quite interested in hearing what Stuart has been thinking about
> >> this subject.  The asynchronous HLA designs are directly relevant to
> >> these issues - racing in the property system is inevitable.  For
> >> enabling threaded Python, threaded Python sub-instances, and designing
> >> the spawning of new FlightGear subsystems via the Python/C++ interface
> >> I'm planning, possibly via a HLA communication mechanism, it would be
> >> good to have a solid plan for property tree system API for enabling
> >> thread-safe operation.  For the idea of mirroring parts of the
> >> property tree, I have a feeling that this will result in very costly
> >> synchronisation issues.
> >
> > I'm 99% sure that HLA isn't the answer here.
> >
> > HLA is really intended to operate at a much higher level of granularity
> and
> > pass objects around a distributed simulation.  The use-case I'm working
> > on at the moment passes an AI instance as an object (with position,
> > velocity, accelerations, model to be displayed).  How a HLA Federate maps
> > that data internally is an implementation decision.  There shouldn't be
> > any race conditions because no assumptions are made about how
> > the data is used or mapped.
> >
> > At present, I'm not thinking that HLA will be used as a mechanism to
> provide
> > a shared property tree across multiple Federates.  I think that's the
> wrong way
> > to use HLA.  Instead you make explicit decisions about the data models
> you
> > will communicate.
> >
> > The other reason HLA isn't a good choice is to do with simulation
> clocking.
> > I'm still getting my head around this properly, but the overall model
> > is one where
> > the simulation clock is advanced, and the instance the updates and
> publishes
> > changes to the RTI that will be picked up by other instances at the next
> clock
> > advance. (I think - I've still to work on a use-case where there is
> significant
> > inter-dependence between instances).
> >
> > I think James is right in considering this largely orthogonal.
> >
> > One point worth making is that there is excellent support for writing
> python HLA
> > Federates.  In fact, I anticipate that once I get the initial HLA work
> > committed, this
> > will be an area where we see a lot of activity simply because people
> will find
> > it easier to write a Federate in python than the alternatives.
> > Obviously this requires
> > developing the data model and hooking that data model up within the
> > FlightGear core,
> > but I think that will be a good driver for this.
> >
> > To give a couple of concrete examples:
> > - It should be straightforward to pull information from flight or ship
> > trackers and generate
> > AI traffic.  I've done this before in a somewhat hacky way, but HLA
> > provides a nice clean
> > way to do so, and python a convenient way to get at the data.
> > - A shared Weather federate publishing both the environment for
> > individual aircraft within
> > the simulation, but also cloud positions so all aircraft had a
> > consistent weather environment.
> >
> > So, even if Python isn't integrated with the FG core, there is plenty
> > of opportunitie to use
> > it within a FG-based simulation.
>
> Hi James and Stuart,
>
> Thanks for all the feedback!  I did expect that RTI would struggle
> with a per-frame update() function call.  Though there is a lot of
> pressure in the forum to use HLA for absolutely everything.  For my
> idea of an aircraft spawning its own subsystem via the Python
> subsystem, I'll have to give this more thought.  This was my idea for
> a viable Python alternative to the Nasal timer concept, i.e. a
> complete, clean-slate redesign of the updating concept, which does not
> appear to be an ideal or even accurate way to update aircraft.  I was
> hoping to use real threads for running FGPythonSys outside of the main
> thread, and child process forking and HLA/RTI communication on the C++
> side for the aircraft subsystem.  This would have given FGPythonSys
> and the aircraft their own embedded Python instance with GIL, so that
> their operation would be fully independent.  The design would have
> really stress-tested FlightGear against racing conditions.
>
> Would there be any interest in optional threaded subsystems?  If HLA
> is not viable for this, then is the threading option on the table?
> Can HLA eliminate all the CPU bottlenecks, or should there be a target
> of using threads to shift most things off the main thread, combined
> with HLA to do the computational heavy lifting?
>
> If threads are on the table, then I'd like to talk about SGThread.  I
> will probably be jumping on a few toes here, but threads would
> probably be the best reason to up the C++ standard to C++11.  Just
> have a look at simgear/threads/SGThread.cxx.  It is clear that this
> was designed before there was real C++ threading on MS Windows.  We
> have our own custom threading system, and it is different on different
> operating systems!  However C++ threading has been standardised with
> C++11:
>
>     http://en.cppreference.com/w/cpp/thread/mutex
>     http://en.cppreference.com/w/cpp/thread/thread
>     http://en.cppreference.com/w/cpp/atomic/atomic
>
> The std::atomic type might in general be good for stability (though
> maybe not a navdb cache solution).  The use of std::thread,
> std::mutex, etc. would allow a lot of the ugly custom SGThread code
> for Windows to be dropped, as well as the old POSIX pthreads code
> under that.  Using the C++11 standard, the whole simgear/threads
> directory could be deleted as it simply duplicates the functionality
> of std::thread, std::mutex and their cousins in a non-standard way.
> This might increase the reliability of FlightGear threading, as the
> standard library threading should be bug free, highly reliable, and
> consistent between different operating systems.  Anyway, I have a lot
> of experience with both threads, locking, and higher level
> multi-processor work, (e.g.
> http://www.nmr-relax.com/api/4.0/gui.analyses.execute.Execute-class.html
> and http://www.nmr-relax.com/api/4.0/multi-module.html ), so something
> useful might come out of the FGPythonSys experiments.
>
> Regards,
>
> Edward
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
> _______________________________________________
> Flightgear-devel mailing list
> Fli...@li...
> https://lists.sourceforge.net/lists/listinfo/flightgear-devel
>