Re: [Quickfix-developers] QuickFIX Python and Twisted

[Robert P. <rob...@gm...>]

That's very helpful information, Reggie.

Are there any suggestions from the quickfix developers on how one
might address this threading issue?

On 11/15/06, Reggie Dugard <re...@me...> wrote:
> Rob,
>
> I attempted to do almost exactly the same thing with almost exactly the
> same results.  I believe the problem starts with Python threading
> behavior:
>
>       * The Python Global Interpreter Lock (GIL) allows only one python
>         thread to be running at any point in time.
>       * Python will execute 10 bytecode instructions from a particular
>         thread and then release the GIL allowing other threads the
>         opportunity to run and acquire the GIL.
>       * A call to a C/C++ extension function is 1 bytecode
>
> Due to this behavior I believe that Python is calling into the QF C++
> code and then staying there without ever giving Python the opportunity
> to release the GIL and allow your twisted thread to run.  I thought of
> trying to edit the quickfix.i file to get swig to generate code to
> release the GIL before calling QF, but I realize I don't have enough
> SWIG/C++ knowledge to do it.  One of the problems I had is that the SWIG
> generated code sometimes calls back into python within a method so you
> can't just release the GIL arbitrarily before calling any QF function.
>
> Unfortunately I have yet to get Twisted and QuickFix integrated in the
> same application.  Sorry I couldn't be of more help, but if you find a
> solution to this I'd appreciate it if you could pass it along.
>
> -Reggie
>
>
>
> On Wed, 2006-11-15 at 13:41 -0500, Robert Parrott wrote:
> > QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html
> > QuickFIX Support: http://www.quickfixengine.org/services.html
> >
> > HI Folks,
> >
> > I'm trying to  build a simply gateway based on QuickFIX with the
> > python bindings, and the Twisted framework (http://twistedmatrix.com).
> > I'm finding some kind of threading issue.
> >
> > My aim is to wrap the quickfix engine with a simple RPC container, so
> > that one can connect to the daemon and invoke calls like "start",
> > "stop", "subscribe to data" etc. The twisted framework provides an
> > asynchronous networking layer to do so.
> >
> > However things seem to lock up when I invoke the
> > SocketInitiator.start() method. I can start up the service, and create
> > the QuickFIX application, and  setup and configure the various
> > QuickFIX components, ala:
> >
> >  class MyQFService:
> >   def __init__
> >         settings     = quickfix.SessionSettings(configFile)
> >         application  = MyQFClient(connStr)
> >         storeFactory = quickfix.OdbcStoreFactory(settings)
> >         logFactory   = quickfix.OdbcLogFactory(settings)
> >
> >         self.quickfixSettings = settings
> >         self.quickfixApplication = application
> >         self.quickfixStoreFactory = storeFactory
> >         self.quickfixLogFactory = logFactory
> >
> >         self.quickfixInitiator = quickfix.SocketInitiator(application,
> > storeFactory, settings, logFactory)
> >
> >  When a remote "START" command is issued, the main thread of the
> > twisted code calls the SocketInitiator to start the QuickFIX thread:
> >
> >   def startQF(self):
> >         log.msg("QuickFIX Engine starting.")
> >         self.quickfixInitiator.start()
> >         log.msg("QuickFIX Engine started.")
> >         return defer.succeed("STARTED")
> >
> > At this point, the quickfix engine thread starts, and I see logon,
> > heartbeats, and other admin messages. The log message immediately
> > after the start() call executes, and returns correctly. However, the
> > main thread of the Twisted application seems to block entirely, and I
> > don't get any response from the application on subsequent RPC
> > attempts. At this point the appliction also stops responding to Ctrl-C
> > on the command line, when before it would gracefully terminate.
> >
> > It seems as if the quickfix thread has taken over, or more likely is
> > completely blocking the main twisted thread. Any thoughts on what
> > might be happening here? My first guess is that it has to do with
> > logging, but I can't be sure.
> >
> > thanks for any insight,
> > rob
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share your
> > opinions on IT & business topics through brief surveys - and earn cash
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > _______________________________________________
> > Quickfix-developers mailing list
> > Qui...@li...
> > https://lists.sourceforge.net/lists/listinfo/quickfix-developers
> --
> Reggie Dugard <re...@me...>
> Merfin, LLC
>
>
>