Re: [Quickfix-developers] QuickFIX Python and Twisted

[Reggie D. <re...@me...>]

Rob,

I attempted to do almost exactly the same thing with almost exactly the
same results.  I believe the problem starts with Python threading
behavior:

      * The Python Global Interpreter Lock (GIL) allows only one python
        thread to be running at any point in time. 
      * Python will execute 10 bytecode instructions from a particular
        thread and then release the GIL allowing other threads the
        opportunity to run and acquire the GIL.
      * A call to a C/C++ extension function is 1 bytecode

Due to this behavior I believe that Python is calling into the QF C++
code and then staying there without ever giving Python the opportunity
to release the GIL and allow your twisted thread to run.  I thought of
trying to edit the quickfix.i file to get swig to generate code to
release the GIL before calling QF, but I realize I don't have enough
SWIG/C++ knowledge to do it.  One of the problems I had is that the SWIG
generated code sometimes calls back into python within a method so you
can't just release the GIL arbitrarily before calling any QF function.

Unfortunately I have yet to get Twisted and QuickFix integrated in the
same application.  Sorry I couldn't be of more help, but if you find a
solution to this I'd appreciate it if you could pass it along.

-Reggie



On Wed, 2006-11-15 at 13:41 -0500, Robert Parrott wrote:
> QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
> 
> HI Folks,
> 
> I'm trying to  build a simply gateway based on QuickFIX with the
> python bindings, and the Twisted framework (http://twistedmatrix.com).
> I'm finding some kind of threading issue.
> 
> My aim is to wrap the quickfix engine with a simple RPC container, so
> that one can connect to the daemon and invoke calls like "start",
> "stop", "subscribe to data" etc. The twisted framework provides an
> asynchronous networking layer to do so.
> 
> However things seem to lock up when I invoke the
> SocketInitiator.start() method. I can start up the service, and create
> the QuickFIX application, and  setup and configure the various
> QuickFIX components, ala:
> 
>  class MyQFService:
>   def __init__
>         settings     = quickfix.SessionSettings(configFile)
>         application  = MyQFClient(connStr)
>         storeFactory = quickfix.OdbcStoreFactory(settings)
>         logFactory   = quickfix.OdbcLogFactory(settings)
> 
>         self.quickfixSettings = settings
>         self.quickfixApplication = application
>         self.quickfixStoreFactory = storeFactory
>         self.quickfixLogFactory = logFactory
> 
>         self.quickfixInitiator = quickfix.SocketInitiator(application,
> storeFactory, settings, logFactory)
> 
>  When a remote "START" command is issued, the main thread of the
> twisted code calls the SocketInitiator to start the QuickFIX thread:
> 
>   def startQF(self):
>         log.msg("QuickFIX Engine starting.")
>         self.quickfixInitiator.start()
>         log.msg("QuickFIX Engine started.")
>         return defer.succeed("STARTED")
> 
> At this point, the quickfix engine thread starts, and I see logon,
> heartbeats, and other admin messages. The log message immediately
> after the start() call executes, and returns correctly. However, the
> main thread of the Twisted application seems to block entirely, and I
> don't get any response from the application on subsequent RPC
> attempts. At this point the appliction also stops responding to Ctrl-C
> on the command line, when before it would gracefully terminate.
> 
> It seems as if the quickfix thread has taken over, or more likely is
> completely blocking the main twisted thread. Any thoughts on what
> might be happening here? My first guess is that it has to do with
> logging, but I can't be sure.
> 
> thanks for any insight,
> rob
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
-- 
Reggie Dugard <re...@me...>
Merfin, LLC