Thread: [Quickfix-developers] Re: ThreadedSocketInitiator vs SocketInitiator
Brought to you by:
orenmnero
Menu
▾
▴
quickfix-developers
|
From: Alexey Z. <ale...@gm...> - 2006-05-10 15:24:22
|
Wow. I tried to get this information a year ago when I start using QuickFix. And now such a big surprise. Thanks, Caleb. Overall QF is doing pretty good but I do have dead locks in one heavy loaded multi-thread application. In my application I have one (not threaded) initiator and an acceptor. When I lost connection in the initiator, the application is stuck (not always, basically I can't reproduce it freely, but it happens often in production). I stopped using ThreadedSocketInitiator because I have only one session in it. Basically I thought the problem is that sessions list is static, and my acceptor and initiator share critical sections incorrectly. I put threaded initiator to the production yesterday and will see if it help. Thanks for the explanations again. Regards, Alexey Zubko > > ------------------------------------------------------------------------ > > Subject: > Re: [Quickfix-developers] ThreadedSocketInitiator vs SocketInitiator > From: > "Caleb Epstein" <cal...@gm...> > Date: > Tue, 9 May 2006 12:08:22 -0400 > To: > "Nick Volpe" <ni...@ad...> > > To: > "Nick Volpe" <ni...@ad...> > CC: > quickfix-developers <qui...@li...> > > > I don't think this made it through the first time. > > On 5/9/06, Nick Volpe <ni...@ad...> wrote: >> >> Can anyone tell me pros/cons of using a ThreadedSocketInitiator versus a >> Socket Initiator? I c an't seem to find much in the documentation or >> the >> mailing list archives. Any help would be greatly appreciated. > > The non-threaded SocketInitiator can actually get into a deadlock > situation whereby your application will block forever. I would > recommend against anyone ever using it. > > This deadlock will happen if your application fills up a socket's > sending buffer while the other side is doing the same (e.g. two > non-threaded QuickFIX apps generating heavy traffic). Neither side > will end up reading from its side of the socket, being blocked in a > send() call, and both sides will deadlock. > > -- > Caleb Epstein > caleb dot epstein at gmail dot com > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers > > |
|
From: Steve B. <sb...@sm...> - 2006-05-11 05:58:48
|
=20 I vote for removing it or making it a fa=E7ade. In QFJ, the primary difference between the two in a single session scenario are the behavior of the block and poll methods. QFJ uses asynchronous IO (NIO) with low level read/write queues so I don't think it will have this particular problem. However, with queues there is another issue with managing their size under heavy load with a fast producer and slow consumer. Steve > -----Original Message----- > From: qui...@li...=20 > [mailto:qui...@li...] On=20 > Behalf Of Oren Miller > Sent: Wednesday, May 10, 2006 6:01 PM > To: Caleb Epstein; Nick Volpe > Cc: quickfix-developers > Subject: Re: [Quickfix-developers] ThreadedSocketInitiator vs=20 > SocketInitiator >=20 > QuickFIX Documentation:=20 > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html >=20 > Yeah, I'm reading that in unix it is typical to do a fork and=20 > have each=20 > process treat it as unidirectional to avoid deadlocks. So we=20 > would have to=20 > have a thread for sending and a thread for receiving. This=20 > is actually=20 > something we had talked about in the past to prevent=20 > deadlocks when sending=20 > from callbacks. Essentially creating a sending queue. I am=20 > wondering=20 > however if we really do need the SocketInitiator/Acceptor or=20 > if it should be=20 > deprecated in favor of the threaded versions. I think the=20 > precense of both=20 > of them has been one of the more confusing aspects of=20 > QuickFIX and not sure=20 > it really is advantageous to have both at this point. Any=20 > thoughts on this? >=20 > --oren >=20 > ----- Original Message -----=20 > From: "Caleb Epstein" <cal...@gm...> > To: "Nick Volpe" <ni...@ad...> > Cc: "quickfix-developers" <qui...@li...> > Sent: Tuesday, May 09, 2006 11:08 AM > Subject: Re: [Quickfix-developers] ThreadedSocketInitiator vs=20 > SocketInitiator >=20 >=20 > QuickFIX Documentation:=20 > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html >=20 > I don't think this made it through the first time. >=20 > On 5/9/06, Nick Volpe <ni...@ad...> wrote: > > > > Can anyone tell me pros/cons of using a=20 > ThreadedSocketInitiator versus a > > Socket Initiator? I c an't seem to find much in the=20 > documentation or the > > mailing list archives. Any help would be greatly appreciated. >=20 > The non-threaded SocketInitiator can actually get into a deadlock > situation whereby your application will block forever. I would > recommend against anyone ever using it. >=20 > This deadlock will happen if your application fills up a socket's > sending buffer while the other side is doing the same (e.g. two > non-threaded QuickFIX apps generating heavy traffic). Neither side > will end up reading from its side of the socket, being blocked in a > send() call, and both sides will deadlock. >=20 > -- > Caleb Epstein > caleb dot epstein at gmail dot com >=20 >=20 > ------------------------------------------------------- > Using Tomcat but need to do more? Need to support web=20 > services, security? > Get stuff done quickly with pre-integrated technology to make=20 > your job=20 > easier > Download IBM WebSphere Application Server v.1.0.1 based on=20 > Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=3Dk&kid=120709&bid&3057&dat=121642 > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers >=20 >=20 >=20 > ------------------------------------------------------- > Using Tomcat but need to do more? Need to support web=20 > services, security? > Get stuff done quickly with pre-integrated technology to make=20 > your job easier > Download IBM WebSphere Application Server v.1.0.1 based on=20 > Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057& > dat=3D121642 > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers >=20 |
|
From: Sean K. <sea...@pi...> - 2006-05-12 20:59:50
|
May I suggest that there also be a thread pool option? If an acceptor = has hundreds of sessions configured, it won't be a good idea to spawn a = thread per session... --Sean -----Original Message----- From: qui...@li... [mailto:qui...@li...]On Behalf Of Oren Miller Sent: Wednesday, May 10, 2006 12:56 PM To: Brian Erst; Caleb Epstein; Nick Volpe Cc: quickfix-developers Subject: Re: [Quickfix-developers] ThreadedSocketInitiator vs SocketInitiator QuickFIX Documentation: = http://www.quickfixengine.org/quickfix/doc/html/index.html QuickFIX Support: http://www.quickfixengine.org/services.html There was, however it was overrun by WikiSpam and I never got around to=20 researching solutions to the problem. --oren > p.s. Speaking of confusion, is there a QuickFIX wiki? While the = existing=20 > documentation is sufficient for getting started, in order to really=20 > understand a lot of QF you have to read the mailing list archives. If = we=20 > had a wiki, I'd be tempted to transfer at least some of the mailing = list=20 > knowledge to the wiki, if only from an ongoing basis. ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, = security? Get stuff done quickly with pre-integrated technology to make your job = easier Download IBM WebSphere Application Server v.1.0.1 based on Apache = Geronimo http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D= 121642 _______________________________________________ Quickfix-developers mailing list Qui...@li... https://lists.sourceforge.net/lists/listinfo/quickfix-developers |
|
From: Alexey Z. <ale...@gm...> - 2006-05-17 17:11:09
|
Hello, There is a problem in VC projects: The file src/c++/Messages.h was removed in the latest versions, but still exist in the projects. Regards, Alexey Zubko |
|
From: Steve B. <sb...@sm...> - 2006-05-29 08:30:25
|
Hi Sean, One complication of a thread pool approach is that we must be sure that the messages for each session are processed in sequence. We can't just dispatch each received message to a thread in the=20 pool. One solution is to maintain a queue for each session and=20 then dispatch the queues to a thread pool. The worker threads=20 would process one or more messages (could be configurable)=20 each time they take a session queue for processing. Do you have any other ideas how to implement an thread pool initiator/acceptor? Do any of the threading experts out there know of an applicable pattern for this scenario? Steve > -----Original Message----- > From: qui...@li...=20 > [mailto:qui...@li...] On=20 > Behalf Of Sean Kirkpatrick > Sent: Friday, May 12, 2006 11:00 PM > To: Oren Miller; Brian Erst; Caleb Epstein; Nick Volpe > Cc: quickfix-developers > Subject: RE: [Quickfix-developers] ThreadedSocketInitiator vs=20 > SocketInitiator >=20 > QuickFIX Documentation:=20 > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html >=20 > May I suggest that there also be a thread pool option? If an=20 > acceptor has hundreds of sessions configured, it won't be a=20 > good idea to spawn a thread per session... >=20 > --Sean >=20 > -----Original Message----- > From: qui...@li... > [mailto:qui...@li...]On=20 > Behalf Of Oren Miller > Sent: Wednesday, May 10, 2006 12:56 PM > To: Brian Erst; Caleb Epstein; Nick Volpe > Cc: quickfix-developers > Subject: Re: [Quickfix-developers] ThreadedSocketInitiator vs=20 > SocketInitiator >=20 >=20 > QuickFIX Documentation:=20 > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html >=20 > There was, however it was overrun by WikiSpam and I never got=20 > around to researching solutions to the problem. >=20 > --oren >=20 > > p.s. Speaking of confusion, is there a QuickFIX wiki? While the=20 > > existing documentation is sufficient for getting started,=20 > in order to=20 > > really understand a lot of QF you have to read the mailing list=20 > > archives. If we had a wiki, I'd be tempted to transfer at=20 > least some=20 > > of the mailing list knowledge to the wiki, if only from an=20 > ongoing basis. >=20 >=20 >=20 > ------------------------------------------------------- > Using Tomcat but need to do more? Need to support web=20 > services, security? > Get stuff done quickly with pre-integrated technology to make=20 > your job easier Download IBM WebSphere Application Server=20 > v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057& > dat=3D121642 > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers >=20 >=20 > ------------------------------------------------------- > Using Tomcat but need to do more? Need to support web=20 > services, security? > Get stuff done quickly with pre-integrated technology to make=20 > your job easier Download IBM WebSphere Application Server=20 > v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=3Dk&kid=120709&bid&3057&dat=121642 > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers >=20 |
|
From: Sean K. <sea...@pi...> - 2006-05-30 12:57:44
|
Hi Steve, I understand your concerns. Guaranteeing message sequencing for a = session becomes more complicated with a thread pool and message queue. = Each session would need some kind of locking mechanism (a mutex would do = the trick), that could be locked when a worker thread is processing the = session's message queue. My thoughts were that it would be useful to = have a set of incoming worker threads and outgoing worker threads. Each = session could then have both an incoming and outgoing queue for messages = that would be processed the same way. That way, any outgoing messages = (even those sent from callback functions) would be written to the = outgoing queue and picked up by a worker thread when all prior outgoing = messages have been sent. This would hopefully eliminate the possibility = of deadlocks. It's just very important not to starve any of the = sessions while processing a queue for a very busy connection... Anyone have additional thoughts / comments / concerns? --Sean -----Original Message----- From: qui...@li... [mailto:qui...@li...]On Behalf Of Steve Bate Sent: Monday, May 29, 2006 4:32 AM Cc: quickfix-developers Subject: RE: [Quickfix-developers] ThreadedSocketInitiator vs SocketInitiator QuickFIX Documentation: = http://www.quickfixengine.org/quickfix/doc/html/index.html QuickFIX Support: http://www.quickfixengine.org/services.html Hi Sean, One complication of a thread pool approach is that we must be sure that the messages for each session are processed in sequence. We can't just dispatch each received message to a thread in the=20 pool. One solution is to maintain a queue for each session and=20 then dispatch the queues to a thread pool. The worker threads=20 would process one or more messages (could be configurable)=20 each time they take a session queue for processing. Do you have any other ideas how to implement an thread pool initiator/acceptor? Do any of the threading experts out there know of an applicable pattern for this scenario? Steve > -----Original Message----- > From: qui...@li...=20 > [mailto:qui...@li...] On=20 > Behalf Of Sean Kirkpatrick > Sent: Friday, May 12, 2006 11:00 PM > To: Oren Miller; Brian Erst; Caleb Epstein; Nick Volpe > Cc: quickfix-developers > Subject: RE: [Quickfix-developers] ThreadedSocketInitiator vs=20 > SocketInitiator >=20 > QuickFIX Documentation:=20 > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html >=20 > May I suggest that there also be a thread pool option? If an=20 > acceptor has hundreds of sessions configured, it won't be a=20 > good idea to spawn a thread per session... >=20 > --Sean >=20 > -----Original Message----- > From: qui...@li... > [mailto:qui...@li...]On=20 > Behalf Of Oren Miller > Sent: Wednesday, May 10, 2006 12:56 PM > To: Brian Erst; Caleb Epstein; Nick Volpe > Cc: quickfix-developers > Subject: Re: [Quickfix-developers] ThreadedSocketInitiator vs=20 > SocketInitiator >=20 >=20 > QuickFIX Documentation:=20 > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html >=20 > There was, however it was overrun by WikiSpam and I never got=20 > around to researching solutions to the problem. >=20 > --oren >=20 > > p.s. Speaking of confusion, is there a QuickFIX wiki? While the=20 > > existing documentation is sufficient for getting started,=20 > in order to=20 > > really understand a lot of QF you have to read the mailing list=20 > > archives. If we had a wiki, I'd be tempted to transfer at=20 > least some=20 > > of the mailing list knowledge to the wiki, if only from an=20 > ongoing basis. >=20 >=20 >=20 > ------------------------------------------------------- > Using Tomcat but need to do more? Need to support web=20 > services, security? > Get stuff done quickly with pre-integrated technology to make=20 > your job easier Download IBM WebSphere Application Server=20 > v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057& > dat=3D121642 > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers >=20 >=20 > ------------------------------------------------------- > Using Tomcat but need to do more? Need to support web=20 > services, security? > Get stuff done quickly with pre-integrated technology to make=20 > your job easier Download IBM WebSphere Application Server=20 > v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=3Dk&kid=120709&bid&3057&dat=121642 > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers >=20 ------------------------------------------------------- All the advantages of Linux Managed Hosting--Without the Cost and Risk! Fully trained technicians. The highest number of Red Hat certifications = in the hosting industry. Fanatical Support. Click to learn more http://sel.as-us.falkag.net/sel?cmd=3Dk&kid=107521&bid$8729&dat=121642 _______________________________________________ Quickfix-developers mailing list Qui...@li... https://lists.sourceforge.net/lists/listinfo/quickfix-developers |
|
From: Ajay K. <Aja...@tr...> - 2006-05-30 14:41:26
|
One useful property of QuickFIX's current processing of outbound messages is that the application can rely upon the knowledge that the message has been written written to the message store to the socket and by the time send() returns. This makes it relatively simple to make sure all messages that the application thinks it has sent successfully will get to the counterparty. If an intermediate queue is introduced between the application and the processing of the message within the FIX engine, then depending upon whether the queue is introduced before QF writes to the message store or after, there might be a need to invent additional mechanisms to allow the application to be informed when a message gets persisted so that it can track the message's progress through the engine. Otherwise messages that the application thought it had sent might still be waiting to be processed by a worker thread in an in-memory queue and might be lost in case of an engine failover. - Ajay -----Original Message----- From: Sean Kirkpatrick [mailto:sea...@pi...]=20 Sent: Tuesday, May 30, 2006 8:58 AM To: Steve Bate Cc: quickfix-developers Subject: RE: [Quickfix-developers] ThreadedSocketInitiator vs SocketInitiator QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html QuickFIX Support: http://www.quickfixengine.org/services.html Hi Steve, I understand your concerns. Guaranteeing message sequencing for a session becomes more complicated with a thread pool and message queue. Each session would need some kind of locking mechanism (a mutex would do the trick), that could be locked when a worker thread is processing the session's message queue. My thoughts were that it would be useful to have a set of incoming worker threads and outgoing worker threads. Each session could then have both an incoming and outgoing queue for messages that would be processed the same way. That way, any outgoing messages (even those sent from callback functions) would be written to the outgoing queue and picked up by a worker thread when all prior outgoing messages have been sent. This would hopefully eliminate the possibility of deadlocks. It's just very important not to starve any of the sessions while processing a queue for a very busy connection... Anyone have additional thoughts / comments / concerns? --Sean -------------------------------------------------------------------------= -- The information in this email is confidential and may be legally = privileged. It is intended solely for the addressee. Access to this email by anyone = else is unauthorized. If you are not the intended recipient, any disclosure, = copying, distribution or any action taken or omitted to be taken in reliance on = it, is prohibited and may be unlawful. TradeWeb reserves the right to monitor and review the content of all = messages sent to or from this e-mail address. Messages sent to or from this e-mail = address may be stored on the TradeWeb e-mail system. |
|
From: Caleb E. <cal...@gm...> - 2006-05-30 14:57:49
|
On 5/30/06, Ajay Kamdar <Aja...@tr...> wrote: > Otherwise messages > that the application thought it had sent might still be waiting to be > processed by a worker thread in an in-memory queue and might be lost in > case of an engine failover. Clearly this would not be an acceptable implementation. -- Caleb Epstein caleb dot epstein at gmail dot com |
|
From: Joerg T. <Joe...@ma...> - 2006-05-30 15:06:24
|
Hi Ajay,
> One useful property of QuickFIX's current processing of outbound
> messages is that the application can rely upon the knowledge that the
> message has been written written to the message store to the socket and
> by the time send() returns. This makes it relatively simple to make sur=
e
> all messages that the application thinks it has sent successfully will
> get to the counterparty.
Agreed, actually this is a very important property. As soon as QF sendToT=
arget() has=20
returned, QF has taken responsibility for the message.
> If an intermediate queue is introduced between the application and the
> processing of the message within the FIX engine, then depending upon
> whether the queue is introduced before QF writes to the message store o=
r
> after, there might be a need to invent additional mechanisms to allow
> the application to be informed when a message gets persisted so that it
> can track the message's progress through the engine. Otherwise messages
> that the application thought it had sent might still be waiting to be
> processed by a worker thread in an in-memory queue and might be lost in
> case of an engine failover.
The correct implementation would make the outbound queue and the message =
store identical.
sendToTarget() just writes the message to the (persistent) message store =
and then returns.=20
A separate thread picks up messages from the message store to be (re-)sen=
d.
Cheers, J=F6rg
--=20
Joerg Thoennes
http://macd.com
Tel.: +49 (0)241 44597-24 Macdonald Associates GmbH
Fax : +49 (0)241 44597-10 Lothringer Str. 52, D-52070 Aachen
|
