Menu
▾
▴
quickfixj-users — General mailing list for QuickFIX/J topics.
You can subscribe to this list here.
| 2006 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(13) |
Jun
(21) |
Jul
(14) |
Aug
(29) |
Sep
(39) |
Oct
(47) |
Nov
(70) |
Dec
(27) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2007 |
Jan
(43) |
Feb
(50) |
Mar
(90) |
Apr
(96) |
May
(84) |
Jun
(40) |
Jul
(58) |
Aug
(55) |
Sep
(55) |
Oct
(52) |
Nov
(38) |
Dec
(75) |
| 2008 |
Jan
(49) |
Feb
(72) |
Mar
(49) |
Apr
(55) |
May
(21) |
Jun
(31) |
Jul
(47) |
Aug
(59) |
Sep
(59) |
Oct
(77) |
Nov
(51) |
Dec
(54) |
| 2009 |
Jan
(52) |
Feb
(57) |
Mar
(17) |
Apr
(27) |
May
(44) |
Jun
(46) |
Jul
(69) |
Aug
(38) |
Sep
(39) |
Oct
(45) |
Nov
(38) |
Dec
(37) |
| 2010 |
Jan
(49) |
Feb
(35) |
Mar
(21) |
Apr
(33) |
May
(52) |
Jun
(28) |
Jul
(39) |
Aug
(34) |
Sep
(21) |
Oct
(82) |
Nov
(36) |
Dec
(20) |
| 2011 |
Jan
(28) |
Feb
(64) |
Mar
(93) |
Apr
(75) |
May
(151) |
Jun
(77) |
Jul
(35) |
Aug
(53) |
Sep
(56) |
Oct
(36) |
Nov
(94) |
Dec
(59) |
| 2012 |
Jan
(105) |
Feb
(43) |
Mar
(68) |
Apr
(91) |
May
(45) |
Jun
(18) |
Jul
(103) |
Aug
(77) |
Sep
(45) |
Oct
(59) |
Nov
(58) |
Dec
(43) |
| 2013 |
Jan
(48) |
Feb
(65) |
Mar
(63) |
Apr
(22) |
May
(41) |
Jun
(60) |
Jul
(43) |
Aug
(17) |
Sep
(20) |
Oct
(20) |
Nov
(42) |
Dec
(43) |
| 2014 |
Jan
(54) |
Feb
(34) |
Mar
(34) |
Apr
(20) |
May
(31) |
Jun
(39) |
Jul
(66) |
Aug
(22) |
Sep
(52) |
Oct
(22) |
Nov
(67) |
Dec
(70) |
| 2015 |
Jan
(18) |
Feb
(5) |
Mar
(40) |
Apr
(32) |
May
(62) |
Jun
(28) |
Jul
(86) |
Aug
(44) |
Sep
(61) |
Oct
(65) |
Nov
(8) |
Dec
(19) |
| 2016 |
Jan
(50) |
Feb
(22) |
Mar
(38) |
Apr
(55) |
May
(30) |
Jun
(42) |
Jul
(11) |
Aug
(9) |
Sep
(4) |
Oct
(51) |
Nov
(38) |
Dec
(31) |
| 2017 |
Jan
(40) |
Feb
(40) |
Mar
(23) |
Apr
(35) |
May
(121) |
Jun
(55) |
Jul
(37) |
Aug
(16) |
Sep
(27) |
Oct
(109) |
Nov
(67) |
Dec
(23) |
| 2018 |
Jan
(52) |
Feb
(6) |
Mar
(23) |
Apr
(28) |
May
(32) |
Jun
(20) |
Jul
(20) |
Aug
(22) |
Sep
(8) |
Oct
(33) |
Nov
(32) |
Dec
(13) |
| 2019 |
Jan
(16) |
Feb
(29) |
Mar
(17) |
Apr
(16) |
May
(1) |
Jun
(2) |
Jul
(25) |
Aug
(50) |
Sep
(17) |
Oct
(29) |
Nov
(16) |
Dec
(7) |
| 2020 |
Jan
|
Feb
|
Mar
(29) |
Apr
(64) |
May
(25) |
Jun
(49) |
Jul
(15) |
Aug
(10) |
Sep
(37) |
Oct
(20) |
Nov
(19) |
Dec
(9) |
| 2021 |
Jan
(33) |
Feb
(10) |
Mar
(67) |
Apr
(40) |
May
(70) |
Jun
(33) |
Jul
(14) |
Aug
(10) |
Sep
|
Oct
(7) |
Nov
(6) |
Dec
(16) |
| 2022 |
Jan
(27) |
Feb
(2) |
Mar
(5) |
Apr
(3) |
May
|
Jun
(2) |
Jul
|
Aug
(1) |
Sep
(6) |
Oct
(2) |
Nov
|
Dec
(10) |
| 2023 |
Jan
(1) |
Feb
(2) |
Mar
(21) |
Apr
(3) |
May
(15) |
Jun
(3) |
Jul
(4) |
Aug
|
Sep
|
Oct
(3) |
Nov
|
Dec
(1) |
| 2024 |
Jan
(7) |
Feb
(2) |
Mar
(8) |
Apr
(11) |
May
(6) |
Jun
(5) |
Jul
|
Aug
|
Sep
|
Oct
(3) |
Nov
|
Dec
|
| 2025 |
Jan
(10) |
Feb
(4) |
Mar
(9) |
Apr
|
May
(3) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Andrew M. <mar...@gm...> - 2019-01-13 15:18:02
|
Hello everyone, I am using quickfixj to talk to a remote service that uses SSL encryption. I can get to the web landing page using curl but I get an SSLException with quickfixj during the handshake. I think this must be an issue with the quickfixj code. Let me explain some more: The remote service cannot be accessed directly because of a corporate firewall. The corporation is a heavy user of Microsoft software/environments so the corporate proxy uses NTLM for authentication. This works fine for web users from their windows workstations. However, it causes problems for some other software. An example is the python pip command. That fails with a proxy authentication error even when the correct credentials are supplied. This was reported as a problem to the python/pip developers. They refused to fix the issue. They said it was because the corporate proxy had been misconfigured and they were not going to change their code to cope with corporate misconfiguration. So the problem remains. To get around this problem I use a windows service called CNTLM. It is basically a proxy for a proxy. I enter my credentials into that and it sorts out the traffic forwarding to the pypi website. So when I hit a similar problem with quickfixj I eventually hit on the idea of using CNTLM. I did and the quickfixj communication now works. This is how I can get things to work on windows but unfortunately the software has to be run in a RHEL environment where CNTLM is not available. I do not know what I am going to be able to do about that. I've heard that other software gets hit by the same issue. There was a problem in chrome some years ago apparently. There is a recently logged issue with talend, as can be seen at https://community.talend.com/t5/Design-and-Development/tRest-with-NTLM-Proxy-Authorization/td-p/95066. There was also (and still might be) a problem with postman, as you can see here: https://github.com/postmanlabs/postman-app-support/issues/3692. Other examples include ArcEarth, https://community.esri.com/thread/189903-web-proxy-ntlm-authentication-error, and gradle, see https://stackoverflow.com/questions/14434101/gradle-not-working-behind-proxy-with-ntlm-on-windows. But there is one bit of software that I am able to connect with ok and that's curl. I am using a very recent version, 2.57. I cannot help thinking that curl has put in a workaround of the kind requested by python pip users. I hope to do some investigation to see if earlier versions of curl have the issue. That might shed some light on where the fix (if any) was supplied. It might be in the curl code or it might be in openssl. I am really not sure. But I am fairly convinced that apps that experience the issue must employ some sort of workaround. I note that a bug report was raised against quickfixj that might be the same issue, see https://www.quickfixj.org/jira/browse/QFJ-949 . It was closed with the statement ". I doubt that this is a bug since many people are using SSL with Java 8.". I am using JDK8 and I see the issue unless I use CNTLM. So perhaps this bug report should be re-opened. I admit it might be a bug in mina, since that is the component that quickfixj uses for the SSL handling. Could a version be built with the most recent version of mina please? -- Regards, Andrew Marlow http://www.andrewpetermarlow.co.uk |
|
From: Christoph J. <chr...@ma...> - 2019-01-11 12:36:47
|
Hi, QuickFIX/J 2.1.1 has been released. This release includes bug fixes only. Many thanks to all the people who have contributed to this release (see release notes). Release notes can be found here: https://github.com/quickfix-j/quickfixj/wiki/QFJ-2.1.1-release-notes Release can be downloaded from here: https://github.com/quickfix-j/quickfixj/releases/download/QFJ_RELEASE_2_1_1/org.quickfixj-2.1.1-bin.zip Cheers, Chris. -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Christoph J. <chr...@ma...> - 2019-01-08 13:46:23
|
Hi Maya, we are also using log4j. But what do you mean with "additional log"? There should be no additional log file but just more information in your normal application log file. Here are some examples of messages that you can expect to be logged when debug logging is turned on: https://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/ReadDebug.html Maybe try to start your application directly from the console and only with minimal properties, i.e. no java agents. Apart from that I am out of ideas... Chris. On 08/01/2019 14:39, Maya Rolnik wrote: > > Hi Chris, > > There is no additional log when I add this flag. > > I am running on windows from idea intellij editor. > > So the console log should be shown in the editor. > > I am also using log4j – maybe that is the issue? > > Thanks, > > Maya > > *From:*Christoph John [mailto:chr...@ma...] > *Sent:* Tuesday, January 08, 2019 3:03 PM > *To:* Maya Rolnik; qui...@li... > *Subject:* Re: [Quickfixj-users] ssl and quickfixj 1.6.4 > > Hi Maya, > > in MINA 2.0+ the SSL filter is included in the mina-core JAR. So nothing should be missing from > the classpath. > Can you send the log file that you have? Is it possible that the logging is lost somewhere? I > don't know much about Windows but maybe the logging goes to the console instead of a log file? Are > you starting your program in a shell or from the desktop? > > Chris. > > On 08/01/2019 12:36, Maya Rolnik wrote: > > Hi Chris, > > Still no logs.. > > "C:\Program Files\Java\jdk1.8.0_121\bin\java" > -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:58789,suspend=y,server=n > -Djavax.net.debug=ssl -Dfile.encoding=UTF-8 -classpath <jars> <MainClass> > > Maybe the issue is related to a specific jar I need in my classpath? > > For quickfix 1.5.3 I used mina 1.1.7, and had an extra jar of mina-filter-ssl. > > For quickfix 1.6.4 I am using mina 2.0.13, and there is no matching ssl jar. > > Thanks, > > Maya > > *From:*Christoph John [mailto:chr...@ma...] > *Sent:* Tuesday, January 08, 2019 1:00 PM > *To:* Maya Rolnik > *Subject:* Re: [Quickfixj-users] ssl and quickfixj 1.6.4 > > Hi Maya, > > if the java property did not produce any more logs then you either did not apply it correctly > or the problem is somewhere else. Normally there is a LOT of logging so I assume option 1. ;) > Could you please try "-Djavax.net.debug=ssl,handshake,trustmanager"? > > Chris. > > On 07/01/2019 17:43, Maya Rolnik wrote: > > Hi Chris, > > I am using JDK 8. > > Those are my relevant settings: > > SocketUseSSL=Y > > SocketKeyStore=*** > > SocketKeyStorePassword=*** > > I have now added the setting you mentioned: > > EnabledProtocols=TLSv1.2 > > But still same result. > > Also -Djavax.net.debug=all did not produce any more logs. > > Thanks, > > Maya > > *From:*Christoph John [mailto:chr...@ma...] > *Sent:* Monday, January 07, 2019 6:07 PM > *To:* qui...@li... > <mailto:qui...@li...>; Maya Rolnik > *Subject:* Re: [Quickfixj-users] ssl and quickfixj 1.6.4 > > IIRC you at least need to add EnabledProtocols=TLSv1.2 with JDK8 to work. > > Chris. > > > > On 07/01/2019 17:03, Christoph John wrote: > > Hi, > > what does your config look like? Did you also upgrade your Java version? Starting with > JDK8 some ciphers are disabled by default. > Try turning on SSL debug logging by starting your process with Java options > "-Djavax.net.debug=all" or "-Djavax.net.debug=handshake". > > Cheers, > Chris. > > > > > On 07/01/2019 16:33, Maya Rolnik wrote: > > QuickFIX/J Documentation:http://www.quickfixj.org/documentation/ > > QuickFIX/J Support:http://www.quickfixj.org/support/ > > > > > > > > > > -- > > Christoph John > > Software Engineering > > T +49 241 557080-28 > > chr...@ma... <mailto:chr...@ma...> > > > > MACD GmbH > > Oppenhoffallee 103 > > 52066 Aachen, Germany > > www.macd.com <http://www.macd.com> > > > > Amtsgericht Aachen: HRB 8151 > > Ust.-Id: DE 813021663 > > Geschäftsführer: George Macdonald > > > > > -- > > Christoph John > > Software Engineering > > T +49 241 557080-28 > > chr...@ma... <mailto:chr...@ma...> > > > > MACD GmbH > > Oppenhoffallee 103 > > 52066 Aachen, Germany > > www.macd.com <http://www.macd.com> > > > > Amtsgericht Aachen: HRB 8151 > > Ust.-Id: DE 813021663 > > Geschäftsführer: George Macdonald > > > > -- > Christoph John > Software Engineering > T +49 241 557080-28 > chr...@ma... <mailto:chr...@ma...> > MACD GmbH > Oppenhoffallee 103 > 52066 Aachen, Germany > www.macd.com <http://www.macd.com> > Amtsgericht Aachen: HRB 8151 > Ust.-Id: DE 813021663 > Geschäftsführer: George Macdonald -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Maya R. <Ma...@ba...> - 2019-01-08 13:39:34
|
Hi Chris, There is no additional log when I add this flag. I am running on windows from idea intellij editor. So the console log should be shown in the editor. I am also using log4j - maybe that is the issue? Thanks, Maya From: Christoph John [mailto:chr...@ma...] Sent: Tuesday, January 08, 2019 3:03 PM To: Maya Rolnik; qui...@li... Subject: Re: [Quickfixj-users] ssl and quickfixj 1.6.4 Hi Maya, in MINA 2.0+ the SSL filter is included in the mina-core JAR. So nothing should be missing from the classpath. Can you send the log file that you have? Is it possible that the logging is lost somewhere? I don't know much about Windows but maybe the logging goes to the console instead of a log file? Are you starting your program in a shell or from the desktop? Chris. On 08/01/2019 12:36, Maya Rolnik wrote: Hi Chris, Still no logs.. "C:\Program Files\Java\jdk1.8.0_121\bin\java" -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:58789,suspend=y,server=n -Djavax.net.debug=ssl -Dfile.encoding=UTF-8 -classpath <jars> <MainClass> Maybe the issue is related to a specific jar I need in my classpath? For quickfix 1.5.3 I used mina 1.1.7, and had an extra jar of mina-filter-ssl. For quickfix 1.6.4 I am using mina 2.0.13, and there is no matching ssl jar. Thanks, Maya From: Christoph John [mailto:chr...@ma...] Sent: Tuesday, January 08, 2019 1:00 PM To: Maya Rolnik Subject: Re: [Quickfixj-users] ssl and quickfixj 1.6.4 Hi Maya, if the java property did not produce any more logs then you either did not apply it correctly or the problem is somewhere else. Normally there is a LOT of logging so I assume option 1. ;) Could you please try "-Djavax.net.debug=ssl,handshake,trustmanager"? Chris. On 07/01/2019 17:43, Maya Rolnik wrote: Hi Chris, I am using JDK 8. Those are my relevant settings: SocketUseSSL=Y SocketKeyStore=*** SocketKeyStorePassword=*** I have now added the setting you mentioned: EnabledProtocols=TLSv1.2 But still same result. Also -Djavax.net.debug=all did not produce any more logs. Thanks, Maya From: Christoph John [mailto:chr...@ma...] Sent: Monday, January 07, 2019 6:07 PM To: qui...@li...<mailto:qui...@li...>; Maya Rolnik Subject: Re: [Quickfixj-users] ssl and quickfixj 1.6.4 IIRC you at least need to add EnabledProtocols=TLSv1.2 with JDK8 to work. Chris. On 07/01/2019 17:03, Christoph John wrote: Hi, what does your config look like? Did you also upgrade your Java version? Starting with JDK8 some ciphers are disabled by default. Try turning on SSL debug logging by starting your process with Java options "-Djavax.net.debug=all" or "-Djavax.net.debug=handshake". Cheers, Chris. On 07/01/2019 16:33, Maya Rolnik wrote: QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ QuickFIX/J Support: http://www.quickfixj.org/support/ -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma...<mailto:chr...@ma...> MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com<http://www.macd.com> Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma...<mailto:chr...@ma...> MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com<http://www.macd.com> Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma...<mailto:chr...@ma...> MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com<http://www.macd.com> Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Christoph J. <chr...@ma...> - 2019-01-08 13:03:10
|
Hi Maya, in MINA 2.0+ the SSL filter is included in the mina-core JAR. So nothing should be missing from the classpath. Can you send the log file that you have? Is it possible that the logging is lost somewhere? I don't know much about Windows but maybe the logging goes to the console instead of a log file? Are you starting your program in a shell or from the desktop? Chris. On 08/01/2019 12:36, Maya Rolnik wrote: > > Hi Chris, > > Still no logs.. > > "C:\Program Files\Java\jdk1.8.0_121\bin\java" > -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:58789,suspend=y,server=n > -Djavax.net.debug=ssl -Dfile.encoding=UTF-8 -classpath <jars> <MainClass> > > Maybe the issue is related to a specific jar I need in my classpath? > > For quickfix 1.5.3 I used mina 1.1.7, and had an extra jar of mina-filter-ssl. > > For quickfix 1.6.4 I am using mina 2.0.13, and there is no matching ssl jar. > > Thanks, > > Maya > > *From:*Christoph John [mailto:chr...@ma...] > *Sent:* Tuesday, January 08, 2019 1:00 PM > *To:* Maya Rolnik > *Subject:* Re: [Quickfixj-users] ssl and quickfixj 1.6.4 > > Hi Maya, > > if the java property did not produce any more logs then you either did not apply it correctly or > the problem is somewhere else. Normally there is a LOT of logging so I assume option 1. ;) > Could you please try "-Djavax.net.debug=ssl,handshake,trustmanager"? > > Chris. > > On 07/01/2019 17:43, Maya Rolnik wrote: > > Hi Chris, > > I am using JDK 8. > > Those are my relevant settings: > > SocketUseSSL=Y > > SocketKeyStore=*** > > SocketKeyStorePassword=*** > > I have now added the setting you mentioned: > > EnabledProtocols=TLSv1.2 > > But still same result. > > Also -Djavax.net.debug=all did not produce any more logs. > > Thanks, > > Maya > > *From:*Christoph John [mailto:chr...@ma...] > *Sent:* Monday, January 07, 2019 6:07 PM > *To:* qui...@li... <mailto:qui...@li...>; > Maya Rolnik > *Subject:* Re: [Quickfixj-users] ssl and quickfixj 1.6.4 > > IIRC you at least need to add EnabledProtocols=TLSv1.2 with JDK8 to work. > > Chris. > > > On 07/01/2019 17:03, Christoph John wrote: > > Hi, > > what does your config look like? Did you also upgrade your Java version? Starting with > JDK8 some ciphers are disabled by default. > Try turning on SSL debug logging by starting your process with Java options > "-Djavax.net.debug=all" or "-Djavax.net.debug=handshake". > > Cheers, > Chris. > > > > On 07/01/2019 16:33, Maya Rolnik wrote: > > QuickFIX/J Documentation:http://www.quickfixj.org/documentation/ > > QuickFIX/J Support:http://www.quickfixj.org/support/ > > > > > > > > > -- > > Christoph John > > Software Engineering > > T +49 241 557080-28 > > chr...@ma... <mailto:chr...@ma...> > > > > MACD GmbH > > Oppenhoffallee 103 > > 52066 Aachen, Germany > > www.macd.com <http://www.macd.com> > > > > Amtsgericht Aachen: HRB 8151 > > Ust.-Id: DE 813021663 > > Geschäftsführer: George Macdonald > > > > -- > Christoph John > Software Engineering > T +49 241 557080-28 > chr...@ma... <mailto:chr...@ma...> > MACD GmbH > Oppenhoffallee 103 > 52066 Aachen, Germany > www.macd.com <http://www.macd.com> > Amtsgericht Aachen: HRB 8151 > Ust.-Id: DE 813021663 > Geschäftsführer: George Macdonald -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Maya R. <Ma...@ba...> - 2019-01-07 16:44:12
|
Hi Chris, I am using JDK 8. Those are my relevant settings: SocketUseSSL=Y SocketKeyStore=*** SocketKeyStorePassword=*** I have now added the setting you mentioned: EnabledProtocols=TLSv1.2 But still same result. Also -Djavax.net.debug=all did not produce any more logs. Thanks, Maya From: Christoph John [mailto:chr...@ma...] Sent: Monday, January 07, 2019 6:07 PM To: qui...@li...; Maya Rolnik Subject: Re: [Quickfixj-users] ssl and quickfixj 1.6.4 IIRC you at least need to add EnabledProtocols=TLSv1.2 with JDK8 to work. Chris. On 07/01/2019 17:03, Christoph John wrote: Hi, what does your config look like? Did you also upgrade your Java version? Starting with JDK8 some ciphers are disabled by default. Try turning on SSL debug logging by starting your process with Java options "-Djavax.net.debug=all" or "-Djavax.net.debug=handshake". Cheers, Chris. On 07/01/2019 16:33, Maya Rolnik wrote: QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ QuickFIX/J Support: http://www.quickfixj.org/support/ -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma...<mailto:chr...@ma...> MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com<http://www.macd.com> Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Christoph J. <chr...@ma...> - 2019-01-07 16:07:28
|
IIRC you at least need to add EnabledProtocols=TLSv1.2 with JDK8 to work. Chris. On 07/01/2019 17:03, Christoph John wrote: > Hi, > > what does your config look like? Did you also upgrade your Java version? Starting with JDK8 some > ciphers are disabled by default. > Try turning on SSL debug logging by starting your process with Java options > "-Djavax.net.debug=all" or "-Djavax.net.debug=handshake". > > Cheers, > Chris. > > > > On 07/01/2019 16:33, Maya Rolnik wrote: >> QuickFIX/J Documentation:http://www.quickfixj.org/documentation/ >> QuickFIX/J Support:http://www.quickfixj.org/support/ >> >> >> -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Christoph J. <chr...@ma...> - 2019-01-07 16:03:26
|
Hi, what does your config look like? Did you also upgrade your Java version? Starting with JDK8 some ciphers are disabled by default. Try turning on SSL debug logging by starting your process with Java options "-Djavax.net.debug=all" or "-Djavax.net.debug=handshake". Cheers, Chris. On 07/01/2019 16:33, Maya Rolnik wrote: > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > > Hi, > > I am using quickfixj for fix initiators. > With version 1.5.3 I am able to connect to ssl based acceptors, using mina jar. > With quickfixj 1.6 and newer, the same settings with the same acceptor do not work. > The connection can’t be made and no further details are received. > > Can anyone help? > > Thanks, > Maya > > > > _______________________________________________ > Quickfixj-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfixj-users -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Maya R. <Ma...@ba...> - 2019-01-07 15:46:27
|
Hi, I am using quickfixj for fix initiators. With version 1.5.3 I am able to connect to ssl based acceptors, using mina jar. With quickfixj 1.6 and newer, the same settings with the same acceptor do not work. The connection can't be made and no further details are received. Can anyone help? Thanks, Maya |
|
From: Imran K. <xxi...@gm...> - 2019-01-04 19:18:07
|
Very sorry, never mind - It was app logic fault - sendToTarget was actually not being called Thanks for responding and sorry for wasting your time On Fri, Jan 4, 2019, 18:43 Colin DuPlantis <co...@ma... wrote: > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support: > http://www.quickfixj.org/support/ > > > Does sendToTarget return true or false? > On 1/4/19 8:22 AM, Imran Kim wrote: > > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > Hi > > After invoking sendToTarget, I *do not* get a callback in toApp() > > What could be wrong? > > This code actually already works against a certain broker, but against a > new broker that we are setting up, nothing happens > > I do get a SequenceReset message from the broker, not sure whether this is > related - If it is, how should I proceed to this request? > > It's FIX 4.2. > > Thanks > Imi > > > _______________________________________________ > Quickfixj-users mailing lis...@li...://lists.sourceforge.net/lists/listinfo/quickfixj-users > > -- > Colin DuPlantis > Chief Architect, Marketcetera > Download, Run, Trade > 888.868.4884https://www.marketcetera.com > > _______________________________________________ > Quickfixj-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfixj-users > |
|
From: Colin D. <co...@ma...> - 2019-01-04 17:42:37
|
Does sendToTarget return true or false? On 1/4/19 8:22 AM, Imran Kim wrote: > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > > Hi > > After invoking sendToTarget, I *do not* get a callback in toApp() > > What could be wrong? > > This code actually already works against a certain broker, but against > a new broker that we are setting up, nothing happens > > I do get a SequenceReset message from the broker, not sure whether > this is related - If it is, how should I proceed to this request? > > It's FIX 4.2. > > Thanks > Imi > > > _______________________________________________ > Quickfixj-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfixj-users -- Colin DuPlantis Chief Architect, Marketcetera Download, Run, Trade 888.868.4884 https://www.marketcetera.com |
|
From: Imran K. <xxi...@gm...> - 2019-01-04 16:23:05
|
Hi After invoking sendToTarget, I *do not* get a callback in toApp() What could be wrong? This code actually already works against a certain broker, but against a new broker that we are setting up, nothing happens I do get a SequenceReset message from the broker, not sure whether this is related - If it is, how should I proceed to this request? It's FIX 4.2. Thanks Imi |
|
From: Christoph J. <chr...@ma...> - 2018-12-17 10:29:39
|
Hi, I don't know why it should not work. It does of course NOT work on the *same* message type. When you try to access the fields on the message you could create the needed group and access it. E.g.: Message message1 = new Message(); Message message2 = new Message(); Group group1 = new Group(295, 55); group1.setString(55, "SYM"); Group group2 = new Group(295, 299); group2.setString(299, "ID"); message1.addGroup(group1); message2.addGroup(group2); message1.getGroup(1, group1); message2.getGroup(1, group2); Cheers, Chris. On 17/12/2018 08:57, Kodippili Arachchige, Asanka wrote: > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > > The server that our tool is communicating with has defined 3 different repeating group messages > with the same fix tag but different delimiters. > > Ex: Sever has 3 different messages (message types “3345", “"3350_1", "3350_2"). All 3 have the > same fix tag “295”, but message type 3345 has the delimiter tag 55 ( FIX Symbol) while 3350_1 and > 3350_2 messages have the delimiter tag “299” ("Fix Quote Entry ID" ) > > <fld name="Fix Number of Quote Entries" fixtag="295" datatype="smsg_array" presence="optional" > ref_msgtype="3345" /> > > <fld name="Fix Number of Quote Entries" fixtag="295" datatype="smsg_array" presence="mandatory" > ref_msgtype="3350_1" /> > > <fld name="Fix Number of Quote Entries" fixtag="295" datatype="smsg_array" presence="optional" > ref_msgtype="3350_2" /> > > <msg name="exchange::Fix5 Quote Entry" type="3345"> > > <fld name="Fix Symbol" fixtag="55" datatype="string" presence="optional" maxlen="25" /> > > <fld name="Fix NoUnderlyings" fixtag="711" datatype="smsg_array" presence="optional" > ref_msgtype="3341" /> > > </msg> > > <msg name="exchange::Fix5 Mass Quote Ack Quote Entry" type="3350_1" rpt_grp_only="true"> > > <fld name="Fix Quote Entry ID" fixtag="299" datatype="string" presence="mandatory" maxlen="20" /> > > <fld name="Fix Symbol" fixtag="55" datatype="string" presence="mandatory" maxlen="25" /> > > <fld name="Fix Bid Price" fixtag="132" datatype="double" presence="optional" decimals="8" /> > > <fld name="Fix Bid Size" fixtag="134" datatype="double" presence="optional" decimals="8" /> > > <fld name="Fix Offer Price" fixtag="133" datatype="double" presence="optional" decimals="8" /> > > <fld name="Fix Offer Size" fixtag="135" datatype="double" presence="optional" decimals="8" /> > > </msg> > > <msg name="exchange::Fix5 Mass Quote Ack Quote Entry" type="3350_2" rpt_grp_only="true"> > > <fld name="Fix Quote Entry ID" fixtag="299" datatype="string" presence="optional" maxlen="20" /> > > <fld name="Fix Symbol" fixtag="55" datatype="string" presence="optional" maxlen="25" /> > > <fld name="Fix Quote Entry Status" fixtag="1167" datatype="int32" presence="optional" /> > > <fld name="Fix Quote Entry Reject Reason" fixtag="368" datatype="int32" presence="optional" /> > > </msg> > > We are using the below method to create the groups via quickfix and have set a limitation in our > end that if multiple repeating groups have the same fix tag, they should all share the same > delimiter. > > *public *Group(*int *field, *int *delim) > > Does quickfix/J support setting multiple delimiters for the same group tag ? > > Thanks, > > Kodippili Arachchige Asanka > Specialist Software Engineer, MillenniumIT > *LSEG Technology* > > Telephone +94 11 241 6000 Ext _26717_ > Mobile +94 77 9627002 > as...@ls... <mailto:as...@ls...> > > MillenniumIT, No 01, Millennium Drive, Malabe > > *www.lseg.com* <http://www.lseg.com/> > > -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Kodippili A. A. <as...@ls...> - 2018-12-17 08:11:10
|
The server that our tool is communicating with has defined 3 different repeating group messages with the same fix tag but different delimiters.
Ex: Sever has 3 different messages (message types "3345", ""3350_1", "3350_2"). All 3 have the same fix tag "295", but message type 3345 has the delimiter tag 55 ( FIX Symbol) while 3350_1 and 3350_2 messages have the delimiter tag "299" ("Fix Quote Entry ID" )
<fld name="Fix Number of Quote Entries" fixtag="295" datatype="smsg_array" presence="optional" ref_msgtype="3345" />
<fld name="Fix Number of Quote Entries" fixtag="295" datatype="smsg_array" presence="mandatory" ref_msgtype="3350_1" />
<fld name="Fix Number of Quote Entries" fixtag="295" datatype="smsg_array" presence="optional" ref_msgtype="3350_2" />
<msg name="exchange::Fix5 Quote Entry" type="3345">
<fld name="Fix Symbol" fixtag="55" datatype="string" presence="optional" maxlen="25" />
<fld name="Fix NoUnderlyings" fixtag="711" datatype="smsg_array" presence="optional" ref_msgtype="3341" />
</msg>
<msg name="exchange::Fix5 Mass Quote Ack Quote Entry" type="3350_1" rpt_grp_only="true">
<fld name="Fix Quote Entry ID" fixtag="299" datatype="string" presence="mandatory" maxlen="20" />
<fld name="Fix Symbol" fixtag="55" datatype="string" presence="mandatory" maxlen="25" />
<fld name="Fix Bid Price" fixtag="132" datatype="double" presence="optional" decimals="8" />
<fld name="Fix Bid Size" fixtag="134" datatype="double" presence="optional" decimals="8" />
<fld name="Fix Offer Price" fixtag="133" datatype="double" presence="optional" decimals="8" />
<fld name="Fix Offer Size" fixtag="135" datatype="double" presence="optional" decimals="8" />
</msg>
<msg name="exchange::Fix5 Mass Quote Ack Quote Entry" type="3350_2" rpt_grp_only="true">
<fld name="Fix Quote Entry ID" fixtag="299" datatype="string" presence="optional" maxlen="20" />
<fld name="Fix Symbol" fixtag="55" datatype="string" presence="optional" maxlen="25" />
<fld name="Fix Quote Entry Status" fixtag="1167" datatype="int32" presence="optional" />
<fld name="Fix Quote Entry Reject Reason" fixtag="368" datatype="int32" presence="optional" />
</msg>
We are using the below method to create the groups via quickfix and have set a limitation in our end that if multiple repeating groups have the same fix tag, they should all share the same delimiter.
public Group(int field, int delim)
Does quickfix/J support setting multiple delimiters for the same group tag ?
Thanks,
Kodippili Arachchige Asanka
Specialist Software Engineer, MillenniumIT
LSEG Technology
Telephone +94 11 241 6000 Ext 26717
Mobile +94 77 9627002
as...@ls...<mailto:as...@ls...>
MillenniumIT, No 01, Millennium Drive, Malabe
www.lseg.com<http://www.lseg.com/>
[Description: cid:image001.png@01D37A69.7DB53A20]
[Description: cid:image002.png@01D37A69.7DB53A20]
Please consider the environment before printing this email.
------------------------------------------------------------------------------------------------------------
Please read these warnings and restrictions:
This e-mail transmission is strictly confidential and intended solely for the ordinary user of the e-mail address to which it was addressed. It may contain legally privileged and/or CONFIDENTIAL information.
The unauthorised use, disclosure, distribution and/or copying of this e-mail or any information it contains is prohibited and could, in certain circumstances, constitute a criminal offence.
If you have received this e-mail in error or are not an intended recipient please inform the London Stock Exchange immediately by return e-mail or telephone 020 7797 1000.
LSEG may collect, process and retain your personal information for its business purposes. For more information please see our Privacy Policy.
We advise that in keeping with good computing practice the recipient of this e-mail should ensure that it is virus free. We do not accept responsibility for any virus that may be transferred by way of this e-mail.
E-mail may be susceptible to data corruption, interception and unauthorised amendment, and we do not accept liability for any such corruption, interception or amendment or any consequences thereof.
Calls to the London Stock Exchange may be recorded to enable the Exchange to carry out its regulatory responsibilities.
London Stock Exchange plc
10 Paternoster Square
London
EC4M 7LS
Registered in England and Wales No 2075721
------------------------------------------------------------------------------------------------------------
|
|
From: <tom...@up...> - 2018-12-06 12:15:37
|
Hi Christoph, Ok, Im going to try it out anyway. I dont expect much of a slowdown, and according to the docs it should be safer to do anyway :D. Thank you for your help! Kind regards // Herzliche Grüße, -- Tom Tempelaere Upsilon SA From: Christoph John <chr...@ma...> Sent: Thursday, 6 December 2018 12:15 To: tom...@up... Cc: qui...@li... Subject: Re: [Quickfixj-users] Server crash leaves QuickFIX/J engine (1.5.3) in corrupt state for FileStorePath=filestore Hi Tom, to be honest: I don't know. That message is in the docs since 2006. :) So I assume with a recent server and an SSD the performance hit should be substantially lower. I never measured it but would assume some milliseconds. Grüße, :) Chris. On 06/12/2018 12:09, tom...@up... <mailto:tom...@up...> wrote: Hi Christoph, I DuckDuckGod FileStoreSync and see in the JavaDoc for 1.6.4 the message its safer to sync, but its also much slower (100x slower in some cases). Im wondering by how much time things are slowed down now. Are we talking about milliseconds per FIX message on current servers? Of course it all depends on the file subsystem (hard drive vs SSD), I know ;-) Kind regards // Herzliche Grüße, -- Tom Tempelaere Upsilon SA From: Christoph John <mailto:chr...@ma...> <chr...@ma...> Sent: Thursday, 6 December 2018 10:26 To: qui...@li... <mailto:qui...@li...> ; tom...@up... <mailto:tom...@up...> Subject: Re: [Quickfixj-users] Server crash leaves QuickFIX/J engine (1.5.3) in corrupt state for FileStorePath=filestore Hi Tom, IMHO you can always run into this kind of problem (data corruption) when the power fails. You could try turning on synchronous writes but this of course will slow down the message processing. The option to turn this on is FileStoreSync=Y. I just discovered that it is not part of the documentation. :-/ Will add it. Hope that helps a bit. Cheers, Chris. On 06/12/2018 09:33, tom...@up... <mailto:tom...@up...> wrote: QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ QuickFIX/J Support: http://www.quickfixj.org/support/ Hello dear readers, This is a situation that Ive encountered twice now, at two different clients of ours. Their application server with our FIX module running on it based on the QuickFIX/J engine (1.5.3) crashes, and leaves the QuickFIX/J engine in a corrupt state. The first time, the server crash was for a virtualized server (some reason the hosting provider never told me) and the second was due a UPS that did not signal the server to shut down properly when the battery started running out. Our configuration has FileStorePath=filestore in both instances. Our instances are buy side (i.e. initiators). On inspection of the QuickFIX/J state files, Im seeing that they are corrupted. I can fix this by removing all state files and restart our module, then all is fine again. The first time happened about a year ago, the second just earlier today. All state files were empty (.body, .header, .senderseqnums, .targetseqnums), except for the .session file (see attachment). Youll see its filled with all ASCII NUL characters. A valid one as Ive noticed has something like <NUL><ETX><some number> as contents. The exceptions Im getting in my modules log file goes like: quickfix.ConfigError: error during session initialization at quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocke tInitiator.java:169) at quickfix.mina.initiator.AbstractSocketInitiator.createSessionInitiators(Abst ractSocketInitiator.java:84) at quickfix.SocketInitiator.initialize(SocketInitiator.java:86) at quickfix.SocketInitiator.start(SocketInitiator.java:64) at <INTERNALS-STRIPPED> at <INTERNALS-STRIPPED> at <INTERNALS-STRIPPED> at java.lang.Thread.run(Unknown Source) Caused by: java.lang.RuntimeException: java.io.IOException: invalid UTC timestamp value: at quickfix.FileStoreFactory.create(FileStoreFactory.java:80) at quickfix.Session.<init>(Session.java:467) at quickfix.DefaultSessionFactory.create(DefaultSessionFactory.java:185) at quickfix.mina.SessionConnector.createSession(SessionConnector.java:140) at quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocke tInitiator.java:163) ... 7 more Caused by: java.io.IOException: invalid UTC timestamp value: at quickfix.FileStore.initializeSessionCreateTime(FileStore.java:137) at quickfix.FileStore.initializeCache(FileStore.java:123) at quickfix.FileStore.initialize(FileStore.java:116) at quickfix.FileStore.<init>(FileStore.java:101) at quickfix.FileStoreFactory.create(FileStoreFactory.java:78) ... 11 more I remember this is the same exception I got the first time one of our clients server crashed. Of course in an ideal world a server should never crash, but alas it does. It would appear that the QuickFIX/J engines state isnt properly flushed to disk? Or maybe this happens because the engines state cannot be written atomically? I do not know, as Ive never had the time to delve into QuickFIX/J internals and see how this situation can arise. Perhaps it has been reported before and fixed in a later version, Ive not had time to delve into that either. Perhaps this is something that simply cant be fixed due to the way filesystems work? Or perhaps it can? Besides this, Im pretty happy with QuickFIX/J 1.5.3 and I wish to thank all contributors to QuickFIX/J! Kind regards, -- Tom Tempelaere Upsilon SA _______________________________________________ Quickfixj-users mailing list Qui...@li... <mailto:Qui...@li...> https://lists.sourceforge.net/lists/listinfo/quickfixj-users -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... <mailto:chr...@ma...> MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com <http://www.macd.com> Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... <mailto:chr...@ma...> MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com <http://www.macd.com> Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Christoph J. <chr...@ma...> - 2018-12-06 11:15:43
|
Hi Tom, to be honest: I don't know. That message is in the docs since 2006. :) So I assume with a recent server and an SSD the performance hit should be substantially lower. I never measured it but would assume some milliseconds. Grüße, :) Chris. On 06/12/2018 12:09, tom...@up... wrote: > > Hi Christoph, > > I DuckDuckGo’d FileStoreSync and see in the JavaDoc for 1.6.4 the message “it’s safer to sync, but > it’s also much slower (100x slower in some cases)”. > > I’m wondering by how much time things are slowed down now. Are we talking about milliseconds per > FIX message on current servers? Of course it all depends on the file subsystem (hard drive vs > SSD), I know ;-) > > Kind regards // Herzliche Grüße, > > -- > > Tom Tempelaere > > Upsilon SA > > *From:*Christoph John <chr...@ma...> > *Sent:* Thursday, 6 December 2018 10:26 > *To:* qui...@li...; tom...@up... > *Subject:* Re: [Quickfixj-users] Server crash leaves QuickFIX/J engine (1.5.3) in corrupt state > for FileStorePath=filestore > > Hi Tom, > > IMHO you can always run into this kind of problem (data corruption) when the power fails. You > could try turning on synchronous writes but this of course will slow down the message processing. > The option to turn this on is FileStoreSync=Y. I just discovered that it is not part of the > documentation. :-/ Will add it. > > Hope that helps a bit. > Cheers, > Chris. > > > On 06/12/2018 09:33, tom...@up... <mailto:tom...@up...>wrote: > > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > > Hello dear readers, > > This is a situation that I’ve encountered twice now, at two different clients of ours. Their > application server with our FIX module running on it based on the QuickFIX/J engine (1.5.3) > crashes, and leaves the QuickFIX/J engine in a corrupt state. The first time, the server crash > was for a virtualized server (some reason the hosting provider never told me) and the second > was due a UPS that did not signal the server to shut down properly when the battery started > running out. Our configuration has /FileStorePath=filestore/ in both instances. Our instances > are buy side (i.e. initiators). > > On inspection of the QuickFIX/J state files, I’m seeing that they are corrupted. I can fix > this by removing all state files and restart our module, then all is fine again. The first > time happened about a year ago, the second just earlier today. All state files were empty > (.body, .header, .senderseqnums, .targetseqnums), except for the .session file (see > attachment). You’ll see it’s filled with all ASCII NUL characters. A valid one as I’ve noticed > has something like /<NUL><ETX><some number>/ as contents. > > The exceptions I’m getting in my module’s log file goes like: > > quickfix.ConfigError: error during session initialization > > at > quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocketInitiator.java:169) > > at > quickfix.mina.initiator.AbstractSocketInitiator.createSessionInitiators(AbstractSocketInitiator.java:84) > > at quickfix.SocketInitiator.initialize(SocketInitiator.java:86) > > at quickfix.SocketInitiator.start(SocketInitiator.java:64) > > at <INTERNALS-STRIPPED> > > at <INTERNALS-STRIPPED> > > at <INTERNALS-STRIPPED> > > at java.lang.Thread.run(Unknown Source) > > Caused by: java.lang.RuntimeException: java.io.IOException: invalid UTC timestamp value: > > at quickfix.FileStoreFactory.create(FileStoreFactory.java:80) > > at quickfix.Session.<init>(Session.java:467) > > at quickfix.DefaultSessionFactory.create(DefaultSessionFactory.java:185) > > at quickfix.mina.SessionConnector.createSession(SessionConnector.java:140) > > at > quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocketInitiator.java:163) > > ... 7 more > > Caused by: java.io.IOException: invalid UTC timestamp value: > > at quickfix.FileStore.initializeSessionCreateTime(FileStore.java:137) > > at quickfix.FileStore.initializeCache(FileStore.java:123) > > at quickfix.FileStore.initialize(FileStore.java:116) > > at quickfix.FileStore.<init>(FileStore.java:101) > > at quickfix.FileStoreFactory.create(FileStoreFactory.java:78) > > ... 11 more > > I remember this is the same exception I got the first time one of our clients’ server crashed. > > Of course in an ideal world a server should never crash, but alas it does. It would appear > that the QuickFIX/J engine’s state isn’t properly flushed to disk? Or maybe this happens > because the engine’s state cannot be written atomically? I do not know, as I’ve never had the > time to delve into QuickFIX/J internals and see how this situation can arise. > > Perhaps it has been reported before and fixed in a later version, I’ve not had time to delve > into that either. Perhaps this is something that simply can’t be fixed due to the way > filesystems work? Or perhaps it can? > > Besides this, I’m pretty happy with QuickFIX/J 1.5.3 and I wish to thank all contributors to > QuickFIX/J! > > Kind regards, > > -- > > Tom Tempelaere > > Upsilon SA > > > > > _______________________________________________ > > Quickfixj-users mailing list > > Qui...@li... <mailto:Qui...@li...> > > https://lists.sourceforge.net/lists/listinfo/quickfixj-users > > > > -- > Christoph John > Software Engineering > T +49 241 557080-28 > chr...@ma... <mailto:chr...@ma...> > MACD GmbH > Oppenhoffallee 103 > 52066 Aachen, Germany > www.macd.com <http://www.macd.com> > Amtsgericht Aachen: HRB 8151 > Ust.-Id: DE 813021663 > Geschäftsführer: George Macdonald -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: <tom...@up...> - 2018-12-06 11:09:36
|
Hi Christoph, I DuckDuckGod FileStoreSync and see in the JavaDoc for 1.6.4 the message its safer to sync, but its also much slower (100x slower in some cases). Im wondering by how much time things are slowed down now. Are we talking about milliseconds per FIX message on current servers? Of course it all depends on the file subsystem (hard drive vs SSD), I know ;-) Kind regards // Herzliche Grüße, -- Tom Tempelaere Upsilon SA From: Christoph John <chr...@ma...> Sent: Thursday, 6 December 2018 10:26 To: qui...@li...; tom...@up... Subject: Re: [Quickfixj-users] Server crash leaves QuickFIX/J engine (1.5.3) in corrupt state for FileStorePath=filestore Hi Tom, IMHO you can always run into this kind of problem (data corruption) when the power fails. You could try turning on synchronous writes but this of course will slow down the message processing. The option to turn this on is FileStoreSync=Y. I just discovered that it is not part of the documentation. :-/ Will add it. Hope that helps a bit. Cheers, Chris. On 06/12/2018 09:33, tom...@up... <mailto:tom...@up...> wrote: QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ QuickFIX/J Support: http://www.quickfixj.org/support/ Hello dear readers, This is a situation that Ive encountered twice now, at two different clients of ours. Their application server with our FIX module running on it based on the QuickFIX/J engine (1.5.3) crashes, and leaves the QuickFIX/J engine in a corrupt state. The first time, the server crash was for a virtualized server (some reason the hosting provider never told me) and the second was due a UPS that did not signal the server to shut down properly when the battery started running out. Our configuration has FileStorePath=filestore in both instances. Our instances are buy side (i.e. initiators). On inspection of the QuickFIX/J state files, Im seeing that they are corrupted. I can fix this by removing all state files and restart our module, then all is fine again. The first time happened about a year ago, the second just earlier today. All state files were empty (.body, .header, .senderseqnums, .targetseqnums), except for the .session file (see attachment). Youll see its filled with all ASCII NUL characters. A valid one as Ive noticed has something like <NUL><ETX><some number> as contents. The exceptions Im getting in my modules log file goes like: quickfix.ConfigError: error during session initialization at quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocke tInitiator.java:169) at quickfix.mina.initiator.AbstractSocketInitiator.createSessionInitiators(Abst ractSocketInitiator.java:84) at quickfix.SocketInitiator.initialize(SocketInitiator.java:86) at quickfix.SocketInitiator.start(SocketInitiator.java:64) at <INTERNALS-STRIPPED> at <INTERNALS-STRIPPED> at <INTERNALS-STRIPPED> at java.lang.Thread.run(Unknown Source) Caused by: java.lang.RuntimeException: java.io.IOException: invalid UTC timestamp value: at quickfix.FileStoreFactory.create(FileStoreFactory.java:80) at quickfix.Session.<init>(Session.java:467) at quickfix.DefaultSessionFactory.create(DefaultSessionFactory.java:185) at quickfix.mina.SessionConnector.createSession(SessionConnector.java:140) at quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocke tInitiator.java:163) ... 7 more Caused by: java.io.IOException: invalid UTC timestamp value: at quickfix.FileStore.initializeSessionCreateTime(FileStore.java:137) at quickfix.FileStore.initializeCache(FileStore.java:123) at quickfix.FileStore.initialize(FileStore.java:116) at quickfix.FileStore.<init>(FileStore.java:101) at quickfix.FileStoreFactory.create(FileStoreFactory.java:78) ... 11 more I remember this is the same exception I got the first time one of our clients server crashed. Of course in an ideal world a server should never crash, but alas it does. It would appear that the QuickFIX/J engines state isnt properly flushed to disk? Or maybe this happens because the engines state cannot be written atomically? I do not know, as Ive never had the time to delve into QuickFIX/J internals and see how this situation can arise. Perhaps it has been reported before and fixed in a later version, Ive not had time to delve into that either. Perhaps this is something that simply cant be fixed due to the way filesystems work? Or perhaps it can? Besides this, Im pretty happy with QuickFIX/J 1.5.3 and I wish to thank all contributors to QuickFIX/J! Kind regards, -- Tom Tempelaere Upsilon SA _______________________________________________ Quickfixj-users mailing list Qui...@li... <mailto:Qui...@li...> https://lists.sourceforge.net/lists/listinfo/quickfixj-users -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... <mailto:chr...@ma...> MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com <http://www.macd.com> Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: <tom...@up...> - 2018-12-06 09:51:41
|
Hi Christoph, Thanks for the hint :). Ill try this one. Our clients are not high volume traders, so a minor performance drop should be OK still. Kind regards // Herzliche Grüße, -- Tom Tempelaere Upsilon SA From: Christoph John <chr...@ma...> Sent: Thursday, 6 December 2018 10:26 To: qui...@li...; tom...@up... Subject: Re: [Quickfixj-users] Server crash leaves QuickFIX/J engine (1.5.3) in corrupt state for FileStorePath=filestore Hi Tom, IMHO you can always run into this kind of problem (data corruption) when the power fails. You could try turning on synchronous writes but this of course will slow down the message processing. The option to turn this on is FileStoreSync=Y. I just discovered that it is not part of the documentation. :-/ Will add it. Hope that helps a bit. Cheers, Chris. On 06/12/2018 09:33, tom...@up... <mailto:tom...@up...> wrote: QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ QuickFIX/J Support: http://www.quickfixj.org/support/ Hello dear readers, This is a situation that Ive encountered twice now, at two different clients of ours. Their application server with our FIX module running on it based on the QuickFIX/J engine (1.5.3) crashes, and leaves the QuickFIX/J engine in a corrupt state. The first time, the server crash was for a virtualized server (some reason the hosting provider never told me) and the second was due a UPS that did not signal the server to shut down properly when the battery started running out. Our configuration has FileStorePath=filestore in both instances. Our instances are buy side (i.e. initiators). On inspection of the QuickFIX/J state files, Im seeing that they are corrupted. I can fix this by removing all state files and restart our module, then all is fine again. The first time happened about a year ago, the second just earlier today. All state files were empty (.body, .header, .senderseqnums, .targetseqnums), except for the .session file (see attachment). Youll see its filled with all ASCII NUL characters. A valid one as Ive noticed has something like <NUL><ETX><some number> as contents. The exceptions Im getting in my modules log file goes like: quickfix.ConfigError: error during session initialization at quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocke tInitiator.java:169) at quickfix.mina.initiator.AbstractSocketInitiator.createSessionInitiators(Abst ractSocketInitiator.java:84) at quickfix.SocketInitiator.initialize(SocketInitiator.java:86) at quickfix.SocketInitiator.start(SocketInitiator.java:64) at <INTERNALS-STRIPPED> at <INTERNALS-STRIPPED> at <INTERNALS-STRIPPED> at java.lang.Thread.run(Unknown Source) Caused by: java.lang.RuntimeException: java.io.IOException: invalid UTC timestamp value: at quickfix.FileStoreFactory.create(FileStoreFactory.java:80) at quickfix.Session.<init>(Session.java:467) at quickfix.DefaultSessionFactory.create(DefaultSessionFactory.java:185) at quickfix.mina.SessionConnector.createSession(SessionConnector.java:140) at quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocke tInitiator.java:163) ... 7 more Caused by: java.io.IOException: invalid UTC timestamp value: at quickfix.FileStore.initializeSessionCreateTime(FileStore.java:137) at quickfix.FileStore.initializeCache(FileStore.java:123) at quickfix.FileStore.initialize(FileStore.java:116) at quickfix.FileStore.<init>(FileStore.java:101) at quickfix.FileStoreFactory.create(FileStoreFactory.java:78) ... 11 more I remember this is the same exception I got the first time one of our clients server crashed. Of course in an ideal world a server should never crash, but alas it does. It would appear that the QuickFIX/J engines state isnt properly flushed to disk? Or maybe this happens because the engines state cannot be written atomically? I do not know, as Ive never had the time to delve into QuickFIX/J internals and see how this situation can arise. Perhaps it has been reported before and fixed in a later version, Ive not had time to delve into that either. Perhaps this is something that simply cant be fixed due to the way filesystems work? Or perhaps it can? Besides this, Im pretty happy with QuickFIX/J 1.5.3 and I wish to thank all contributors to QuickFIX/J! Kind regards, -- Tom Tempelaere Upsilon SA _______________________________________________ Quickfixj-users mailing list Qui...@li... <mailto:Qui...@li...> https://lists.sourceforge.net/lists/listinfo/quickfixj-users -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... <mailto:chr...@ma...> MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com <http://www.macd.com> Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Christoph J. <chr...@ma...> - 2018-12-06 09:26:10
|
Hi Tom, IMHO you can always run into this kind of problem (data corruption) when the power fails. You could try turning on synchronous writes but this of course will slow down the message processing. The option to turn this on is FileStoreSync=Y. I just discovered that it is not part of the documentation. :-/ Will add it. Hope that helps a bit. Cheers, Chris. On 06/12/2018 09:33, tom...@up... wrote: > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > > Hello dear readers, > > This is a situation that I’ve encountered twice now, at two different clients of ours. Their > application server with our FIX module running on it based on the QuickFIX/J engine (1.5.3) > crashes, and leaves the QuickFIX/J engine in a corrupt state. The first time, the server crash was > for a virtualized server (some reason the hosting provider never told me) and the second was due a > UPS that did not signal the server to shut down properly when the battery started running out. Our > configuration has /FileStorePath=filestore/ in both instances. Our instances are buy side (i.e. > initiators). > > On inspection of the QuickFIX/J state files, I’m seeing that they are corrupted. I can fix this by > removing all state files and restart our module, then all is fine again. The first time happened > about a year ago, the second just earlier today. All state files were empty (.body, .header, > .senderseqnums, .targetseqnums), except for the .session file (see attachment). You’ll see it’s > filled with all ASCII NUL characters. A valid one as I’ve noticed has something like > /<NUL><ETX><some number>/ as contents. > > The exceptions I’m getting in my module’s log file goes like: > > quickfix.ConfigError: error during session initialization > > at > quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocketInitiator.java:169) > > at > quickfix.mina.initiator.AbstractSocketInitiator.createSessionInitiators(AbstractSocketInitiator.java:84) > > at quickfix.SocketInitiator.initialize(SocketInitiator.java:86) > > at quickfix.SocketInitiator.start(SocketInitiator.java:64) > > at <INTERNALS-STRIPPED> > > at <INTERNALS-STRIPPED> > > at <INTERNALS-STRIPPED> > > at java.lang.Thread.run(Unknown Source) > > Caused by: java.lang.RuntimeException: java.io.IOException: invalid UTC timestamp value: > > at quickfix.FileStoreFactory.create(FileStoreFactory.java:80) > > at quickfix.Session.<init>(Session.java:467) > > at quickfix.DefaultSessionFactory.create(DefaultSessionFactory.java:185) > > at quickfix.mina.SessionConnector.createSession(SessionConnector.java:140) > > at > quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocketInitiator.java:163) > > ... 7 more > > Caused by: java.io.IOException: invalid UTC timestamp value: > > at quickfix.FileStore.initializeSessionCreateTime(FileStore.java:137) > > at quickfix.FileStore.initializeCache(FileStore.java:123) > > at quickfix.FileStore.initialize(FileStore.java:116) > > at quickfix.FileStore.<init>(FileStore.java:101) > > at quickfix.FileStoreFactory.create(FileStoreFactory.java:78) > > ... 11 more > > I remember this is the same exception I got the first time one of our clients’ server crashed. > > Of course in an ideal world a server should never crash, but alas it does. It would appear that > the QuickFIX/J engine’s state isn’t properly flushed to disk? Or maybe this happens because the > engine’s state cannot be written atomically? I do not know, as I’ve never had the time to delve > into QuickFIX/J internals and see how this situation can arise. > > Perhaps it has been reported before and fixed in a later version, I’ve not had time to delve into > that either. Perhaps this is something that simply can’t be fixed due to the way filesystems work? > Or perhaps it can? > > Besides this, I’m pretty happy with QuickFIX/J 1.5.3 and I wish to thank all contributors to > QuickFIX/J! > > Kind regards, > > -- > > Tom Tempelaere > > Upsilon SA > > > > _______________________________________________ > Quickfixj-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfixj-users -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: <tom...@up...> - 2018-12-06 08:49:32
|
Hello dear readers,
This is a situation that I've encountered twice now, at two different
clients of ours. Their application server with our FIX module running on it
based on the QuickFIX/J engine (1.5.3) crashes, and leaves the QuickFIX/J
engine in a corrupt state. The first time, the server crash was for a
virtualized server (some reason the hosting provider never told me) and the
second was due a UPS that did not signal the server to shut down properly
when the battery started running out. Our configuration has
FileStorePath=filestore in both instances. Our instances are buy side (i.e.
initiators).
On inspection of the QuickFIX/J state files, I'm seeing that they are
corrupted. I can fix this by removing all state files and restart our
module, then all is fine again. The first time happened about a year ago,
the second just earlier today. All state files were empty (.body, .header,
.senderseqnums, .targetseqnums), except for the .session file (see
attachment). You'll see it's filled with all ASCII NUL characters. A valid
one as I've noticed has something like <NUL><ETX><some number> as contents.
The exceptions I'm getting in my module's log file goes like:
quickfix.ConfigError: error during session initialization
at
quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocke
tInitiator.java:169)
at
quickfix.mina.initiator.AbstractSocketInitiator.createSessionInitiators(Abst
ractSocketInitiator.java:84)
at quickfix.SocketInitiator.initialize(SocketInitiator.java:86)
at quickfix.SocketInitiator.start(SocketInitiator.java:64)
at <INTERNALS-STRIPPED>
at <INTERNALS-STRIPPED>
at <INTERNALS-STRIPPED>
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: java.io.IOException: invalid UTC
timestamp value:
at quickfix.FileStoreFactory.create(FileStoreFactory.java:80)
at quickfix.Session.<init>(Session.java:467)
at
quickfix.DefaultSessionFactory.create(DefaultSessionFactory.java:185)
at
quickfix.mina.SessionConnector.createSession(SessionConnector.java:140)
at
quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocke
tInitiator.java:163)
... 7 more
Caused by: java.io.IOException: invalid UTC timestamp value:
at
quickfix.FileStore.initializeSessionCreateTime(FileStore.java:137)
at quickfix.FileStore.initializeCache(FileStore.java:123)
at quickfix.FileStore.initialize(FileStore.java:116)
at quickfix.FileStore.<init>(FileStore.java:101)
at quickfix.FileStoreFactory.create(FileStoreFactory.java:78)
... 11 more
I remember this is the same exception I got the first time one of our
clients' server crashed.
Of course in an ideal world a server should never crash, but alas it does.
It would appear that the QuickFIX/J engine's state isn't properly flushed to
disk? Or maybe this happens because the engine's state cannot be written
atomically? I do not know, as I've never had the time to delve into
QuickFIX/J internals and see how this situation can arise.
Perhaps it has been reported before and fixed in a later version, I've not
had time to delve into that either. Perhaps this is something that simply
can't be fixed due to the way filesystems work? Or perhaps it can?
Besides this, I'm pretty happy with QuickFIX/J 1.5.3 and I wish to thank all
contributors to QuickFIX/J!
Kind regards,
--
Tom Tempelaere
Upsilon SA
|
|
From: Daniel C. <da...@bl...> - 2018-12-04 10:17:26
|
Hi Christopher, Looking at the code, I agree with the approach, The Exception should be handed off to the eventHandlingStrategy to ensure that any pending messages are processed before disconnecting. I have created a JIRA issue for this : https://www.quickfixj.org/jira/browse/QFJ-966 I will see if i can find some time to try replicate and fix. Regards Daniel On Mon, Dec 3, 2018 at 11:54 AM Christoph John <chr...@ma...> wrote: > Hi Daniel, > > thanks for the stack trace and the kudos! :) > The stack trace is quite helpful for the diagnosis. So this seems to be a > special case that is not covered by the unit test. > The problem is that the session is disconnected the "hard way" through the > IOException, not waiting for messages to be processed that might still be > in flight. This might have to be changed in the AbstractIoHandler to behave > similar to the sessionClosed() callback. > > But this is only a first quick analysis. Of course this has to be made > reproducable in a unit test and then fixed. > > Cheers, > Chris. > > On 03/12/2018 12:17, Daniel Cullender wrote: > > Hi Chris, > > Thanks for the response and for all the hard work. > > spot on - it is not in the QFJ log file but I can see it in the tcp dump. > > I did have a case where I received it once, so I think there is a race > condition somewhere > > Funnily enough I was actually playing around with that exact test case > this weekend trying to replicate it. The issue though is on the initiator > side not receiving the admin logout message, not the acceptor side. > > The initiator stack trace (ie connection reset) looks like > > "..." > "\tat quickfix.Session.disconnect(Session.java:2028)" > "\tat > quickfix.mina.AbstractIoHandler.exceptionCaught(AbstractIoHandler.java:84)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.exceptionCaught(DefaultIoFilterChain.java:828)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:937)" > "\tat > org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:102)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:937)" > "\tat > org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:102)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.fireExceptionCaught(DefaultIoFilterChain.java:580)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:729)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:659)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:648)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:68)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1120)" > "\tat > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)" > "\tat > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)" > "\tat > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)" > "\tat java.base/java.lang.Thread.run(Thread.java:844)" > > and all I receive on the initiator side is > > "Initiated logon request" > "Disconnecting: Socket exception (/207.239.27.218:25265): > java.io.IOException: Connection reset by peer" > "onLogout: FIX.4.2:blk2->JLQD" > > > Thanks again > Daniel > > > > On Mon, Dec 3, 2018 at 10:38 AM Christoph John <chr...@ma...> > wrote: > >> So you only see the Logout message in the tcpdump but not in the QFJ log >> file? >> >> Yes, I actually thought that this has been fixed some time ago and we >> have unit tests which test this specific behaviour. >> E.g. >> https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/mina/LostLogoutTest.java >> >> Cheers, >> Chris. >> >> >> On 30/11/2018 19:14, Daniel Cullender via Quickfixj-users wrote: >> >> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ >> QuickFIX/J Support: http://www.quickfixj.org/support/ >> >> >> Hi, >> >> I think this is an old issue, but still seems to be a problem with the >> latest version of quickfixj 2.1.0. >> >> After my initiator sends a Logon message, the acceptor immediately sends >> back a Logout Message followed by closing the connection. Looking at the >> tcpdump, it is clear why the connection is closed >> >> x.x.x.x.56988 > x.x.x.x.25265: Flags [P.], cksum 0x9863 (incorrect -> >> 0xe431), seq 1:104, ack 1, win 21, length 103 >> E.....@.@..t..........b >> .(.7.>..+P....c..8=FIX.4.2.9=81.35=A.34=1161.49=XXX.52=20181130-15:08:36.259.56=***.98=0.108=10.554=***.10=026. >> 15:08:36.336321 IP (tos 0x60, ttl 48, id 0, offset 0, flags [DF], proto >> TCP (6), length 188) >> x.x.x.x.x > x.x.x.x.56988: Flags [P.], cksum 0xdd5d (correct), seq >> 1:149, ack 104, win 4096, length 148 >> E`....@.0...........b...>..+(.8YP....]..*8=FIX.4.2.9=0124.35=5.34=552397.49=XXX.52=20181130-15:08:36.298.56=XXX.789=469881.58=MsgSeqNum >> too low, expecting 469881 but received 1161.10=079*. >> >> The problem is that *fromAdmin* is *never* called before *onLogout* because >> of the connection reset following the logout message. I really need to >> access this logout message for a couple of internal reasons. >> >> Is there any workaround for this issue available? >> >> Thanks >> Daniel Cullender >> >> >> > -- > Christoph John > Software Engineering > T +49 241 557...@ma... > > MACD GmbH > Oppenhoffallee 103 > 52066 Aachen, Germanywww.macd.com > > Amtsgericht Aachen: HRB 8151 > Ust.-Id: DE 813021663 > Geschäftsführer: George Macdonald > > |
|
From: Christoph J. <chr...@ma...> - 2018-12-03 11:54:25
|
Hi Daniel, thanks for the stack trace and the kudos! :) The stack trace is quite helpful for the diagnosis. So this seems to be a special case that is not covered by the unit test. The problem is that the session is disconnected the "hard way" through the IOException, not waiting for messages to be processed that might still be in flight. This might have to be changed in the AbstractIoHandler to behave similar to the sessionClosed() callback. But this is only a first quick analysis. Of course this has to be made reproducable in a unit test and then fixed. Cheers, Chris. On 03/12/2018 12:17, Daniel Cullender wrote: > Hi Chris, > > Thanks for the response and for all the hard work. > > spot on - it is not in the QFJ log file but I can see it in the tcp dump. > > I did have a case where I received it once, so I think there is a race condition somewhere > > Funnily enough I was actually playing around with that exact test case this weekend trying to > replicate it. The issue though is on the initiator side not receiving the admin logout message, > not the acceptor side. > > The initiator stack trace (ie connection reset) looks like > > "..." > "\tat quickfix.Session.disconnect(Session.java:2028)" > "\tat quickfix.mina.AbstractIoHandler.exceptionCaught(AbstractIoHandler.java:84)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.exceptionCaught(DefaultIoFilterChain.java:828)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" > "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:937)" > "\tat org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:102)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" > "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:937)" > "\tat org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:102)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" > "\tat > org.apache.mina.core.filterchain.DefaultIoFilterChain.fireExceptionCaught(DefaultIoFilterChain.java:580)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:729)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:659)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:648)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:68)" > "\tat > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1120)" > "\tat org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)" > "\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)" > "\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)" > "\tat java.base/java.lang.Thread.run(Thread.java:844)" > > and all I receive on the initiator side is > > "Initiated logon request" > "Disconnecting: Socket exception (/207.239.27.218:25265 <http://207.239.27.218:25265>): > java.io.IOException: Connection reset by peer" > "onLogout: FIX.4.2:blk2->JLQD" > > > Thanks again > Daniel > > > > On Mon, Dec 3, 2018 at 10:38 AM Christoph John <chr...@ma... > <mailto:chr...@ma...>> wrote: > > So you only see the Logout message in the tcpdump but not in the QFJ log file? > > Yes, I actually thought that this has been fixed some time ago and we have unit tests which > test this specific behaviour. > E.g. > https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/mina/LostLogoutTest.java > > Cheers, > Chris. > > > On 30/11/2018 19:14, Daniel Cullender via Quickfixj-users wrote: >> QuickFIX/J Documentation:http://www.quickfixj.org/documentation/ >> QuickFIX/J Support:http://www.quickfixj.org/support/ >> >> >> >> Hi, >> >> I think this is an old issue, but still seems to be a problem with the latest version of >> quickfixj 2.1.0. >> >> After my initiator sends a Logon message, the acceptor immediately sends back a Logout >> Message followed by closing the connection. Looking at the tcpdump, it is clear why the >> connection is closed >> >> x.x.x.x.56988 > x.x.x.x.25265: Flags [P.], cksum 0x9863 (incorrect -> 0xe431), seq 1:104, ack >> 1, win 21, length 103 >> E.....@.@..t..........b >> <mailto:E.....@.@..t..........b>.(.7.>..+P....c..8=FIX.4.2.9=81.35=A.34=1161.49=XXX.52=20181130-15:08:36.259.56=***.98=0.108=10.554=***.10=026. >> 15:08:36.336321 IP (tos 0x60, ttl 48, id 0, offset 0, flags [DF], proto TCP (6), length 188) >> x.x.x.x.x > x.x.x.x.56988: Flags [P.], cksum 0xdd5d (correct), seq 1:149, ack 104, win >> 4096, length 148 >> E`....@.0...........b...>..+(.8YP....]..*8=FIX.4.2.9=0124.35=5.34=552397.49=XXX.52=20181130-15:08:36.298.56=XXX.789=469881.58=MsgSeqNum >> too low, expecting 469881 but received 1161.10=079*. >> >> The problem is that *fromAdmin*is /never/ called before *onLogout* because of the connection >> reset following the logout message. I really need to access this logout message for a couple >> of internal reasons. >> >> Is there any workaround for this issue available? >> >> Thanks >> Daniel Cullender > -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Daniel C. <da...@bl...> - 2018-12-03 11:18:19
|
Hi Chris, Thanks for the response and for all the hard work. spot on - it is not in the QFJ log file but I can see it in the tcp dump. I did have a case where I received it once, so I think there is a race condition somewhere Funnily enough I was actually playing around with that exact test case this weekend trying to replicate it. The issue though is on the initiator side not receiving the admin logout message, not the acceptor side. The initiator stack trace (ie connection reset) looks like "..." "\tat quickfix.Session.disconnect(Session.java:2028)" "\tat quickfix.mina.AbstractIoHandler.exceptionCaught(AbstractIoHandler.java:84)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.exceptionCaught(DefaultIoFilterChain.java:828)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:937)" "\tat org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:102)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:937)" "\tat org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:102)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)" "\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.fireExceptionCaught(DefaultIoFilterChain.java:580)" "\tat org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:729)" "\tat org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:659)" "\tat org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:648)" "\tat org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:68)" "\tat org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1120)" "\tat org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)" "\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)" "\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)" "\tat java.base/java.lang.Thread.run(Thread.java:844)" and all I receive on the initiator side is "Initiated logon request" "Disconnecting: Socket exception (/207.239.27.218:25265): java.io.IOException: Connection reset by peer" "onLogout: FIX.4.2:blk2->JLQD" Thanks again Daniel On Mon, Dec 3, 2018 at 10:38 AM Christoph John <chr...@ma...> wrote: > So you only see the Logout message in the tcpdump but not in the QFJ log > file? > > Yes, I actually thought that this has been fixed some time ago and we have > unit tests which test this specific behaviour. > E.g. > https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/mina/LostLogoutTest.java > > Cheers, > Chris. > > > On 30/11/2018 19:14, Daniel Cullender via Quickfixj-users wrote: > > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > Hi, > > I think this is an old issue, but still seems to be a problem with the > latest version of quickfixj 2.1.0. > > After my initiator sends a Logon message, the acceptor immediately sends > back a Logout Message followed by closing the connection. Looking at the > tcpdump, it is clear why the connection is closed > > x.x.x.x.56988 > x.x.x.x.25265: Flags [P.], cksum 0x9863 (incorrect -> > 0xe431), seq 1:104, ack 1, win 21, length 103 > E.....@.@..t..........b > .(.7.>..+P....c..8=FIX.4.2.9=81.35=A.34=1161.49=XXX.52=20181130-15:08:36.259.56=***.98=0.108=10.554=***.10=026. > 15:08:36.336321 IP (tos 0x60, ttl 48, id 0, offset 0, flags [DF], proto > TCP (6), length 188) > x.x.x.x.x > x.x.x.x.56988: Flags [P.], cksum 0xdd5d (correct), seq > 1:149, ack 104, win 4096, length 148 > E`....@.0...........b...>..+(.8YP....]..*8=FIX.4.2.9=0124.35=5.34=552397.49=XXX.52=20181130-15:08:36.298.56=XXX.789=469881.58=MsgSeqNum > too low, expecting 469881 but received 1161.10=079*. > > The problem is that *fromAdmin* is *never* called before *onLogout* because > of the connection reset following the logout message. I really need to > access this logout message for a couple of internal reasons. > > Is there any workaround for this issue available? > > Thanks > Daniel Cullender > > > -- > Christoph John > Software Engineering > T +49 241 557...@ma... > > MACD GmbH > Oppenhoffallee 103 > 52066 Aachen, Germanywww.macd.com > > Amtsgericht Aachen: HRB 8151 > Ust.-Id: DE 813021663 > Geschäftsführer: George Macdonald > > |
|
From: Christoph J. <chr...@ma...> - 2018-12-03 10:38:27
|
So you only see the Logout message in the tcpdump but not in the QFJ log file? Yes, I actually thought that this has been fixed some time ago and we have unit tests which test this specific behaviour. E.g. https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/mina/LostLogoutTest.java Cheers, Chris. On 30/11/2018 19:14, Daniel Cullender via Quickfixj-users wrote: > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > > Hi, > > I think this is an old issue, but still seems to be a problem with the latest version of quickfixj > 2.1.0. > > After my initiator sends a Logon message, the acceptor immediately sends back a Logout Message > followed by closing the connection. Looking at the tcpdump, it is clear why the connection is closed > > x.x.x.x.56988 > x.x.x.x.25265: Flags [P.], cksum 0x9863 (incorrect -> 0xe431), seq 1:104, ack > 1, win 21, length 103 > E.....@.@..t..........b.(.7.>..+P....c..8=FIX.4.2.9=81.35=A.34=1161.49=XXX.52=20181130-15:08:36.259.56=***.98=0.108=10.554=***.10=026. > 15:08:36.336321 IP (tos 0x60, ttl 48, id 0, offset 0, flags [DF], proto TCP (6), length 188) > x.x.x.x.x > x.x.x.x.56988: Flags [P.], cksum 0xdd5d (correct), seq 1:149, ack 104, win 4096, > length 148 > E`....@.0...........b...>..+(.8YP....]..*8=FIX.4.2.9=0124.35=5.34=552397.49=XXX.52=20181130-15:08:36.298.56=XXX.789=469881.58=MsgSeqNum > too low, expecting 469881 but received 1161.10=079*. > > The problem is that *fromAdmin*is /never/ called before *onLogout* because of the connection reset > following the logout message. I really need to access this logout message for a couple of internal > reasons. > > Is there any workaround for this issue available? > > Thanks > Daniel Cullender -- Christoph John Software Engineering T +49 241 557080-28 chr...@ma... MACD GmbH Oppenhoffallee 103 52066 Aachen, Germany www.macd.com Amtsgericht Aachen: HRB 8151 Ust.-Id: DE 813021663 Geschäftsführer: George Macdonald |
|
From: Colin D. <co...@ma...> - 2018-12-01 13:24:39
|
It sounds like you're not actually objecting to the connection being closed, right? It's that, despite it being closed legitimately, you still need access to the logout message? I don't have an opinion about whether the behavior you describe is defective or not, but, if you want a workaround, you could create a custom message log or store and make the message available from that. On 11/30/18 10:14 AM, Daniel Cullender via Quickfixj-users wrote: > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/ > QuickFIX/J Support: http://www.quickfixj.org/support/ > > > > Hi, > > I think this is an old issue, but still seems to be a problem with the > latest version of quickfixj 2.1.0. > > After my initiator sends a Logon message, the acceptor immediately sends > back a Logout Message followed by closing the connection. Looking at the > tcpdump, it is clear why the connection is closed > > x.x.x.x.56988 > x.x.x.x.25265: Flags [P.], cksum 0x9863 (incorrect > -> 0xe431), seq 1:104, ack 1, win 21, length 103 > E.....@.@..t..........b.(.7.>..+P....c..8=FIX.4.2.9=81.35=A.34=1161.49=XXX.52=20181130-15:08:36.259.56=***.98=0.108=10.554=***.10=026. > 15:08:36.336321 IP (tos 0x60, ttl 48, id 0, offset 0, flags [DF], proto > TCP (6), length 188) > x.x.x.x.x > x.x.x.x.56988: Flags [P.], cksum 0xdd5d (correct), seq > 1:149, ack 104, win 4096, length 148 > E`....@.0...........b...>..+(.8YP....]..*8=FIX.4.2.9=0124.35=5.34=552397.49=XXX.52=20181130-15:08:36.298.56=XXX.789=469881.58=MsgSeqNum > too low, expecting 469881 but received 1161.10=079*. > > The problem is that *fromAdmin*is /never/ called before > *onLogout* because of the connection reset following the logout message. > I really need to access this logout message for a couple of internal > reasons. > > Is there any workaround for this issue available? > > Thanks > Daniel Cullender > > > > > _______________________________________________ > Quickfixj-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfixj-users > -- Colin DuPlantis Chief Architect, Marketcetera Download, Run, Trade 888.868.4884 https://www.marketcetera.com |
103 messages has been excluded from this view by a project administrator.
