quickfixj-users — General mailing list for QuickFIX/J topics.

Re: [Quickfixj-users] outgoing seqnum

[Robert B. <rbr...@me...>]

You're correct Joerg...I have been unclear...I'm using both the jdbc loggin=
g AND jdbc store.  I finally got the store to work with Oracle yesterday af=
ternoon by bypassing the proxool libraries altogether (As per Steve's sugge=
stion) and modifying the JdbcUtil class to use OracleConnection and OracleD=
ataSource objects directly...I then also had to severely modify the  JdbcSt=
ore class because we're not using session qualifiers in our config file, so=
 the session qualifier is an empty string, which in Oracle is NULL...all of=
 the update, delete and insert statements in the JdbcStore class use prepar=
ed statements like "WHERE session_qualifier=3D?" and this is invalid sql fo=
r oracle when you have a null/empty string condition...you cannot use "WHER=
E session_qualifier =3D null" for a logic check, you either need to leave i=
t out of the prepared statement or use "WHERE session_qualifier IS NULL".=
=20=20

For all of the update, delete and insert statements I had to create a secon=
d version of them and in all of the methods using these, I had to check the=
 value of the session qualifier and if it was an empty string I have to use=
 my NULL version of the prepared statements...I also had to add connection.=
commit(); to the bottom of every update, insert and delete method in order =
for my oracle store to work.=20=20

Today is the first true day we'll be running the engine with my custom libr=
ary changes so I'm sure I'll have to iron out some bugs.  Luckily we're sti=
ll in development phase!  When I have some time, I'd like to actually exten=
d the existing JdbcUtil, JdbcLog, JdbcStore classes with Oracle-friendly im=
plementations and submit it for review to be a part of the actual quickfix/=
j library...that's if my testing over the next few days goes well.

Thanks!

robert l. brueckmann
vice president
merlin securities
712 fifth avenue
new york, ny 10019
p: 212.822.4821
f: 212.822.4820



Merlin Securities - #1 Prime Broker North America, #1 Prime Broker Single S=
trategy Funds, #1 Prime Broker Funds Under $100M - Global Custodian 2007


From: qui...@li... [mailto:quickfixj-users=
-bo...@li...] On Behalf Of Joerg Thoennes
Sent: Thursday, May 24, 2007 6:04 AM
To: qui...@li...
Subject: Re: [Quickfixj-users] outgoing seqnum

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
Hi Robert,

we currently use JdbcStore, but not JdbcLog (you sometimes call things "log=
ging" where I would
expect the MessageStore; is it possible to you mix logging and saving state=
?).

In our test environment, all works well with JDBC access to PostgreSQL. Our=
 customer uses Oracle 8
with OCI, there are currently installing our new version and maybe I get so=
me feedback soon.

If I remember right, you are using Oracle 9? Which JDBC driver -- "thin:" o=
r "oci:"? Please give me
some more details here.

For the moment, I would recommend to replace the JdbcLog by Log4J logging. =
This is flexible enough
for most purpose. For failover, it is important to keep the JdbcStore.

In general, I am very interested in your experiences because our customer a=
lso uses Oracle.

Cheers, J=F6rg

On 05/21/07 20:55, Robert Brueckmann wrote:
> Thanks for the reply.  I did as you said and set up a test
> environment...I am using JDBC logging against Oracle.  I'm the one that
> had a barrage of emails a few weeks ago about the problems with JDBC
> logging and Oracle playing nicely together.  I'm still intermittently
> getting the recursive exception about:
>=20
> "JdbcLog cannot log SQLException due to recursive log errors!
> java.sql.SQLException: We are already in the process of making 1
> connections and the number of simultaneous builds has been throttled to
> 0"
>=20
> Thing is, it IS logging to the database.  I get a record inserted in the
> Sessions table with the initial information, only the incoming AND
> outgoing seqnums are always set to 1.  I then enter an order in the test
> client the one vendor I'm testing with provided us to buy 500 shares of
> IBM.  The messages come through...all the messages are stored in the
> database, I let the heartbeat run for a while, then I shut down the
> engine and start it back up again.
>=20
> Immediately I begin receiving:
>=20
> "Serious Error:  Message sequence number: 1 is less than expected
> sequence number: 28"
>=20
> And then the engine continues to logout and login, each time, internally
> incrementing that message count, but never once updating the session
> entry in the SESSIONS table in the database with the newly incremented
> outgoing sequence number.  Currently, the engine is up to 22 and the
> SESSIONS table record for this engine instance is still showing 1 for
> both the incoming and outgoing sequence numbers...but internally, it's
> obviously incrementing the count.
>=20
> Nowhere on the console are there any database exceptions relating to
> having a problem inserting into the SESSIONS table or updating that
> table.  Every so often I'll get something like this:
>=20
> "2007-05-21 14:44:47,361 DEBUG [HouseKeeper] proxool.ConnectionPool
> (ConnectionPool.java:447) - 000079 (00/01/00) - #0001 removed because it
> has been active for too long.
> 2007-05-21 14:44:47,377 WARN  [HouseKeeper] proxool.HouseKeeper
> (HouseKeeper.java:149) - #0001 was active for 11562 milliseconds and has
> been removed automaticaly. The Thread responsible was named 'QFJ Timer',
> but the last SQL it performed is unknown because the trace property is
> not enabled.
> JdbcLog cannot log SQLException due to recursive log errors!
> java.sql.SQLException: We are already in the process of making 1
> connections and the number of simultaneous builds has been throttled to
> 0"
>=20
> I don't know what to do about these exceptions or the one I mentioned
> above, but messages and the message log tables both seem to be capturing
> the messages fine and running the engine without shutting it down and
> restarting for a long period of time, quite a few days in a row...the
> engine seems to work as does the logging without a hitch...but the
> minute we shut it down, that internal incrementing is lost and the
> sessions table sequence numbers are always initiated with 1 and we get
> stuck in a sequence reset loop until the engine increments to a number
> the endpoint session can handle, then a successful reset request is
> handled and things carry on normally.
>=20
> Any thoughts?  Should I just give up on JDBC logging in conjunction with
> an Oracle database?=20=20
>=20
> Is there a better way to handle fail-over?  The reason the IT guys here
> wanted database logging was so if the FIX engine server went down, we
> could fire up the fail-over server and it would be able to look at the
> database logs (which reside on a totally different server) and pick up
> seamlessly where the initial instance left off.  If we use just basic
> file logging and the entire FIX server crashes and we can't get to the
> log files, what do we do?  Am I completely confused in thinking that the
> database logging approach would be better for my scenario or are there
> alternatives?
>=20
> Thanks so much!
>=20
> robert l. brueckmann
> vice president
> merlin securities
> 712 fifth avenue
> new york, ny 10019
> p: 212.822.4821
> f: 212.822.4820
>=20
>=20
>=20
> Merlin Securities - #1 Prime Broker North America, #1 Prime Broker Single=
 Strategy Funds, #1 Prime Broker Funds Under $100M - Global Custodian 2007
>=20
>=20
> From: qui...@li...
> [mailto:qui...@li...] On Behalf Of
> Steve Bate
> Sent: Sunday, May 20, 2007 6:01 PM
> To: qui...@li...
> Subject: Re: [Quickfixj-users] outgoing seqnum
>=20
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>> We had to stop our engine mid-day today to do some database=20
>> maintenance.  ...
>>
>> Why is the sessions table not being updated each time the=20
>> outgoing seq num is being incremented by the engine?  Why is=20
>> this not being maintained?=20=20
>=20
> I don't know based on the information you've provided. You
> say you were doing database maintenance. If the engine's
> database connection was bad it might explain the problem.
> You'd see information in the log file and/or event log
> table if this was the cause. This could happen if the
> database was restarted but the FIX engines were not.
>=20
> Was the outgoing sequence number =3D 1 in the sessions table?
> This information won't help to explain the lack of updates,
> but it will tell me if the session was reset or whether the
> engine just couldn't read the information from the database.
>=20
>> If we need to stop and start the=20
>> engine in the middle of the day, we obviously don't want this=20
>> to happen every time.  What can we do?=20=20
>=20
> I'd be surprised is this was an issue directly related to
> stopping and starting the engine midday. Set up a client
> and server test environment using a JDBC message store and
> try stopping and starting. It should work. I don't personally
> use the JDBC message store, but there are tests for the
> implementation and others use it. There are no bug reports
> related to this type of behavior.
>=20
>> In this instance, we=20
>> contacted the endpoint and they reset the seq num on their=20
>> end so we could catch up to a reasonable number and then we=20
>> captured all missed messages. Is there a bug in the jdbc logging?=20=20
>=20
> Based on the information so far, I'd guess not. Check the log
> files for additional clues.=20
>=20
>> Any thoughts on what we can do to better handle a mid-day=20
>> shutdown and restart that would allows us to pick back up=20
>> where we left off with our endpoint?
>=20
> You just be able to restart. If you have any other information,
> please let us know.
>=20
> Steve
--=20
Joerg Thoennes
http://www.macd.com                 Tel.: +49 (0)241 44597-24
Macdonald Associates GmbH           Gesch=E4ftsf=FChrer: Roger Macdonald
Lothringer Str. 52, D-52070 Aachen  Amtsgericht Aachen, HRB 8151, Ust.-Id D=
E813021663

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Quickfixj-users mailing list
Qui...@li...
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
=20

--------------------------------------------------------

This message contains information from Merlin Securities, LLC, or from one =
of its affiliates, that may be confidential and privileged. If you are not =
an intended recipient, please refrain from any disclosure, copying, distrib=
ution or use of this information and note that such actions are prohibited.=
 If you have received this transmission in error, please notify the sender =
immediately by telephone or by replying to this transmission.
=20
Merlin Securities, LLC is a registered broker-dealer. Services offered thro=
ugh Merlin Securities, LLC are not insured by the FDIC or any other Federal=
 Government Agency, are not deposits of or guaranteed by Merlin Securities,=
 LLC and may lose value. Nothing in this communication shall constitute a s=
olicitation or recommendation to buy or sell a particular security.

[Quickfixj-users] Initiator.stop leaves file handles open when using FileStore

[Robert M. <rob...@eu...>]

We are finding that our application is running out of file handles. The
problem occurs because our auto recovery code having detected an
acceptor initiated logout, attempts to re-start the connection by
constructing a new SocketIinitiator (having stopped the old one first)
and then calling start on it.

=20

The problem seems to be that the FileStore maintained by the Session
object does not close its file handles (except when a reset is done at
which point it promptly opens the files again). The Socket initiator
creates a new Session object and so we find that with each disconnect we
are using up more file handles. The problem also seems to occur with the
event and message logs.

=20

Has anyone had to handle this situation before and found a way around
it?

=20

Thanks in advance

=20

=20


Eurobase International Limited and its subsidiaries (Eurobase) are =
unable to exercise control over the content of information in E-Mails. =
Any views and opinions expressed may be personal to the sender and are =
not necessarily those of Eurobase. Eurobase will not enter into any =
contractual obligations in respect of any part of its business in any =
E-mail.=20

Privileged / confidential information may be contained in this message =
and /or any attachments. This E-mail is intended for the use of the =
addressee(s) only and may contain confidential information. If you are =
not the / an intended recipient, you are hereby notified that any use or =
dissemination of this communication is strictly prohibited.  If you =
receive this transmission in error, please notify us immediately, and =
then delete this E-mail.=20

Neither the sender nor Eurobase accepts any liability whatsoever for any =
defects of any kind either in or arising from this E-mail transmission. =
E-Mail transmission cannot be guaranteed to be secure or error-free, as =
messages can be intercepted, lost, corrupted, destroyed, contain =
viruses, or arrive late or incomplete. Eurobase does not accept any =
responsibility for viruses and it is your responsibility to scan any =
attachments.

Eurobase Systems Limited is the main trading company in the Eurobase =
International Group; registered in England and Wales as company number =
02251162; registered address: Essex House, 2 County Place, Chelmsford, =
Essex CM2 0RE, UK.

Re: [Quickfixj-users] outgoing seqnum

[Joerg T. <Joe...@ma...>]

Hi Robert,

we currently use JdbcStore, but not JdbcLog (you sometimes call things "logging" where I would
expect the MessageStore; is it possible to you mix logging and saving state?).

In our test environment, all works well with JDBC access to PostgreSQL. Our customer uses Oracle 8
with OCI, there are currently installing our new version and maybe I get some feedback soon.

If I remember right, you are using Oracle 9? Which JDBC driver -- "thin:" or "oci:"? Please give me
some more details here.

For the moment, I would recommend to replace the JdbcLog by Log4J logging. This is flexible enough
for most purpose. For failover, it is important to keep the JdbcStore.

In general, I am very interested in your experiences because our customer also uses Oracle.

Cheers, Jörg

On 05/21/07 20:55, Robert Brueckmann wrote:
> Thanks for the reply.  I did as you said and set up a test
> environment...I am using JDBC logging against Oracle.  I'm the one that
> had a barrage of emails a few weeks ago about the problems with JDBC
> logging and Oracle playing nicely together.  I'm still intermittently
> getting the recursive exception about:
> 
> "JdbcLog cannot log SQLException due to recursive log errors!
> java.sql.SQLException: We are already in the process of making 1
> connections and the number of simultaneous builds has been throttled to
> 0"
> 
> Thing is, it IS logging to the database.  I get a record inserted in the
> Sessions table with the initial information, only the incoming AND
> outgoing seqnums are always set to 1.  I then enter an order in the test
> client the one vendor I'm testing with provided us to buy 500 shares of
> IBM.  The messages come through...all the messages are stored in the
> database, I let the heartbeat run for a while, then I shut down the
> engine and start it back up again.
> 
> Immediately I begin receiving:
> 
> "Serious Error:  Message sequence number: 1 is less than expected
> sequence number: 28"
> 
> And then the engine continues to logout and login, each time, internally
> incrementing that message count, but never once updating the session
> entry in the SESSIONS table in the database with the newly incremented
> outgoing sequence number.  Currently, the engine is up to 22 and the
> SESSIONS table record for this engine instance is still showing 1 for
> both the incoming and outgoing sequence numbers...but internally, it's
> obviously incrementing the count.
> 
> Nowhere on the console are there any database exceptions relating to
> having a problem inserting into the SESSIONS table or updating that
> table.  Every so often I'll get something like this:
> 
> "2007-05-21 14:44:47,361 DEBUG [HouseKeeper] proxool.ConnectionPool
> (ConnectionPool.java:447) - 000079 (00/01/00) - #0001 removed because it
> has been active for too long.
> 2007-05-21 14:44:47,377 WARN  [HouseKeeper] proxool.HouseKeeper
> (HouseKeeper.java:149) - #0001 was active for 11562 milliseconds and has
> been removed automaticaly. The Thread responsible was named 'QFJ Timer',
> but the last SQL it performed is unknown because the trace property is
> not enabled.
> JdbcLog cannot log SQLException due to recursive log errors!
> java.sql.SQLException: We are already in the process of making 1
> connections and the number of simultaneous builds has been throttled to
> 0"
> 
> I don't know what to do about these exceptions or the one I mentioned
> above, but messages and the message log tables both seem to be capturing
> the messages fine and running the engine without shutting it down and
> restarting for a long period of time, quite a few days in a row...the
> engine seems to work as does the logging without a hitch...but the
> minute we shut it down, that internal incrementing is lost and the
> sessions table sequence numbers are always initiated with 1 and we get
> stuck in a sequence reset loop until the engine increments to a number
> the endpoint session can handle, then a successful reset request is
> handled and things carry on normally.
> 
> Any thoughts?  Should I just give up on JDBC logging in conjunction with
> an Oracle database?  
> 
> Is there a better way to handle fail-over?  The reason the IT guys here
> wanted database logging was so if the FIX engine server went down, we
> could fire up the fail-over server and it would be able to look at the
> database logs (which reside on a totally different server) and pick up
> seamlessly where the initial instance left off.  If we use just basic
> file logging and the entire FIX server crashes and we can't get to the
> log files, what do we do?  Am I completely confused in thinking that the
> database logging approach would be better for my scenario or are there
> alternatives?
> 
> Thanks so much!
> 
> robert l. brueckmann
> vice president
> merlin securities
> 712 fifth avenue
> new york, ny 10019
> p: 212.822.4821
> f: 212.822.4820
> 
> 
> 
> Merlin Securities - #1 Prime Broker North America, #1 Prime Broker Single Strategy Funds, #1 Prime Broker Funds Under $100M - Global Custodian 2007
> 
> 
> From: qui...@li...
> [mailto:qui...@li...] On Behalf Of
> Steve Bate
> Sent: Sunday, May 20, 2007 6:01 PM
> To: qui...@li...
> Subject: Re: [Quickfixj-users] outgoing seqnum
> 
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>> We had to stop our engine mid-day today to do some database 
>> maintenance.  ...
>>
>> Why is the sessions table not being updated each time the 
>> outgoing seq num is being incremented by the engine?  Why is 
>> this not being maintained?  
> 
> I don't know based on the information you've provided. You
> say you were doing database maintenance. If the engine's
> database connection was bad it might explain the problem.
> You'd see information in the log file and/or event log
> table if this was the cause. This could happen if the
> database was restarted but the FIX engines were not.
> 
> Was the outgoing sequence number = 1 in the sessions table?
> This information won't help to explain the lack of updates,
> but it will tell me if the session was reset or whether the
> engine just couldn't read the information from the database.
> 
>> If we need to stop and start the 
>> engine in the middle of the day, we obviously don't want this 
>> to happen every time.  What can we do?  
> 
> I'd be surprised is this was an issue directly related to
> stopping and starting the engine midday. Set up a client
> and server test environment using a JDBC message store and
> try stopping and starting. It should work. I don't personally
> use the JDBC message store, but there are tests for the
> implementation and others use it. There are no bug reports
> related to this type of behavior.
> 
>> In this instance, we 
>> contacted the endpoint and they reset the seq num on their 
>> end so we could catch up to a reasonable number and then we 
>> captured all missed messages. Is there a bug in the jdbc logging?  
> 
> Based on the information so far, I'd guess not. Check the log
> files for additional clues. 
> 
>> Any thoughts on what we can do to better handle a mid-day 
>> shutdown and restart that would allows us to pick back up 
>> where we left off with our endpoint?
> 
> You just be able to restart. If you have any other information,
> please let us know.
> 
> Steve
-- 
Joerg Thoennes
http://www.macd.com                 Tel.: +49 (0)241 44597-24
Macdonald Associates GmbH           Geschäftsführer: Roger Macdonald
Lothringer Str. 52, D-52070 Aachen  Amtsgericht Aachen, HRB 8151, Ust.-Id DE813021663

[Quickfixj-users] Configuration of weekly session

[Yong-Jhen H. <wi...@ic...>]

Hi all!

I work as an engineer in a local trading software company in Taiwan.
In our solution for outbound trading, we use QuickFIX/J for
communication with FIX gateways. It works fine for us most of the time,
thanks for the great work, developers!

I find some problems with weekly session in QuickFIX/J in our test
environment these days though:

1. I can't confiure a weekly session using SessionSettings with
an InputStream object (in my case, a FileInputStream object),
because SessionSettings doesn't work well with multibyte characters,
and the name of weekdays in our locale (zh-TW) is in multibyte
characters. Therefore I change the code to use InputStreamReader to read
characters instead.

2. When parsing name of weekdays in class DayConverter, only the leading
two characters are used for comparison, maybe for easier configuration.
But the leading two characters in our locale is all the same for the
7 weekdays, thus it always match the first one (Sunday).
I would suggest not just using the leading two characters for
comparison, it will still work fine for people using only two characters
to confiure.

3. Weekdays (StartDay, EndDay) is always in UTC, even when I set
TimeZone to our local one. For example:

TimeZone=Asia/Taipei
StartTime=06:45:00
EndTime=06:44:59

If I set both StartDay and EndDay to Wednesday (the equivalent,
localized one, in multibyte characters, of course ;)
it would be a weekly session of from Wednesday 22:45:00 UTC
to Wednesday 22:44:59 UTC,
instead of from Tuesday 22:45:00 UTC to Tuesdasy 22:44:59 UTC.
(Note: Wednesday 06:45:00 in Taipei is Tuesday 22:45:00 UTC)


I made some changes to solve the 3 problems above in my own way,
the patch is attached with this mail.
Hope it can be adoped by our developers or help somebody else ;)

Regards,
yongjhen

--
Yong-Jhen Hong (wi...@ic...)
Senior Software Engineer
Financial Solution Dept.
ICE Technology Corp.
Taiwan

Re: [Quickfixj-users] A deadlock while using the SSL

[Alexander G. <ago...@ac...>]

Hi Steve,

Thanks for the answer. For the moment I've removed the synchronization from 
the get/setDataDictionary methods and restarted the application. Should 
there be any other synchronization issues I'll let you know.

When do you expect the next release (including the sync redesign) to be 
available?

Thank you,
Alex.

----- Original Message ----- 
From: "Steve Bate" <st...@te...>
To: <qui...@li...>
Sent: Wednesday, May 23, 2007 5:05 PM
Subject: Re: [Quickfixj-users] A deadlock while using the SSL


>
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>> As far as I can see, there is a high probability of having similar
>> deadlocks in other parts of the filter chain when using the SSLFilter. 
>> The
>> problem can be solved by changing the synchronization model for the
>> quickfix.Session class. I understand though that this task is rather
>> complex and time consuming. Do you see other ways of solving this? Please
>> advise, I am willing to contribute if necessary.
>
> Hi,
>
> Thanks for the detailed information. I've actually done some significant
> redesign of the synchronization strategies for the next release. Like you
> said, it makes verification of thread correctness more difficult but the
> current approach is too prone to deadlocks. The basic strategy for the
> sync redesign is to be sure there are no active quickfix.Session locks
> when an application callback is invoked or the communication layer (MINA)
> is accessed. I can test this with the acceptance tests by evaluating
> active locks during each quickfix.Application callback and ensuring the
> session is not locked. It's a little trickers for the MINA locks, but I
> think it's clean. All tests are passing, but unfortunately bugs related to
> locking can be nondeterministic.
>
> I also don't allow setting the DataDictionary on a quickfix.Session. The
> data dictionary is typically supplied when the Session is created. The
> ability to modify the DD reference in the session meant that access had to
> be synchronized or the reference had to be volatile. I've now made the
> reference final so avoid the related synchronization. This means the
> setDataDictionary() method is now unsupported (unlike in the QF JNI). If
> that's a problem for anyone, I'll reconsider the current approach. I
> can also add a dedicated lock only for the data dictionary reference.
>
> Steve
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users 

Re: [Quickfixj-users] A deadlock while using the SSL

[Steve B. <st...@te...>]

> As far as I can see, there is a high probability of having similar
> deadlocks in other parts of the filter chain when using the SSLFilter. The
> problem can be solved by changing the synchronization model for the
> quickfix.Session class. I understand though that this task is rather
> complex and time consuming. Do you see other ways of solving this? Please
> advise, I am willing to contribute if necessary.

Hi,

Thanks for the detailed information. I've actually done some significant
redesign of the synchronization strategies for the next release. Like you
said, it makes verification of thread correctness more difficult but the
current approach is too prone to deadlocks. The basic strategy for the
sync redesign is to be sure there are no active quickfix.Session locks
when an application callback is invoked or the communication layer (MINA)
is accessed. I can test this with the acceptance tests by evaluating
active locks during each quickfix.Application callback and ensuring the
session is not locked. It's a little trickers for the MINA locks, but I
think it's clean. All tests are passing, but unfortunately bugs related to
locking can be nondeterministic.

I also don't allow setting the DataDictionary on a quickfix.Session. The
data dictionary is typically supplied when the Session is created. The
ability to modify the DD reference in the session meant that access had to
be synchronized or the reference had to be volatile. I've now made the
reference final so avoid the related synchronization. This means the
setDataDictionary() method is now unsupported (unlike in the QF JNI). If
that's a problem for anyone, I'll reconsider the current approach. I
can also add a dedicated lock only for the data dictionary reference.

Steve

[Quickfixj-users] A deadlock while using the SSL

[Alexander G. <ago...@ac...>]

Hello all!

I am using the quickfix/j 1.1.0 to communicate with the FIX counterparty =
using the SSL connection. And the application periodically freezes which =
makes it impossible to use it in the production. I investigated the code =
and the problem lies in the implementation of the code synchronization =
for the org.apache.mina.filter.SSLFilter and the quickfix.Session. The =
code in these classes is synchronised by the current SSLHandler instance =
and the current Session instance respectively. And since the filter =
chain goes from one filter to another without releasing the lock, there =
is a high probability for an application to run into a deadlock.=20

That's what happened with my application. Below are the stack traces of =
the reading and the writing threads that caused the deadlock with my =
comments:

// 1. The reading thread:=20

quickfix.Session.getDataDictionary(Session.java:1649) <--- attempt to =
acquire the Session lock which is held by the writing thread [deadlock]
quickfix.mina.AbstractIoHandler.messageReceived(AbstractIoHandler.java:11=
1)
org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageRe=
ceived(AbstractIoFilterChain.java:705)
org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageRecei=
ved(AbstractIoFilterChain.java:362)
org.apache.mina.common.support.AbstractIoFilterChain.access$5(AbstractIoF=
ilterChain.java:356)
org.apache.mina.common.support.AbstractIoFilterChain$1.messageReceived(Ab=
stractIoFilterChain.java:802)
org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(Si=
mpleProtocolDecoderOutput.java:60)
org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(Protocol=
CodecFilter.java:184)
org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageRecei=
ved(AbstractIoFilterChain.java:362)
org.apache.mina.common.support.AbstractIoFilterChain.access$5(AbstractIoF=
ilterChain.java:356)
org.apache.mina.common.support.AbstractIoFilterChain$1.messageReceived(Ab=
stractIoFilterChain.java:802)
org.apache.mina.filter.SSLFilter.handleAppDataRead(SSLFilter.java:627)
org.apache.mina.filter.SSLFilter.handleSSLData(SSLFilter.java:605)
org.apache.mina.filter.SSLFilter.messageReceived(SSLFilter.java:395) =
<--- acquisition of the SSLHandler lock

...

// 2. The writing thread:

org.apache.mina.filter.SSLFilter.filterWrite(SSLFilter.java:456) <--- =
attempt to acquire the SSLHandler lock which is held by the reading =
thread [deadlock]
org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWr=
ite(AbstractIoFilterChain.java:447)
org.apache.mina.common.support.AbstractIoFilterChain.access$8(AbstractIoF=
ilterChain.java:440)
org.apache.mina.common.support.AbstractIoFilterChain$1.filterWrite(Abstra=
ctIoFilterChain.java:826)
org.apache.mina.filter.codec.ProtocolCodecFilter$ProtocolEncoderOutputImp=
l.doFlush(ProtocolCodecFilter.java:403)
org.apache.mina.filter.codec.support.SimpleProtocolEncoderOutput.flush(Si=
mpleProtocolEncoderOutput.java:110)
org.apache.mina.filter.codec.ProtocolCodecFilter.filterWrite(ProtocolCode=
cFilter.java:220)
org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWr=
ite(AbstractIoFilterChain.java:447)
org.apache.mina.common.support.AbstractIoFilterChain.access$8(AbstractIoF=
ilterChain.java:440)
org.apache.mina.common.support.AbstractIoFilterChain$1.filterWrite(Abstra=
ctIoFilterChain.java:826)
org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.filterWri=
te(AbstractIoFilterChain.java:729)
org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWr=
ite(AbstractIoFilterChain.java:447)
org.apache.mina.common.support.AbstractIoFilterChain.fireFilterWrite(Abst=
ractIoFilterChain.java:437)
org.apache.mina.transport.socket.nio.SocketSessionImpl.write0(SocketSessi=
onImpl.java:191)
org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java:145=
)
org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java:131=
)
quickfix.mina.IoSessionResponder.send(IoSessionResponder.java:38)
quickfix.Session.send(Session.java:1627)
quickfix.Session.sendRaw(Session.java:1576) <--- acquisition of the =
Session lock
...

As far as I can see, there is a high probability of having similar =
deadlocks in other parts of the filter chain when using the SSLFilter. =
The problem can be solved by changing the synchronization model for the =
quickfix.Session class. I understand though that this task is rather =
complex and time consuming. Do you see other ways of solving this? =
Please advise, I am willing to contribute if necessary.

Thank you,
Alexander Gorbachev.

[Quickfixj-users] CN=Alvin Wang/O=FFTW is out of the office.

[Alvin W. <AW...@FF...>]

I will be out of the office starting  05/23/2007 and will not return until
05/24/2007.




                                                                *******************************************************************************
This e-mail message is intended solely for the use of the addressee.
The message may contain information that is privileged and 
confidential.
Disclosure to anyone other than the intended recipient is prohibited.
If you are not the intended recipient, please do not disseminate,
distribute or copy this communication, by e-mail or otherwise. Instead,
please notify us immediately by return e-mail(including the original
message with your reply) and then delete and discard all copies of the
message. We have taken precautions to minimize the risk of  transmitting
software viruses but nevertheless advise you to carry out your own
virus checks on any attachment to this message.  We accept
no liability for any loss or damage caused by software viruses.
*******************************************************************************

Re: [Quickfixj-users] Problems in using the NoPartyIDs Block

[gore <gre...@qu...>]

Toli, thanks for your answer.

The counter is useless. i figured that out yesterday, because my initiator
set the NoPartyIDs to 1 anyway in the message. I get the 'Conditionally
required field missing' error anyway. When i looked at the source code of
the NewOrderSingle class the embedded NoPartyIDs class uses array for the
field order with 5 fields, instead of just PartyID, PartyIDSource, PartyRole
and its inner PartySubIDs Block. The last field in the array is set to 0.
But there is no Quickfix field 0. Maybe that's the problem?

Any ideas?


toli wrote:
> 
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> Gregor,
> 
> If i'm not mistaken, you are only inserting 1 group instead of 3.
> You are inserting 3 fields that all belong to the same repeating
> group, but only one group.
> 
> so try it with NoPartyID to 1.
> 
> also, if i'm not mistaken, you don't actually need to set the
> NoPartyID field at all - the underlying QFJ code will set the totall #
> of repeating groups for you. You only need to set it if the number of
> groups is 0.
> 
> see this writeup for reference:
> http://www.quickfixj.org/quickfixj/usermanual/usage/repeating_groups.html
> 
> On 5/22/07, gore <gre...@qu...> wrote:
>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>
>> I construct my parties block for a NewOrderSingle the following way:
>>
>> ******** CODE ********
>>
>> int partyIDcnt = 0; // counter for the NoPartyIDs tag, incremented for
>> every
>> field added PartyID
>> NewOrderSingle.NoPartyIDs partiesBlock = new NewOrderSingle.NoPartyIDs();
>> //
>> the empty parties block
>>
>> if (!partyID.equals(""))
>> {
>>         partiesBlock.set(partyID);
>>         partyIDcnt++;
>> }
>> if (!partyIDSource.toString().equals(""))
>> {
>>         partiesBlock.set(partyIDSource);
>>         partyIDcnt++;
>> }
>> if (!partyRole.toString().equals(""))
>> {
>>         partiesBlock.set(partyRole);
>>         partyIDcnt++;
>> }
>>
>> newOrderSingle.addGroup(partiesBlock);
>> NoPartyIDs noPartyIDs = new NoPartyIDs(partyIDcnt);
>> newOrderSingle.set(noPartyIDs);
>>
>> ******** /CODE ********
>>
>> I get a 'Message Rejected: Conditionally Required Field Missing' error
>> from
>> my Acceptor. I don't see any field missing, as all fields in the parties
>> block are no compulsory fields. Any ideas or experience with creating the
>> parties block? Is there any code-source for the right implementation of a
>> parties block and the NoPartyIDs to a NewOrderSingle?
>>
>> My if conditions are ok, as all of the three values are received by the
>> Acceptor in the message.
>>
>> Thanks for any support!
>> --
>> View this message in context:
>> http://www.nabble.com/Problems-in-using-the-NoPartyIDs-Block-tf3796035.html#a10736855
>> Sent from the QuickFIX/J mailing list archive at Nabble.com.
>>
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by DB2 Express
>> Download DB2 Express C - the FREE version of DB2 express and take
>> control of your XML. No limits. Just data. Click to get it now.
>> http://sourceforge.net/powerbar/db2/
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>
> 
> 
> -- 
> Toli Kuznets
> http://www.marketcetera.com: Open-Source Trading Platform
> download.run.trade.
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
> 
> 

-- 
View this message in context: http://www.nabble.com/Problems-in-using-the-NoPartyIDs-Block-tf3796035.html#a10759162
Sent from the QuickFIX/J mailing list archive at Nabble.com.

Re: [Quickfixj-users] Problems in using the NoPartyIDs Block

[Toli K. <to...@ma...>]

Gregor,

If i'm not mistaken, you are only inserting 1 group instead of 3.
You are inserting 3 fields that all belong to the same repeating
group, but only one group.

so try it with NoPartyID to 1.

also, if i'm not mistaken, you don't actually need to set the
NoPartyID field at all - the underlying QFJ code will set the totall #
of repeating groups for you. You only need to set it if the number of
groups is 0.

see this writeup for reference:
http://www.quickfixj.org/quickfixj/usermanual/usage/repeating_groups.html

On 5/22/07, gore <gre...@qu...> wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
> I construct my parties block for a NewOrderSingle the following way:
>
> ******** CODE ********
>
> int partyIDcnt = 0; // counter for the NoPartyIDs tag, incremented for every
> field added PartyID
> NewOrderSingle.NoPartyIDs partiesBlock = new NewOrderSingle.NoPartyIDs(); //
> the empty parties block
>
> if (!partyID.equals(""))
> {
>         partiesBlock.set(partyID);
>         partyIDcnt++;
> }
> if (!partyIDSource.toString().equals(""))
> {
>         partiesBlock.set(partyIDSource);
>         partyIDcnt++;
> }
> if (!partyRole.toString().equals(""))
> {
>         partiesBlock.set(partyRole);
>         partyIDcnt++;
> }
>
> newOrderSingle.addGroup(partiesBlock);
> NoPartyIDs noPartyIDs = new NoPartyIDs(partyIDcnt);
> newOrderSingle.set(noPartyIDs);
>
> ******** /CODE ********
>
> I get a 'Message Rejected: Conditionally Required Field Missing' error from
> my Acceptor. I don't see any field missing, as all fields in the parties
> block are no compulsory fields. Any ideas or experience with creating the
> parties block? Is there any code-source for the right implementation of a
> parties block and the NoPartyIDs to a NewOrderSingle?
>
> My if conditions are ok, as all of the three values are received by the
> Acceptor in the message.
>
> Thanks for any support!
> --
> View this message in context: http://www.nabble.com/Problems-in-using-the-NoPartyIDs-Block-tf3796035.html#a10736855
> Sent from the QuickFIX/J mailing list archive at Nabble.com.
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>


-- 
Toli Kuznets
http://www.marketcetera.com: Open-Source Trading Platform
download.run.trade.

Re: [Quickfixj-users] outgoing seqnum

[Steve B. <st...@te...>]

> ... the incoming AND outgoing seqnums are always set to 1.

Hi Robert,

It sounds like you can't write to the session table for some
reason. It might be related to the Proxool exceptions you
described. That would explain the apparent reset when you
restart. It's not really a reset, but the sequence numbers
were never incremented.

I don't have access to Oracle so I can't try it out myself.
A quick Google search didn't find any reported problems
that sound similar. You can try modifying the JdbcUtil.
getDataSource() method to return an instance of an
OracleDataSource instead of using Proxool to wrap
and pool Oracle connections.

You are right that using JDBC for the quickfix.MessageStore
is a more flexible way to support failover. I'd still
recommend using files for logging (quickfix.Log) because
they it's much faster than JDBC.

I will modify the JdbcLogFactory and JdbcStoreFactory in
the next version of QFJ so it will be easier to set up
your own data source (like I'm suggesting here).

Let us know if you learn some more about the problem.

Thanks,

Steve

[Quickfixj-users] Problems in using the NoPartyIDs Block

[gore <gre...@qu...>]

I construct my parties block for a NewOrderSingle the following way:

******** CODE ********

int partyIDcnt = 0; // counter for the NoPartyIDs tag, incremented for every
field added PartyID
NewOrderSingle.NoPartyIDs partiesBlock = new NewOrderSingle.NoPartyIDs(); //
the empty parties block

if (!partyID.equals(""))
{
	partiesBlock.set(partyID);
	partyIDcnt++;
}
if (!partyIDSource.toString().equals(""))
{
	partiesBlock.set(partyIDSource);
	partyIDcnt++;
}		
if (!partyRole.toString().equals(""))
{
	partiesBlock.set(partyRole);
	partyIDcnt++;
}

newOrderSingle.addGroup(partiesBlock);
NoPartyIDs noPartyIDs = new NoPartyIDs(partyIDcnt);
newOrderSingle.set(noPartyIDs);

******** /CODE ********

I get a 'Message Rejected: Conditionally Required Field Missing' error from
my Acceptor. I don't see any field missing, as all fields in the parties
block are no compulsory fields. Any ideas or experience with creating the
parties block? Is there any code-source for the right implementation of a
parties block and the NoPartyIDs to a NewOrderSingle?

My if conditions are ok, as all of the three values are received by the
Acceptor in the message.

Thanks for any support!
-- 
View this message in context: http://www.nabble.com/Problems-in-using-the-NoPartyIDs-Block-tf3796035.html#a10736855
Sent from the QuickFIX/J mailing list archive at Nabble.com.

Re: [Quickfixj-users] outgoing seqnum

[Robert B. <rbr...@me...>]

Hey Steve,

Thanks for the reply.  I did as you said and set up a test
environment...I am using JDBC logging against Oracle.  I'm the one that
had a barrage of emails a few weeks ago about the problems with JDBC
logging and Oracle playing nicely together.  I'm still intermittently
getting the recursive exception about:

"JdbcLog cannot log SQLException due to recursive log errors!
java.sql.SQLException: We are already in the process of making 1
connections and the number of simultaneous builds has been throttled to
0"

Thing is, it IS logging to the database.  I get a record inserted in the
Sessions table with the initial information, only the incoming AND
outgoing seqnums are always set to 1.  I then enter an order in the test
client the one vendor I'm testing with provided us to buy 500 shares of
IBM.  The messages come through...all the messages are stored in the
database, I let the heartbeat run for a while, then I shut down the
engine and start it back up again.

Immediately I begin receiving:

"Serious Error:  Message sequence number: 1 is less than expected
sequence number: 28"

And then the engine continues to logout and login, each time, internally
incrementing that message count, but never once updating the session
entry in the SESSIONS table in the database with the newly incremented
outgoing sequence number.  Currently, the engine is up to 22 and the
SESSIONS table record for this engine instance is still showing 1 for
both the incoming and outgoing sequence numbers...but internally, it's
obviously incrementing the count.

Nowhere on the console are there any database exceptions relating to
having a problem inserting into the SESSIONS table or updating that
table.  Every so often I'll get something like this:

"2007-05-21 14:44:47,361 DEBUG [HouseKeeper] proxool.ConnectionPool
(ConnectionPool.java:447) - 000079 (00/01/00) - #0001 removed because it
has been active for too long.
2007-05-21 14:44:47,377 WARN  [HouseKeeper] proxool.HouseKeeper
(HouseKeeper.java:149) - #0001 was active for 11562 milliseconds and has
been removed automaticaly. The Thread responsible was named 'QFJ Timer',
but the last SQL it performed is unknown because the trace property is
not enabled.
JdbcLog cannot log SQLException due to recursive log errors!
java.sql.SQLException: We are already in the process of making 1
connections and the number of simultaneous builds has been throttled to
0"

I don't know what to do about these exceptions or the one I mentioned
above, but messages and the message log tables both seem to be capturing
the messages fine and running the engine without shutting it down and
restarting for a long period of time, quite a few days in a row...the
engine seems to work as does the logging without a hitch...but the
minute we shut it down, that internal incrementing is lost and the
sessions table sequence numbers are always initiated with 1 and we get
stuck in a sequence reset loop until the engine increments to a number
the endpoint session can handle, then a successful reset request is
handled and things carry on normally.

Any thoughts?  Should I just give up on JDBC logging in conjunction with
an Oracle database?=20=20

Is there a better way to handle fail-over?  The reason the IT guys here
wanted database logging was so if the FIX engine server went down, we
could fire up the fail-over server and it would be able to look at the
database logs (which reside on a totally different server) and pick up
seamlessly where the initial instance left off.  If we use just basic
file logging and the entire FIX server crashes and we can't get to the
log files, what do we do?  Am I completely confused in thinking that the
database logging approach would be better for my scenario or are there
alternatives?

Thanks so much!

robert l. brueckmann
vice president
merlin securities
712 fifth avenue
new york, ny 10019
p: 212.822.4821
f: 212.822.4820



Merlin Securities - #1 Prime Broker North America, #1 Prime Broker Single S=
trategy Funds, #1 Prime Broker Funds Under $100M - Global Custodian 2007


From: qui...@li...
[mailto:qui...@li...] On Behalf Of
Steve Bate
Sent: Sunday, May 20, 2007 6:01 PM
To: qui...@li...
Subject: Re: [Quickfixj-users] outgoing seqnum

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
> We had to stop our engine mid-day today to do some database=20
> maintenance.  ...
>
> Why is the sessions table not being updated each time the=20
> outgoing seq num is being incremented by the engine?  Why is=20
> this not being maintained?=20=20

I don't know based on the information you've provided. You
say you were doing database maintenance. If the engine's
database connection was bad it might explain the problem.
You'd see information in the log file and/or event log
table if this was the cause. This could happen if the
database was restarted but the FIX engines were not.

Was the outgoing sequence number =3D 1 in the sessions table?
This information won't help to explain the lack of updates,
but it will tell me if the session was reset or whether the
engine just couldn't read the information from the database.

> If we need to stop and start the=20
> engine in the middle of the day, we obviously don't want this=20
> to happen every time.  What can we do?=20=20

I'd be surprised is this was an issue directly related to
stopping and starting the engine midday. Set up a client
and server test environment using a JDBC message store and
try stopping and starting. It should work. I don't personally
use the JDBC message store, but there are tests for the
implementation and others use it. There are no bug reports
related to this type of behavior.

> In this instance, we=20
> contacted the endpoint and they reset the seq num on their=20
> end so we could catch up to a reasonable number and then we=20
> captured all missed messages. Is there a bug in the jdbc logging?=20=20

Based on the information so far, I'd guess not. Check the log
files for additional clues.=20

> Any thoughts on what we can do to better handle a mid-day=20
> shutdown and restart that would allows us to pick back up=20
> where we left off with our endpoint?

You just be able to restart. If you have any other information,
please let us know.

Steve



------------------------------------------------------------------------
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Quickfixj-users mailing list
Qui...@li...
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
=20

--------------------------------------------------------

This message contains information from Merlin Securities, LLC, or from one =
of its affiliates, that may be confidential and privileged. If you are not =
an intended recipient, please refrain from any disclosure, copying, distrib=
ution or use of this information and note that such actions are prohibited.=
 If you have received this transmission in error, please notify the sender =
immediately by telephone or by replying to this transmission.
=20
Merlin Securities, LLC is a registered broker-dealer. Services offered thro=
ugh Merlin Securities, LLC are not insured by the FDIC or any other Federal=
 Government Agency, are not deposits of or guaranteed by Merlin Securities,=
 LLC and may lose value. Nothing in this communication shall constitute a s=
olicitation or recommendation to buy or sell a particular security.

Re: [Quickfixj-users] outgoing seqnum

[Steve B. <st...@te...>]

> We had to stop our engine mid-day today to do some database 
> maintenance.  ...
>
> Why is the sessions table not being updated each time the 
> outgoing seq num is being incremented by the engine?  Why is 
> this not being maintained?  

I don't know based on the information you've provided. You
say you were doing database maintenance. If the engine's
database connection was bad it might explain the problem.
You'd see information in the log file and/or event log
table if this was the cause. This could happen if the
database was restarted but the FIX engines were not.

Was the outgoing sequence number = 1 in the sessions table?
This information won't help to explain the lack of updates,
but it will tell me if the session was reset or whether the
engine just couldn't read the information from the database.

> If we need to stop and start the 
> engine in the middle of the day, we obviously don't want this 
> to happen every time.  What can we do?  

I'd be surprised is this was an issue directly related to
stopping and starting the engine midday. Set up a client
and server test environment using a JDBC message store and
try stopping and starting. It should work. I don't personally
use the JDBC message store, but there are tests for the
implementation and others use it. There are no bug reports
related to this type of behavior.

> In this instance, we 
> contacted the endpoint and they reset the seq num on their 
> end so we could catch up to a reasonable number and then we 
> captured all missed messages. Is there a bug in the jdbc logging?  

Based on the information so far, I'd guess not. Check the log
files for additional clues. 

> Any thoughts on what we can do to better handle a mid-day 
> shutdown and restart that would allows us to pick back up 
> where we left off with our endpoint?

You just be able to restart. If you have any other information,
please let us know.

Steve

[Quickfixj-users] CN=Alvin Wang/O=FFTW is out of the office.

[Alvin W. <AW...@FF...>]

I will be out of the office starting  05/18/2007 and will not return until
05/21/2007.




                                                                *******************************************************************************
This e-mail message is intended solely for the use of the addressee.
The message may contain information that is privileged and 
confidential.
Disclosure to anyone other than the intended recipient is prohibited.
If you are not the intended recipient, please do not disseminate,
distribute or copy this communication, by e-mail or otherwise. Instead,
please notify us immediately by return e-mail(including the original
message with your reply) and then delete and discard all copies of the
message. We have taken precautions to minimize the risk of  transmitting
software viruses but nevertheless advise you to carry out your own
virus checks on any attachment to this message.  We accept
no liability for any loss or damage caused by software viruses.
*******************************************************************************

[Quickfixj-users] outgoing seqnum

[Robert B. <rbr...@me...>]

Hey guys,

We had to stop our engine mid-day today to do some database maintenance.
This is the first time we did this.  It properly sent out a logout
message to our endpoint and shutdown the engine gracefully.  We are
using jdbc logging.  We fired the engine back up about an hour and a
half later and the engine started a new session and started BOTH
incoming and outgoing sequence numbers back at 1.  This caused our FIX
to start a seqnum loop with our endpoint...we would send them 1 and they
would send us that we should be sending 905.  The next heartbeat would
show we sent them 2 and that we should have sent them 905...and so
forth.  You can see the problem...we'd have to wait 903 more iterations
at 30 seconds a pop before the engine would have iterated back up to a
sequence number worthy of receiving messages from our endpoint.  We
looked at the session table from the jdbc looging and the engine does
not seem to be updating this table each time the outgoing sequence
number, so when we stop and start the engine it starts from 1 every
time.  We have NOTHING in our configuration file that tells the engine
to always start from 1...I know there are config parameters that allow
for this...but we are not using any of them...our config file is bare
bones...ip, port, start and stop times, sender and target ids, fix
version...nothing else aside from the jdbc logging config params also.=20=
=20

Why is the sessions table not being updated each time the outgoing seq
num is being incremented by the engine?  Why is this not being
maintained?  If we need to stop and start the engine in the middle of
the day, we obviously don't want this to happen every time.  What can we
do?  In this instance, we contacted the endpoint and they reset the seq
num on their end so we could catch up to a reasonable number and then we
captured all missed messages. Is there a bug in the jdbc logging?=20=20

Any thoughts on what we can do to better handle a mid-day shutdown and
restart that would allows us to pick back up where we left off with our
endpoint?

Thanks,
rob

Merlin Securities - #1 Prime Broker North America, #1 Prime Broker Single S=
trategy Funds, #1 Prime Broker Funds Under $100M - Global Custodian 2007


=20

--------------------------------------------------------

This message contains information from Merlin Securities, LLC, or from one =
of its affiliates, that may be confidential and privileged. If you are not =
an intended recipient, please refrain from any disclosure, copying, distrib=
ution or use of this information and note that such actions are prohibited.=
 If you have received this transmission in error, please notify the sender =
immediately by telephone or by replying to this transmission.
=20
Merlin Securities, LLC is a registered broker-dealer. Services offered thro=
ugh Merlin Securities, LLC are not insured by the FDIC or any other Federal=
 Government Agency, are not deposits of or guaranteed by Merlin Securities,=
 LLC and may lose value. Nothing in this communication shall constitute a s=
olicitation or recommendation to buy or sell a particular security.

[Quickfixj-users] CN=Alvin Wang/O=FFTW is out of the office.

[Alvin W. <AW...@FF...>]

I will be out of the office starting  05/16/2007 and will not return until
05/17/2007.




                                                                *******************************************************************************
This e-mail message is intended solely for the use of the addressee.
The message may contain information that is privileged and 
confidential.
Disclosure to anyone other than the intended recipient is prohibited.
If you are not the intended recipient, please do not disseminate,
distribute or copy this communication, by e-mail or otherwise. Instead,
please notify us immediately by return e-mail(including the original
message with your reply) and then delete and discard all copies of the
message. We have taken precautions to minimize the risk of  transmitting
software viruses but nevertheless advise you to carry out your own
virus checks on any attachment to this message.  We accept
no liability for any loss or damage caused by software viruses.
*******************************************************************************

Re: [Quickfixj-users] SSL with QuickFIX/J 1.1.0

[Christian Z. <chr...@re...>]

Thanks Steve, and thanks Toli,

I fixed the problem in LogUtilTest by setting the default locale to US 
in the setUp() method:

    protected void setUp() throws Exception {
        super.setUp();
        SystemTime.setTimeSource(new 
MockSystemTimeSource(System.currentTimeMillis()));
        Locale.setDefault(Locale.US);
    }

Now I'm wondering about the MultiAcceptorTest, which seems to be waiting 
on something to happen... but what? The junit text file is nearly empty, 
it just shows the name of the class. Is there any documentation about 
the unit tests? In the meantime I'll do my code changes without running 
the unit tests.

I'll be happy to share the changes for configuring a trust store when it 
works. I'm trying to add that to the QFJ config file.

Thanks for the prompt reply!

Regards,

Christian

Re: [Quickfixj-users] SSL with QuickFIX/J 1.1.0

[Steve B. <st...@te...>]

Hi Christian,

Can you look at the test log that Toli described to see what happened 
with the LogUtil test? It passes for me. My guess is that this is a 
locale-related problem. I removed the session settings for start and 
end day from the test in case that is what's causing the problem. 
Although it's a good idea to run the tests, you can also just 
build the QFJ jar without running the tests and doing a complete 
release build, which is relatively time consuming.

The SSL code is QFJ is heavily based on examples from the MINA
project. It's very possible that it could be extended to be more
flexible. I added SSL because I thought it would be useful to the
user community but I don't use it myself so any information and/or
patches you can provide would be appreciated.

Have you tried using the TrustManagerFactory technique you mentioned?
Does that work? I'm currently very busy with my job so I may not be
able to try it for several days.

Steve

[Quickfixj-users] Strange QuoteRequest issue, missing field data

[t.s. <tru...@cb...>]

Hello again everyone :),

Thanks to everyone's help, i managed to write a very basic initiator. So 
now I'm trying to create a QuoteRequest object like this :

...
try {
   quickfix.fix43.QuoteRequest qr = new quickfix.fix43.QuoteRequest();
   qr.getHeader().setField(new SenderSubID("XX"));

   qr.set(new QuoteReqID());
   qr.set(new NoRelatedSym(1));

   quickfix.fix43.QuoteRequest.NoRelatedSym group =
	new quickfix.fix43.QuoteRequest.NoRelatedSym();
   group.set(new Symbol("AUD/USD"));
   group.set(new Product(4));
   group.set(new OrderQty(5000000));
   qr.addGroup(group);

   Session.sendToTarget(qr, "SENDER", "TARGET");
} catch (Exception e) {
   e.printStackTrace();
   System.out.println("error sending quote request");
}
...

For my own debugging purposes, i have a toApp() method like this:
...
public void toApp(Message message, SessionID sessionID) throws DoNotSend {
   if (isMessageOfType(message, MsgType.QUOTE_REQUEST)) {
     System.out.println("QuoteRequest:" + message.toString()); 

   }
}
...

Now here's the strange thing. Screen output from the toApp() method 
shows a "50=XX" field (the SenderSubID Field), but the server log and 
quickfixj's own log shows no such fields. Almost as if the field got 
overwritten somehow. Viewing the log with the QuickFix Log Viewer shows 
that 'SessionRejectReason(373)=RequiredTagMissing' and 'RefTagID(371)=50'.

Did i mess up the QuoteRequest construction ? Please help.
Thank you very much in advance,
regards,
t.s.

PS: I also see some errors about "MsgSeqNum too high' in the event log, 
but only appears intermittenly. Is this significant, or is this a 
different issue altogether ?

Re: [Quickfixj-users] SSL with QuickFIX/J 1.1.0

[Toli K. <to...@ma...>]

Christian,

Haven't played witih SSL/trust stores so can't help you with that.
Just to make sure, you are setting the SocketUseSSL=Y setting in the
config file, right?

 > Any suggestion on how to use a custom trust store is welcome! Or
any hint on how to find out what actually failed in the test!

the failed test part is easy. look under target/test/junit/. you
should see a bunch of TEST-quickfix.xxxTest.txt files

If you are in the top-level directory, you may have to check
core/target/test/junit


-- 
Toli Kuznets
http://www.marketcetera.com: Open-Source Trading Platform
download.run.trade.

[Quickfixj-users] SSL with QuickFIX/J 1.1.0

[Christian Z. <chr...@re...>]

Hi there,

I'm new to QuickFIX/J, just starting to feel at ease handling some 
simple custom messages. I'm implementing a client side application and 
need to connect to our partner using SSL. Of course, I'd like to use 
JSSE rather than running stunnel, since this is supposed to work fine 
since release 1.1.0.

My question: How can I use the custom trust store that was provided by 
my trading partner?
Usually, with JSSE, it's just a matter of defining two system properties:
- javax.net.ssl.trustStore
- javax.net.ssl.trustStorePassword

Unfortunately, this doesn't seem to work. The handshake fails because 
the server's certificate is not trusted. The root certificate being in 
the trust store, it really looks like my trust store is ignored. Niether 
the documentation nor the source code show any way of setting up a trust 
store in the QuickFIX config file.

I see in the QFJ source code that there is a SimpleTrustManagerFactory. 
I don't really get if that trust manager factory does anything. Half of 
the methods are empty placeholders. Why isn't it just a default 
TrustManagerFactory.getInstance("SunX509") ?

Trying to fix the source code on my own, I then stumbled on a build 
problem. Downloading the source zip file and building the whole 
"release.timestamped" target, it fails on LogUtilsTest:
quickfixj\build.xml:25: The following error occurred while executing 
this line:
quickfixj\build.xml:9: The following error occurred while executing this 
line:
quickfixj\core\build.xml:103: The following error occurred while 
executing this line:
quickfixj\core\build.xml:175: Test quickfix.LogUtilTest failed

Any suggestion on how to use a custom trust store is welcome! Or any 
hint on how to find out what actually failed in the test!

Thanks a lot for any help! Looking forward to contribute where I can.

Best regards,

-- 
*Christian Zapf
Realtime Forex SA
*
48, route des Acacias | 1227 Geneva | Switzerland
Tel: +41 22 827 44 53 | Fax: +41 22 827 44 40

<http://www.realtimeforex.com/> <http://www.realtimeforex.com/>

Re: [Quickfixj-users] writing an initiator

[Tommy H. <th...@bo...>]

T.S.,

I found myself asking similar questions when I started with QFJ a few  
weeks ago.  My approach was to create 2 applications that communicate  
with one another and see what happened.  This approach gave me great  
insight into the way QFJ works.  You can create as many sessions as  
you want between the two apps.

Do not be uneasy with the settings file; it actually works much  
better than you might think.  Session(s) are created for each entry  
in the settings file.  Each session defined, based on its type of  
either initiator or acceptor, is created when you call the  
corresponding start() method.

Your application is actually not "single" threaded.  The start()  
method runs a separate thread from your main one.  This thread calls  
back your fromApp() method when a message is received (BTW, I would  
use the message cracker crack() method in fromApp() as the  
documentation suggests).  All you have to do after calling the start 
() method(s) is block (e.g. wait() or sleep(), in a loop if  
preferred).  The "threaded" approach is referring to one thread per  
session as opposed to one for all.  This requires you to make your  
callback method(s) and supporting code thread safe, but allows for  
greater throughput and CPU utilization.

I hope that helps.

-Tommy

On May 10, 2007, at 3:34 PM, t.s. wrote:

> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> Hello everyone,
>
> It's me again :) Here's the outline of the code so far
> ...
> public static void main(String[] args) throws Exception {
>    String filename = "config.cfg";
>
>    Application application = new Main();
>    SessionSettings settings = new SessionSettings(
> 	new FileInputStream(filename));
>
>    MessageStoreFactory storeFactory = new FileStoreFactory(settings);
>    LogFactory logFactory = new FileLogFactory(settings);
>    MessageFactory messageFactory = new DefaultMessageFactory();
>
>    Initiator initiator = new SocketInitiator ( application,  
> storeFactory,
> 	settings, logFactory, messageFactory);
>
>    initiator.start();
>          //1. create a QuoteRequest object
>          //2. init fields
>          //3. send QuoteRequest
>          //4. how to receive quotes ? via fromApp() ?
>          //5. disconnect/housekeeping
>    initiator.stop();
> }
> ...
>
> I have several questions at this point:
> 1. How does QuickFix negotiate login ? Couldn't find anything on the
> subject in the documentation.
>
> 2. If i understand correctly, we're supposed to get messages via the
> fromApp() event, (see #4 in code above), but I'm not sure how the
> program flow should work, since this is a single threaded code. Do i
> have to add a busy loop somewhere, like:
> ...
>     initiator.start();
>     sendSomeMessageToTheOtherSide();
>     while (flag == 0) do { waitaround() };  //catch msg via fromApp();
>     disconnectAndCleanup();
>     initiator.stop();
> ...
>
> 3. I managed to scrape together a configuration file from the docs to
> use with my code, but i feel a bit uneasy about the approach. Is the
> config file mandatory ? Can't i just set the settings via code ?
>
> 4. In the code above, i think it's assumed that quickfix/j will  
> create a
> default session for my use, but what to do when we need to use more  
> than
> one? (probably a bit too advanced for me at this point, and sorry if
> this is a RTFM question).
>
> 5. Still related to question #3 and #4, a settings/config file can
> define multiple session parameter, how do we specify which one to  
> use? I
> can't see how these things work together. For example, from the  
> code above,
> ...
>     SessionSettings settings = new SessionSettings(
> 	new FileInputStream(filename));
>     ... followed by ...
>     Initiator initiator = new SocketInitiator (application,  
> storeFactory,
> 	settings, logFactory, messageFactory);
> ...
> So initiator is initiated by the settings file, but if the settings  
> file
> have multiple sessions, how do quickfixj tell which one to use?
>
> Thank you very much in advance,
> Regards,
> t.s.
>
> PS: Found some issues with the documentation... Where should i  
> report this?
>
> 1. In the 'sending messages' chapter, the following source code is  
> given
> in the last section, the 'Most Type Safe' one.
> ...
> void sendOrderCancelRequest()
> {
>    quickfix.fix41.OrderCancelRequest message(
> 	new OrigClOrdID("123"),
> 	new ClOrdID("321"),
> 	new Symbol("LNUX"),
> 	new Side(Side.BUY));
>
>    message.set(new Text("Cancel My Order!"));
>    Session.sendToTarget(message, "TW", "TARGET");
> }
> ...
> I'm having problem with the line that creates the message object,  
> since
> it doesn't even look like valid java code. I was guessing that it  
> meant
> something like :
> ...
>    OrderCancelRequest ocr = new OrderCancelRequest(blah blah);
> ...
> but it turns out that the constructor has no parameter.
>
> 2. There's a small typo at
> http://www.quickfixj.org/quickfixj/usermanual/usage/configuration.html
>> [DEFAULT]
>> ConnectionType=initator <<<< HERE
>> ReconnectInterval=60
> The usermanual distributed with the latest quickfixj appeared to have
> fixed this.
>
> 3. Earlier, i keep getting problems with NoClassDefFoundError
> Exceptions, problem solved after including the JARs in \lib  
> directory in
> my project (using netbeans, btw). Maybe this should be pointed out in
> the documentation ?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

[Quickfixj-users] writing an initiator

[t.s. <tru...@cb...>]

Hello everyone,

It's me again :) Here's the outline of the code so far
...
public static void main(String[] args) throws Exception {
   String filename = "config.cfg";

   Application application = new Main();
   SessionSettings settings = new SessionSettings(
	new FileInputStream(filename));

   MessageStoreFactory storeFactory = new FileStoreFactory(settings);
   LogFactory logFactory = new FileLogFactory(settings);
   MessageFactory messageFactory = new DefaultMessageFactory();

   Initiator initiator = new SocketInitiator ( application, storeFactory,
	settings, logFactory, messageFactory);

   initiator.start();
         //1. create a QuoteRequest object
         //2. init fields
         //3. send QuoteRequest
         //4. how to receive quotes ? via fromApp() ?
         //5. disconnect/housekeeping
   initiator.stop();
}
...

I have several questions at this point:
1. How does QuickFix negotiate login ? Couldn't find anything on the
subject in the documentation.

2. If i understand correctly, we're supposed to get messages via the
fromApp() event, (see #4 in code above), but I'm not sure how the
program flow should work, since this is a single threaded code. Do i
have to add a busy loop somewhere, like:
...
    initiator.start();
    sendSomeMessageToTheOtherSide();
    while (flag == 0) do { waitaround() };  //catch msg via fromApp();
    disconnectAndCleanup();
    initiator.stop();
...

3. I managed to scrape together a configuration file from the docs to
use with my code, but i feel a bit uneasy about the approach. Is the
config file mandatory ? Can't i just set the settings via code ?

4. In the code above, i think it's assumed that quickfix/j will create a
default session for my use, but what to do when we need to use more than
one? (probably a bit too advanced for me at this point, and sorry if
this is a RTFM question).

5. Still related to question #3 and #4, a settings/config file can
define multiple session parameter, how do we specify which one to use? I
can't see how these things work together. For example, from the code above,
...
    SessionSettings settings = new SessionSettings(
	new FileInputStream(filename));
    ... followed by ...
    Initiator initiator = new SocketInitiator (application, storeFactory,
	settings, logFactory, messageFactory);
...
So initiator is initiated by the settings file, but if the settings file
have multiple sessions, how do quickfixj tell which one to use?

Thank you very much in advance,
Regards,
t.s.

PS: Found some issues with the documentation... Where should i report this?

1. In the 'sending messages' chapter, the following source code is given
in the last section, the 'Most Type Safe' one.
...
void sendOrderCancelRequest()
{
   quickfix.fix41.OrderCancelRequest message(
	new OrigClOrdID("123"),
	new ClOrdID("321"),
	new Symbol("LNUX"),
	new Side(Side.BUY));

   message.set(new Text("Cancel My Order!"));
   Session.sendToTarget(message, "TW", "TARGET");
}
...
I'm having problem with the line that creates the message object, since
it doesn't even look like valid java code. I was guessing that it meant
something like :
...
   OrderCancelRequest ocr = new OrderCancelRequest(blah blah);
...
but it turns out that the constructor has no parameter.

2. There's a small typo at
http://www.quickfixj.org/quickfixj/usermanual/usage/configuration.html
> [DEFAULT]
> ConnectionType=initator <<<< HERE
> ReconnectInterval=60
The usermanual distributed with the latest quickfixj appeared to have
fixed this.

3. Earlier, i keep getting problems with NoClassDefFoundError
Exceptions, problem solved after including the JARs in \lib directory in
my project (using netbeans, btw). Maybe this should be pointed out in
the documentation ?

Re: [Quickfixj-users] Message.toXML does not contain repeating groups

[<hue...@ya...>]

Steve,

i=C2=B4m using toXML(DataDictionary).
The callback does not list the groups, but goes through the group iterating
process (added debugging statements in the library). It seems, that the
callback message does not contain the full message contents, because the
toString() method does not show the Group fields as well. Very strange.
If i read the message back from Message Storage, then the same function
lists the groups.
I=C2=B4m using the latest quickfix/j 1.1.0 version; i could try an older on=
e but
i need the ssl support.

Regards=20
Thomas


Steve Bate wrote:
>=20
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> Are you calling the toXML(DataDictionary) method? The toXML() cannot
> process repeating groups.
>=20
>> -----Original Message-----
>> From: qui...@li... [mailto:quickfixj-
>> use...@li...] On Behalf Of Thomas H=C3=BCgel
>> Sent: Wednesday, May 09, 2007 10:09 AM
>> To: qui...@li...
>> Subject: [Quickfixj-users] Message.toXML does not contain repeating
>> groups
>>=20
>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>> QuickFIX/J Support: http://www.quickfixj.org/support/
>> Hello,
>>=20
>> when receiving messages in fromApp callback and calling toString() or
>> toXML() methods, the repeating groups are not shown.
>> Looking in the message log i can see the repeating groups. Is this a
>> bug ?. If not, how can i get the groups using message.toXML() method ?.
>>=20
>>=20
>> Regards
>> Thomas Huegel
>>=20
>> ------------------------------------------------------------------------=
-
>> This SF.net email is sponsored by DB2 Express
>> Download DB2 Express C - the FREE version of DB2 express and take
>> control of your XML. No limits. Just data. Click to get it now.
>> http://sourceforge.net/powerbar/db2/
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>=20
>=20
>=20
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>=20
>=20

--=20
View this message in context: http://www.nabble.com/Message.toXML-does-not-=
contain-repeating-groups-tf3715985.html#a10408686
Sent from the QuickFIX/J mailing list archive at Nabble.com.