Re: [Quickfixj-users] outgoing seqnum

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

You're correct Joerg...I have been unclear...I'm using both the jdbc loggin=
g AND jdbc store.  I finally got the store to work with Oracle yesterday af=
ternoon by bypassing the proxool libraries altogether (As per Steve's sugge=
stion) and modifying the JdbcUtil class to use OracleConnection and OracleD=
ataSource objects directly...I then also had to severely modify the  JdbcSt=
ore class because we're not using session qualifiers in our config file, so=
 the session qualifier is an empty string, which in Oracle is NULL...all of=
 the update, delete and insert statements in the JdbcStore class use prepar=
ed statements like "WHERE session_qualifier=3D?" and this is invalid sql fo=
r oracle when you have a null/empty string condition...you cannot use "WHER=
E session_qualifier =3D null" for a logic check, you either need to leave i=
t out of the prepared statement or use "WHERE session_qualifier IS NULL".=
=20=20

For all of the update, delete and insert statements I had to create a secon=
d version of them and in all of the methods using these, I had to check the=
 value of the session qualifier and if it was an empty string I have to use=
 my NULL version of the prepared statements...I also had to add connection.=
commit(); to the bottom of every update, insert and delete method in order =
for my oracle store to work.=20=20

Today is the first true day we'll be running the engine with my custom libr=
ary changes so I'm sure I'll have to iron out some bugs.  Luckily we're sti=
ll in development phase!  When I have some time, I'd like to actually exten=
d the existing JdbcUtil, JdbcLog, JdbcStore classes with Oracle-friendly im=
plementations and submit it for review to be a part of the actual quickfix/=
j library...that's if my testing over the next few days goes well.

Thanks!

robert l. brueckmann
vice president
merlin securities
712 fifth avenue
new york, ny 10019
p: 212.822.4821
f: 212.822.4820

Merlin Securities - #1 Prime Broker North America, #1 Prime Broker Single S=
trategy Funds, #1 Prime Broker Funds Under $100M - Global Custodian 2007

From: qui...@li... [mailto:quickfixj-users=
-bo...@li...] On Behalf Of Joerg Thoennes
Sent: Thursday, May 24, 2007 6:04 AM
To: qui...@li...
Subject: Re: [Quickfixj-users] outgoing seqnum

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
Hi Robert,

we currently use JdbcStore, but not JdbcLog (you sometimes call things "log=
ging" where I would
expect the MessageStore; is it possible to you mix logging and saving state=
?).

In our test environment, all works well with JDBC access to PostgreSQL. Our=
 customer uses Oracle 8
with OCI, there are currently installing our new version and maybe I get so=
me feedback soon.

If I remember right, you are using Oracle 9? Which JDBC driver -- "thin:" o=
r "oci:"? Please give me
some more details here.

For the moment, I would recommend to replace the JdbcLog by Log4J logging. =
This is flexible enough
for most purpose. For failover, it is important to keep the JdbcStore.

In general, I am very interested in your experiences because our customer a=
lso uses Oracle.

Cheers, J=F6rg

On 05/21/07 20:55, Robert Brueckmann wrote:
> Thanks for the reply.  I did as you said and set up a test
> environment...I am using JDBC logging against Oracle.  I'm the one that
> had a barrage of emails a few weeks ago about the problems with JDBC
> logging and Oracle playing nicely together.  I'm still intermittently
> getting the recursive exception about:
>=20
> "JdbcLog cannot log SQLException due to recursive log errors!
> java.sql.SQLException: We are already in the process of making 1
> connections and the number of simultaneous builds has been throttled to
> 0"
>=20
> Thing is, it IS logging to the database.  I get a record inserted in the
> Sessions table with the initial information, only the incoming AND
> outgoing seqnums are always set to 1.  I then enter an order in the test
> client the one vendor I'm testing with provided us to buy 500 shares of
> IBM.  The messages come through...all the messages are stored in the
> database, I let the heartbeat run for a while, then I shut down the
> engine and start it back up again.
>=20
> Immediately I begin receiving:
>=20
> "Serious Error:  Message sequence number: 1 is less than expected
> sequence number: 28"
>=20
> And then the engine continues to logout and login, each time, internally
> incrementing that message count, but never once updating the session
> entry in the SESSIONS table in the database with the newly incremented
> outgoing sequence number.  Currently, the engine is up to 22 and the
> SESSIONS table record for this engine instance is still showing 1 for
> both the incoming and outgoing sequence numbers...but internally, it's
> obviously incrementing the count.
>=20
> Nowhere on the console are there any database exceptions relating to
> having a problem inserting into the SESSIONS table or updating that
> table.  Every so often I'll get something like this:
>=20
> "2007-05-21 14:44:47,361 DEBUG [HouseKeeper] proxool.ConnectionPool
> (ConnectionPool.java:447) - 000079 (00/01/00) - #0001 removed because it
> has been active for too long.
> 2007-05-21 14:44:47,377 WARN  [HouseKeeper] proxool.HouseKeeper
> (HouseKeeper.java:149) - #0001 was active for 11562 milliseconds and has
> been removed automaticaly. The Thread responsible was named 'QFJ Timer',
> but the last SQL it performed is unknown because the trace property is
> not enabled.
> JdbcLog cannot log SQLException due to recursive log errors!
> java.sql.SQLException: We are already in the process of making 1
> connections and the number of simultaneous builds has been throttled to
> 0"
>=20
> I don't know what to do about these exceptions or the one I mentioned
> above, but messages and the message log tables both seem to be capturing
> the messages fine and running the engine without shutting it down and
> restarting for a long period of time, quite a few days in a row...the
> engine seems to work as does the logging without a hitch...but the
> minute we shut it down, that internal incrementing is lost and the
> sessions table sequence numbers are always initiated with 1 and we get
> stuck in a sequence reset loop until the engine increments to a number
> the endpoint session can handle, then a successful reset request is
> handled and things carry on normally.
>=20
> Any thoughts?  Should I just give up on JDBC logging in conjunction with
> an Oracle database?=20=20
>=20
> Is there a better way to handle fail-over?  The reason the IT guys here
> wanted database logging was so if the FIX engine server went down, we
> could fire up the fail-over server and it would be able to look at the
> database logs (which reside on a totally different server) and pick up
> seamlessly where the initial instance left off.  If we use just basic
> file logging and the entire FIX server crashes and we can't get to the
> log files, what do we do?  Am I completely confused in thinking that the
> database logging approach would be better for my scenario or are there
> alternatives?
>=20
> Thanks so much!
>=20
> robert l. brueckmann
> vice president
> merlin securities
> 712 fifth avenue
> new york, ny 10019
> p: 212.822.4821
> f: 212.822.4820
>=20
>=20
>=20
> Merlin Securities - #1 Prime Broker North America, #1 Prime Broker Single=
 Strategy Funds, #1 Prime Broker Funds Under $100M - Global Custodian 2007
>=20
>=20
> From: qui...@li...
> [mailto:qui...@li...] On Behalf Of
> Steve Bate
> Sent: Sunday, May 20, 2007 6:01 PM
> To: qui...@li...
> Subject: Re: [Quickfixj-users] outgoing seqnum
>=20
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>> We had to stop our engine mid-day today to do some database=20
>> maintenance.  ...
>>
>> Why is the sessions table not being updated each time the=20
>> outgoing seq num is being incremented by the engine?  Why is=20
>> this not being maintained?=20=20
>=20
> I don't know based on the information you've provided. You
> say you were doing database maintenance. If the engine's
> database connection was bad it might explain the problem.
> You'd see information in the log file and/or event log
> table if this was the cause. This could happen if the
> database was restarted but the FIX engines were not.
>=20
> Was the outgoing sequence number =3D 1 in the sessions table?
> This information won't help to explain the lack of updates,
> but it will tell me if the session was reset or whether the
> engine just couldn't read the information from the database.
>=20
>> If we need to stop and start the=20
>> engine in the middle of the day, we obviously don't want this=20
>> to happen every time.  What can we do?=20=20
>=20
> I'd be surprised is this was an issue directly related to
> stopping and starting the engine midday. Set up a client
> and server test environment using a JDBC message store and
> try stopping and starting. It should work. I don't personally
> use the JDBC message store, but there are tests for the
> implementation and others use it. There are no bug reports
> related to this type of behavior.
>=20
>> In this instance, we=20
>> contacted the endpoint and they reset the seq num on their=20
>> end so we could catch up to a reasonable number and then we=20
>> captured all missed messages. Is there a bug in the jdbc logging?=20=20
>=20
> Based on the information so far, I'd guess not. Check the log
> files for additional clues.=20
>=20
>> Any thoughts on what we can do to better handle a mid-day=20
>> shutdown and restart that would allows us to pick back up=20
>> where we left off with our endpoint?
>=20
> You just be able to restart. If you have any other information,
> please let us know.
>=20
> Steve
--=20
Joerg Thoennes
http://www.macd.com                 Tel.: +49 (0)241 44597-24
Macdonald Associates GmbH           Gesch=E4ftsf=FChrer: Roger Macdonald
Lothringer Str. 52, D-52070 Aachen  Amtsgericht Aachen, HRB 8151, Ust.-Id D=
E813021663

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Quickfixj-users mailing list
Qui...@li...
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
=20

--------------------------------------------------------

This message contains information from Merlin Securities, LLC, or from one =
of its affiliates, that may be confidential and privileged. If you are not =
an intended recipient, please refrain from any disclosure, copying, distrib=
ution or use of this information and note that such actions are prohibited.=
 If you have received this transmission in error, please notify the sender =
immediately by telephone or by replying to this transmission.
=20
Merlin Securities, LLC is a registered broker-dealer. Services offered thro=
ugh Merlin Securities, LLC are not insured by the FDIC or any other Federal=
 Government Agency, are not deposits of or guaranteed by Merlin Securities,=
 LLC and may lose value. Nothing in this communication shall constitute a s=
olicitation or recommendation to buy or sell a particular security.