Thread: [Quickfix-users] Quickfix for FIX 5.0.....
Brought to you by:
orenmnero
Menu
▾
▴
quickfix-users
|
From: Eranga S. <pe...@ri...> - 2006-10-25 05:50:21
|
Hi, How is the quickfix / quickfixj preparation for FIX 5.0? Is quickfix need architecture change to address the new loosely coupled FIX application layer and FIX session layer? BR, Eranga |
|
From: Nick E. <ni...@de...> - 2006-10-25 14:33:50
|
Hi, don't know about fix 5.0 quickfix obviously needs architecture changes in it's current state it = suited well only for fix sandboxes, as it has no explicit transaction = management, hence it has no batching operations. it has brokenly = designed synchronization and as a bonus has numerous errors in reset = sequence handling and unexpected crashes in native code in connection = handling. The last two issues are implementation details, but the first two are = crucial for high volume transaction processing. We'd been evaluating quickfix for about half year, but finally decided = to drop it in favor of homegrown engine because of issues I've written = above. Just wonder that nowadays, after all the litrechure regarding highspeed = message parsing available for $30 bucks at amazon, we have such brokenly = designed product :( ----- Original Message -----=20 From: Eranga Samararathna=20 To: qui...@li... ; = qui...@li...=20 Sent: Wednesday, October 25, 2006 9:50 AM Subject: [Quickfix-users] Quickfix for FIX 5.0..... QuickFIX Documentation: = http://www.quickfixengine.org/quickfix/doc/html/index.html QuickFIX Support: http://www.quickfixengine.org/services.html -------------------------------------------------------------------------= ----- Hi, =20 How is the quickfix / quickfixj preparation for FIX 5.0? Is quickfix = need architecture change to address the new loosely coupled FIX = application layer and FIX session layer? =20 BR, =20 Eranga -------------------------------------------------------------------------= ----- = -------------------------------------------------------------------------= Using Tomcat but need to do more? Need to support web services, = security? Get stuff done quickly with pre-integrated technology to make your job = easier Download IBM WebSphere Application Server v.1.0.1 based on Apache = Geronimo = http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D= 121642 -------------------------------------------------------------------------= ----- _______________________________________________ Quickfix-users mailing list Qui...@li... https://lists.sourceforge.net/lists/listinfo/quickfix-users |
|
From: Oren M. <or...@qu...> - 2006-10-25 15:12:05
|
Great points Nick. If anyone wants a real fix engine, contact ni...@de.... --oren On Oct 25, 2006, at 9:37 AM, Nick Evgeniev wrote: > QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/ > html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > Hi, > > don't know about fix 5.0 > quickfix obviously needs architecture changes in it's current state > it suited well only for fix sandboxes, as it has no explicit > transaction management, hence it has no batching operations. it has > brokenly designed synchronization and as a bonus has numerous > errors in reset sequence handling and unexpected crashes in native > code in connection handling. > The last two issues are implementation details, but the first two > are crucial for high volume transaction processing. > We'd been evaluating quickfix for about half year, but finally > decided to drop it in favor of homegrown engine because of issues > I've written above. > Just wonder that nowadays, after all the litrechure regarding > highspeed message parsing available for $30 bucks at amazon, we > have such brokenly designed product :( > |
|
From: Nick E. <ni...@de...> - 2006-10-26 08:29:53
|
Hi,
I'm neither selling nor advertising our engine, just stating the fact. =
If you think that my points aren't valid just tell me where I'm wrong. =
Probably commercial engines are much worse than quickfix just don't =
know, haven't tried them.
I can just repeat mine:
1. quickfix does implicit transaction management passing fix messages =
one-by-one (transaction completed if client didn't trhow an exception). =
obviously while serving the message I have to perform tranaction in the =
database. hence we have obvious (just for me?) limitation on messages =
per second throughput. Hey, ma! where is the batches??? Even jms api has =
nice contract -- message.confirm(); confirming all the previous messages =
received so far.
2. Broken push message model implementation: thread in native code =
holding the lock while propagating message to java. get deadlock prone =
code as a bonus right? It's easy to fix but much better to have pull =
message model.
3. Never ending reset sequence fixes which are just never working right =
in all (valid) cases.
4. At least as of version 1.11.* native code crashes (or silent death of =
acceptor thread) on invalid messages in login phase.
5. Last but not the least -- very GC unfriendly. shure it's not the =
issue until you implement batches, or explicit transaction management.
----- Original Message -----=20
From: Oren Miller=20
To: Nick Evgeniev=20
Cc: Eranga Samararathna ; qui...@li... ; =
qui...@li...=20
Sent: Wednesday, October 25, 2006 7:05 PM
Subject: Re: [Quickfix-users] Quickfix for FIX 5.0.....
Great points Nick. If anyone wants a real fix engine, contact =
ni...@de....
--oren
On Oct 25, 2006, at 9:37 AM, Nick Evgeniev wrote:
QuickFIX Documentation: =
http://www.quickfixengine.org/quickfix/doc/html/index.html
QuickFIX Support: http://www.quickfixengine.org/services.html
Hi,
don't know about fix 5.0
quickfix obviously needs architecture changes in it's current state =
it suited well only for fix sandboxes, as it has no explicit transaction =
management, hence it has no batching operations. it has brokenly =
designed synchronization and as a bonus has numerous errors in reset =
sequence handling and unexpected crashes in native code in connection =
handling.
The last two issues are implementation details, but the first two =
are crucial for high volume transaction processing.
We'd been evaluating quickfix for about half year, but finally =
decided to drop it in favor of homegrown engine because of issues I've =
written above.
Just wonder that nowadays, after all the litrechure regarding =
highspeed message parsing available for $30 bucks at amazon, we have =
such brokenly designed product :(
|
|
From: Nick E. <ni...@de...> - 2006-10-26 15:12:25
|
Hi,
Thank you for such emotional answer I'll try to reply not to the =
emotions but facts
1. quickfix does implicit transaction management passing fix =
messages one-by-one (transaction completed if client didn't trhow an =
exception). obviously while serving the message I have to perform =
tranaction in the database. hence we have obvious (just for me?) =
limitation on messages per second throughput. Hey, ma! where is the =
batches??? Even jms api has nice contract -- message.confirm(); =
confirming all the previous messages received so far.
Wrong. Wrong. Wrong. Not even wrong. You have somehow come under the =
impression that QuickFIX is a message queuing product. Let me be the =
first to dissuade you of that notion. You want a transactional message =
queue? Then stick the message in a transactional message queue. There =
are lots of products out there that do that, QuickFIX isn't one of them. =
Users who build high performance systems do not want the added over head =
of a transactional message queue just because you can't figure out how =
to piece the two together. Believe it or not, some who write high =
frequency trading systems have figured out how to do so without a =
message queue or a database. I'll leave you to figure out how.
You're missing the whole point. I'm not talking about storing messages =
in the database I'm talking about business transactions. They are about =
to happen somethere either in sync or async way, right? Hence the point =
was that quickfix doesn't make things easier in this way, because I have =
to use another abstraction to do batching. Please read it literally -- =
"I want to do batch insert at the end of the chain to insert 1000 orders =
in database at once". The only working solution with quickfix is to =
route it's messages somethere to perform async processing. Hence to use =
it not like queue but rather like transport adapter... You also pointed =
this out but in very inpolite manner. Such infrastructure involves a =
couple of rmi calls which also slows things down. It's always a good =
position to tell -- hey you're an idiot we've provided you a tool that =
is just a trasport! don't expect more. But may I ask you a question? Is =
architecture I'm describing would be harder to implement? Would it =
result bigger code? (I would not say for c++ but in java the code will =
be even simpler:)) As result you will have much more flexible product.
Organizations like Reuters, CME, Pipeline, optionsXpress and many =
others have built massive systems with QuickFIX which run hundreds of =
simultaneous sessions processing hundreds or thousands of messages per =
second. The fact that you can't only speaks to your incompetence. =
Please. Leave QuickFIX out of it.
100 * 100 * 1000 =3D=3D 10 000 000 optionsXpress is able to process 10 =
millions orders per second? don't make my shoes laugh :)
Jumping on person is your style right? Is it because of lacking =
arguments? Do you know how many boxes do they use to handle it? Do you =
know why? because if you use fixengine just as transport you have to =
build/buy messaging infrastracture set several boxes to perform task =
that could be done just by one blade server... Sometimes you don't care =
about it sometimes you do.
2. Broken push message model implementation: thread in native code =
holding the lock while propagating message to java. get deadlock prone =
code as a bonus right? It's easy to fix but much better to have pull =
message model.
Broken is highly exaggerated and isn't born out in the field. Since =
you say it's easy to fix it is surprising you bring this up as a major =
issue and even more surprising you've not submitted a patch.
It was not a top issue it's just an annoying implementation detail. And =
I'm not a c++ developer. just java. sorry. But basicly all you need is =
not to hold monitor while calling java code, but use lock idiom -- flag =
checking under the mutex.
3. Never ending reset sequence fixes which are just never working =
right in all (valid) cases.
Most of the reset scenarios have been fixed. Almost nobody notices =
bugs here because almost nobody uses this functionality. One person who =
did, you, didn't do anything to help the situation. Not all scenarios =
have been implemented because no one is really demanding them. We went =
for years without any reset support and no one even noticed. =20
Everything is quite simple. Nobody warns me neither in documentation nor =
other way. Hey, dude! our reset sequence code is alpha qaulity use it on =
your own risk. It was kind of surprise to me.. and finally decison was =
made to get rid of quickfix because of the reason #1 not #3.
I'll tell you what's going on here Nick. You made some promises to =
deliver, you failed, and now QuickFIX is going to be the fall guy. It =
took you 6 months to figure out that QuickFIX didn't meet your needs? =
That's some evaluation, did someone pay for you to waste all that time?
Probably you do have god's number because you know things you're not =
supposed to know. would you send it to me? Yes I was paid because =
solution with qf was developed in about a month, most time of which were =
spent on integration issues. And it was working ok... But finally we =
decided to develop ourown implementation in pure java that is why I'm =
talking about six month...
QuickFIX is designed to meet the needs of thousands of users. It is =
designed to be portable, works with standard and broken implementation =
of the FIX protocol, integrates with any database, file storage, custom =
storage, custom and out of the box logging, has a web interface, =
documentation, tons of settings, is written in portable C++ that =
compiles under many compilers on many operating systems. It builds =
right out of the box and also comes in pre-built binaries for windows. =
It has APIs in C++, Java, .NET, Ruby and Python plus a full from scratch =
pure Java implementation. That's just for starters. It's been in =
production use for 5 years and has, let me just say, an astounding =
amount of production implementations by an even more astounding group of =
well respected financial companies. =
.
Probably this should be output to log on startup to prevent complains :)
So you can write an implementation in a single platform that meets =
your specific needs designed to fit into your exact architecture. Big =
deal hotshot.
java is supposed to work on many platforms :)... bigdeal?
I'm sorry to hear that your leaving, but Don't worry about us Nick. =
>From now on everyone on the list will look to what someone once posted =
on this very list as our guiding inspiration.
"as for everything else -- just keep moving! quickfix rocks :)"
-- Nick Evgeniev (May 31, 2005)
I don't reclaim my words :)... Keep moving! change it to be more high =
performant, as changes are not as expensive :))
|
|
From: Nick E. <ni...@de...> - 2006-10-26 17:36:29
|
Hi,
Well, actually the whole story of initial message was quite simple -- my =
eye caugth word architecture, not the subject. Thus, you've qualified it =
as blame... Why I didn't adopt quickfix/j? It was early beta those days =
and relplicates original quickfix design, hence also didn't provide =
batch processing, which is crucial (I don't like idea of processing =
messages one-by-one on any stage (even transport), sorry).
On other side what is nice for quickfix/j at least implementation =
details (synchronization issues/sequence reset) could be easily fixed by =
java developers that's a huge bonus as I think about 90% of quickfix =
deployments is java platform. And it's free from mistery sig segv =
crashes we've seen on sparc solaris (x86 linux was ok).
As for quoting/replying issue... You've just stole my words :). Would =
you read carefully my questions and your answers?
10mlns per sec transaction issue. Well yes, it was a trick from my side =
to show you how easy to missinterpret the opponent's words. Exactly what =
you've done.
Once again Don't you really think I'm spending my time posting here to =
down quickfix??? If so, you are wrong! It was attempt to share thoughts =
and expirience. You didn't like it. That's ok. The truth is that I like =
quickfix initiative (why do you think I'm still reading mail list?).=20
My words were taugh, but I've figured out that this is the only approach =
then things get stagnating... I'm sorry about that.
----- Original Message -----=20
From: Oren Miller=20
To: Nick Evgeniev=20
Cc: qui...@li... ; =
qui...@li...=20
Sent: Saturday, October 28, 2006 8:54 PM
Subject: Re: [Quickfix-developers] [Quickfix-users] Quickfix for FIX =
5.0.....
No Nick. You bounded into the message board guns a blazin' attacking =
the project while thread-jacking a discussion of FIX 5.0 support, and =
then sleek into the role of the tempered rationalist when someone calls =
you on it. You then use the boring old trick of cherry picking the =
points you want to respond to and completely removing the others =
entirely. If I don't quote it it's like it never happened! I don't =
need to justify my statement and I can make the other person look =
foolish at the same time!
You're the guy that sucker punches someone then steps back and says =
"Whoa whoa, don't be so emotional, let's be rational about this." =
Classy. You didn't come here for a rational exchange, you came to take =
potshots.
If it's so easy, why you didn't take a couple days to adapt QuickFIX/J =
to your needs instead of writing a whole engine from scratch is rather =
curious. But that's your choice. If what you have works for you, =
that's great. QuickFIX isn't for everyone. You've made it very clear =
it's not for you. You're not the first nor the last that will take a =
pass on QuickFIX. Hopefully some day you will release your bounty unto =
the world.
For the record, the number of transactions was meant to be total, not =
per session. I make no claim of anyone processing 10,000,000 orders a =
second. No one does that many. The sentence was awkward and your =
interpretation is understandable, but was not the meaning I meant to =
convey. Most of the systems I refer to run these sessions in a single =
process. Point is that calling QuickFIX nothing more than a sandbox =
implementation is patently false.
--oren
On Oct 26, 2006, at 10:15 AM, Nick Evgeniev wrote:
Hi,
Thank you for such emotional answer I'll try to reply not to the =
emotions but facts
1. quickfix does implicit transaction management passing fix =
messages one-by-one (transaction completed if client didn't trhow an =
exception). obviously while serving the message I have to perform =
tranaction in the database. hence we have obvious (just for me?) =
limitation on messages per second throughput. Hey, ma! where is the =
batches??? Even jms api has nice contract -- message.confirm(); =
confirming all the previousmessages received so far.
Wrong. Wrong. Wrong. Not even wrong. You have somehow come under =
the impression that QuickFIX is a message queuing product. Let me be =
the first to dissuade you of that notion. You want a transactional =
message queue? Then stick the message in a transactional message queue. =
There are lots of products out there that do that, QuickFIX isn't one =
of them. Users who build high performance systems do not want the added =
over head of a transactional message queue just because you can't figure =
out how to piece the two together. Believe it or not, some who write =
high frequency trading systems have figured out how to do so without a =
message queue or a database. I'll leave you to figure out how.
You're missing the whole point. I'm not talking about storing =
messages in the database I'm talking about business transactions. They =
are about to happen somethere either in sync or async way, right? Hence =
the point was that quickfix doesn't make things easier in this way, =
because I have to use another abstraction to do batching. Please read it =
literally -- "I want to do batch insert at the end of the chain to =
insert 1000 orders in database at once". The only working solution with =
quickfix is to route it's messages somethere to perform async =
processing. Hence to use it not like queue but rather like transport =
adapter... You also pointed this out but in very inpolite manner. Such =
infrastructure involves a couple of rmi calls which also slows things =
down. It's always a good position to tell -- hey you're an idiot we've =
provided you a tool that is just a trasport! don't expect more. But may =
I ask you a question? Is architecture I'm describing would be harder to =
implement? Would it result bigger code? (I would not say for c++ but in =
java the code will be even simpler:)) As result you will have much more =
flexible product.
Organizations like Reuters, CME, Pipeline, optionsXpress and many =
others have built massive systems with QuickFIX which run hundreds of =
simultaneous sessions processing hundreds or thousands of messages per =
second. The fact that you can't only speaks to your incompetence. =
Please. Leave QuickFIX out of it.
100 * 100 * 1000 =3D=3D 10 000 000 optionsXpress is able to process =
10 millions orders per second? don't make my shoes laugh :)
Jumping on person is your style right? Is it because of lacking =
arguments? Do you know how many boxes do they use to handle it? Do you =
know why? because if you use fixengine just as transport you have to =
build/buy messaging infrastracture set several boxes to perform task =
that could be done just by one blade server... Sometimes you don't care =
about it sometimes you do.
2. Broken push message model implementation: thread in native =
code holding the lock while propagating message to java. get deadlock =
prone code as a bonus right? It's easy to fix but much better to have =
pull message model.
Broken is highly exaggerated and isn't born out in the field. =
Since you say it's easy to fix it is surprising you bring this up as a =
major issue and even more surprising you've not submitted a patch.
It was not a top issue it's just an annoying implementation detail. =
And I'm not a c++ developer. just java. sorry. But basicly all you need =
is not to hold monitor while calling java code, but use lock idiom -- =
flag checking under the mutex.
3. Never ending reset sequence fixes which are just never =
working right in all (valid) cases.
Most of the reset scenarios have been fixed. Almost nobody =
notices bugs here because almost nobody uses this functionality. One =
person who did, you, didn't do anything to help the situation. Not all =
scenarios have been implemented because no one is really demanding them. =
We went for years without any reset support and no one even noticed. =20
Everything is quite simple. Nobody warns me neither in documentation =
nor other way. Hey, dude! our reset sequence code is alpha qaulity use =
it on your own risk. It was kind of surprise to me.. and finally decison =
was made to get rid of quickfix because of the reason #1 not #3.
I'll tell you what's going on here Nick. You made some promises =
to deliver, you failed, and now QuickFIX is going to be the fall guy. =
It took you 6 months to figure out that QuickFIX didn't meet your needs? =
That's some evaluation, did someone pay for you to waste all that time?
Probably you do have god's number because you know things you're not =
supposed to know. would you send it to me? Yes I was paid because =
solution with qf was developed in about a month, most time of which were =
spent on integration issues. And it was working ok... But finally we =
decided to develop ourown implementation in pure java that is why I'm =
talking about six month...
QuickFIX is designed to meet the needs of thousands of users. It =
is designed to be portable, works with standard and broken =
implementation of the FIX protocol, integrates with any database, file =
storage, custom storage, custom and out of the box logging, has a web =
interface, documentation, tons of settings, is written in portable C++ =
that compiles under many compilers on many operating systems. It builds =
right out of the box and also comes in pre-built binaries for windows. =
It has APIs in C++, Java, .NET, Ruby and Python plus a full from scratch =
pure Java implementation. That's just for starters. It's been in =
production use for 5 years and has, let me just say, an astounding =
amount of production implementations by an even more astounding group of =
well respected financial companies. =
.
Probably this should be output to log on startup to prevent =
complains :)
So you can write an implementation in a single platform that meets =
your specific needs designed to fit into your exact architecture. Big =
deal hotshot.
java is supposed to work on many platforms :)... bigdeal?
I'm sorry to hear that your leaving, but Don't worry about us =
Nick. From now on everyone on the list will look to what someone once =
posted on this very list as our guiding inspiration.
"as for everything else -- just keep moving! quickfix rocks :)"
-- Nick Evgeniev (May 31, 2005)
I don't reclaim my words :)... Keep moving! change it to be more =
high performant, as changes are not as expensive :))
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X