Re: [Quickfix-developers] 4.2 MessageCracker/MessageFactory efficiency

[Igor S. <se...@tb...>]

Hi Rich,

Yes, not a single crash so far. We are using it in production.

I'll put it to QuickFIX GIT hub soon.

Regards,
Igor


-----Original Message-----
From: Rich Holm [mailto:rh...@wo...] 
Sent: Thursday, November 19, 2009 5:52 PM
To: Igor Seleznev
Cc: 'Kenny Stone'; qui...@li...
Subject: Re: [Quickfix-developers] 4.2 MessageCracker/MessageFactory
efficiency



Igor,

Are those releases stable? Would you recommend them for production use?

Cheers,
Rich



Igor Seleznev wrote:
> QuickFIX Documentation:
http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
>   
>
> ------------------------------------------------------------------------
>
> Hi,
>
> We made a significant performance improvement of QuickFIX a time ago 
> by replacing std::string objects with C strings - it turned out that 
> std::string from stlport under Solaris was performing allocations 
> through some internal lock, which was a bottle-neck in a such 
> configuration.
>
> Anyway, such a change showed a performance improvement under other 
> platforms (like Linux) where there was no issue with std::string's 
> internal lock. That is, we eventually eliminated most of unnecessary 
> alloc/free which std::string was a reason of.
>
> With GIT it is now easier for us to share these quite big changes, if 
> anyone is interested, of course.
>
> Regards,
>
> Igor
>
> *From:* Kenny Stone [mailto:ks...@co...]
> *Sent:* Thursday, November 19, 2009 3:16 AM
> *To:* qui...@li...
> *Subject:* Re: [Quickfix-developers] 4.2 MessageCracker/MessageFactory 
> efficiency
>
> If you decide to make some changes, we would be interested to see what 
> you find out. There are already a few interesting branches 
> <http://github.com/jaubrey/quickfix/network> on the quickfix github 
> page, one of which is performance related that will probably get 
> merged at some point.
>
> --
> Kenny Stone
> Connamara Systems, LLC
>
> On Wed, Nov 18, 2009 at 5:02 PM, Dale Wilson <wi...@oc... 
> <mailto:wi...@oc...>> wrote:
>
> QuickFIX Documentation: 
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Hi Rick,
>
> Rick Lane wrote:
> > We were recently taking a look through the FIX42_MessageCracker and
> > MessageFactory code (we're using the .NET wrapper for Quickfix) to
> > see if we could make some improvements in our order routing internal
> > latency. We noticed that in both crack() and create()
> >
> > 1) it was doing string comparisons for the message type (when for FIX
> > 4.2 all message types are a single character)
> > 2) it was doing if-statements rather than a switch statement
> > 3) the if-statement for ExecutionReport (which is the most frequent
> > type of message) was 30th in this list!
> >
> > So we figured we could speed this up substantially by:
> >
> > 1) creating a switch() statement (that is constant time) instead of 60
> > or so if-statements and
> > 2) simply switch on the int value of the character (since we know it's
> > only 1 character)
> >
> > However, in practice, we haven't noticed any speed gains and we almost
> > seem to think we're seeing /slower /performance (this is tough to
> > measure because we have to measure in conjunction with WireShark to
> > see how long the message was outside our server (transit latency) and
> > subtract that from the measured time in code).
> >
> > We're wondering if perhaps we aren't doing something correctly at
> > compile time, if anyone has seen these if-statements and thought the
> > same thing we thought, and if anyone has any input in general here.
> >
> > Any help is greatly appreciated.
> I recently spent several months reducing the latency in the QuickFAST
> (not QuickFIX) decoder by two orders of magnitude (i.e. a factor of
> 100) These observations are based on that work:
>
> With the complexity of C++ and the sometimes remarkable ability of
> modern compilers to optimize code, inspecting the C++ source code to
> improve performance is rarely effective. For example, there are some
> compilers that generate exactly the same code for a switch statement and
> a chain of if-else's. As you pointed out, if the possible values are
> sufficiently constrained it is possible to decide which code to execute
> in constant time. Trying to improve performance by rearranging source
> code with a compiler like that can be an exercise in futility.
>
> A similar argument applies to the string comparison. If the comparison
> is inline and the values being compared to are constant literals (like,
> for example FIX tags) then the compiler may very well generate that
> single character comparison for you!
>
> The first step in improving performance should ALWAYS be to measure.
> You need to run a profiler and watch the system under load. If you
> work on a section of the code that is responsible for, say 1% of the
> overall latency, and you find the perfect optimization, the best result
> you could hope for is a 1% improvement in the overall system.
>
> The next step in improving performance is to eliminate the noise.
> Admittedly in the real world you need to be concerned with network
> transmission time and efficiency of the communication stack, etc. but
> if you're trying to improve the latency introduced by QuickFIX, you need
> to get rid of these variables altogether, not just factor them out..
> Otherwise trivial variations in the behavior of the network will totally
> swamp the performance differences you are trying to measure. [Which
> brings up a different point. The payoff in improving the network can
> be far higher than the payoff in improving code at the QuickFIX level,
> but I'll assume that's already done and you still want better 
> performance..]
>
> When the profiler directs your attention to a particular portion of the
> code and you very carefully hand-optimize that section of code to
> squeeze every possible CPU cycle out of it, run the profiler again!
> Don't assume that just because the code looks faster, that it will
> actually run faster. During the QuickFAST optimization process I spent
> a lot of time creating a cache of very commonly used objects that could
> be reused rather than being "new" ed and "deleted" every time. Net
> result -- the performance was worse. I ended up throwing all that work
> away, and trying a different approach.
>
> And lastly, I hope you'll share the results of your efforts. I am very
> interested in what works and what doesn't work when trying to improve
> program performance.
>
> Regards,
>
> Dale
>
> --
> Dale Wilson
> Principal Software Engineer
> Object Computing, Inc. (www.ociweb.com <http://www.ociweb.com>)
> Lead developer for QuickFAST (http:www.quickfast.org 
> <http://www.quickfast.org>)
>
>
>
>
>
>
>
>
>
> >
> > Rick
> >
> > P.S. I'm not sure what the process is (it's been so long now) but if
> > you could add dan...@ti... <mailto:dan...@ti...> 
> <mailto:dan...@ti... <mailto:dan...@ti...>> to
> > the mailing list so it will accept emails from him. Thanks again.
> > ------------------------------------------------------------------------
> >
> > 
>
----------------------------------------------------------------------------
--
> > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 
> 30-Day
> > trial. Simplify your report design, integration and deployment - and 
> focus on
> > what you do best, core application coding. Discover what's new with
> > Crystal Reports now. http://p.sf.net/sfu/bobj-july
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Quickfix-developers mailing list
> > Qui...@li... 
> <mailto:Qui...@li...>
> > https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>
>
>
----------------------------------------------------------------------------
--
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 
> 30-Day
> trial. Simplify your report design, integration and deployment - and 
> focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li... 
> <mailto:Qui...@li...>
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>
> ------------------------------------------------------------------------
>
>
----------------------------------------------------------------------------
--
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
30-Day 
> trial. Simplify your report design, integration and deployment - and focus
on 
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> ------------------------------------------------------------------------
>
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers



This message is intended only for the personal and confidential use of the
recipients named above. 
If the reader of this email is not the intended recipient, you have received
this email in error and any review, 
dissemination, distribution or copying is strictly prohibited. If you have
received this email in error, 
please notify the sender immediately by return email and permanently delete
the copy you received. 

This message is provided for informational purposes and should not be
construed as a solicitation or offer 
to buy or sell any securities or related financial instruments. Wolverine is
not responsible for any 
recommendation, solicitation, offer or agreement or any information about
any transaction, customer account 
or account activity that may be attached to or contained in this
communication. Wolverine accepts no 
liability for any content contained in the email, or any errors or omissions
arising as a result of 
email transmission. Any opinions contained in this email constitute the
sender's best judgment at this 
time and are subject to change without notice.