Re: [Quickfix-developers] 4.2 MessageCracker/MessageFactory efficiency

[Kenny S. <ks...@co...>]

If you decide to make some changes, we would be interested to see what you
find out.  There are already a few interesting
branches<http://github.com/jaubrey/quickfix/network>on the quickfix
github page, one of which is performance related that will
probably get merged at some point.

--
Kenny Stone
Connamara Systems, LLC


On Wed, Nov 18, 2009 at 5:02 PM, Dale Wilson <wi...@oc...> wrote:

> QuickFIX Documentation:
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Hi Rick,
>
> Rick Lane wrote:
> > We were recently taking a look through the FIX42_MessageCracker and
> > MessageFactory code (we're using the .NET wrapper for Quickfix)  to
> > see if we could make some improvements in our order routing internal
> > latency.  We noticed that in both crack() and create()
> >
> > 1) it was doing string comparisons for the message type (when for FIX
> > 4.2 all message types are a single character)
> > 2) it was doing if-statements rather than a switch statement
> > 3) the if-statement for ExecutionReport (which is the most frequent
> > type of message) was 30th in this list!
> >
> > So we figured we could speed this up substantially by:
> >
> > 1) creating a switch() statement (that is constant time) instead of 60
> > or so if-statements and
> > 2) simply switch on the int value of the character (since we know it's
> > only 1 character)
> >
> > However, in practice, we haven't noticed any speed gains and we almost
> > seem to think we're seeing /slower /performance (this is tough to
> > measure because we have to measure in conjunction with WireShark to
> > see how long the message was outside our server (transit latency) and
> > subtract that from the measured time in code).
> >
> > We're wondering if perhaps we aren't doing something correctly at
> > compile time, if anyone has seen these if-statements and thought the
> > same thing we thought, and if anyone has any input in general here.
> >
> > Any help is greatly appreciated.
> I recently spent several months reducing the latency in the QuickFAST
> (not QuickFIX) decoder by two orders of magnitude (i.e. a factor of
> 100)  These observations are based on that work:
>
> With the complexity of C++ and the sometimes remarkable ability of
> modern compilers to optimize code, inspecting the C++ source code to
> improve performance is rarely effective.  For example, there are some
> compilers that generate exactly the same code for a switch statement and
> a chain of if-else's. As you pointed out, if the possible values are
> sufficiently constrained it is possible to decide which code to execute
> in constant time.   Trying to improve performance by rearranging source
> code with a compiler like that can be an exercise in futility.
>
> A similar argument applies to the string comparison.  If the comparison
> is inline and the values being compared to are constant literals (like,
> for example FIX tags) then the compiler may very well generate that
> single character comparison for you!
>
> The first step in improving performance should ALWAYS be to measure.
> You need to run a profiler and watch the system under load.   If you
> work on a section of the code that is responsible for, say 1% of the
> overall latency, and you find the perfect optimization, the best result
> you could hope for is a 1% improvement in the overall system.
>
> The next step in improving performance is to eliminate the noise.
> Admittedly in the real world you need to be concerned with network
> transmission time and efficiency of the communication stack, etc.  but
> if you're trying to improve the latency introduced by QuickFIX, you need
> to get rid of these variables altogether, not just factor them out..
> Otherwise trivial variations in the behavior of the network will totally
> swamp the performance differences you are trying to measure.   [Which
> brings up a different point.   The payoff in improving the network can
> be far higher than the payoff in improving code at the QuickFIX level,
> but I'll assume that's already done and you still want better
> performance..]
>
> When the profiler directs your attention to a particular portion of the
> code and you very carefully hand-optimize that section of code to
> squeeze every possible CPU cycle out of it, run the profiler again!
> Don't assume that just because the code looks faster, that it will
> actually run faster.  During the QuickFAST optimization process I spent
> a lot of time creating a cache of very commonly used objects that could
> be reused rather than being "new" ed and "deleted" every time.  Net
> result -- the performance was worse.  I ended up throwing all that work
> away, and trying a different approach.
>
> And lastly, I hope you'll share the results of your efforts.  I am very
> interested in what works and what doesn't work when trying to improve
> program performance.
>
> Regards,
>
> Dale
>
> --
> Dale Wilson
> Principal Software Engineer
> Object Computing, Inc. (www.ociweb.com)
> Lead developer for QuickFAST (http:www.quickfast.org)
>
>
>
>
>
>
>
>
>
> >
> > Rick
> >
> > P.S.  I'm not sure what the process is (it's been so long now) but if
> > you could add dan...@ti... <mailto:dan...@ti...> to
> > the mailing list so it will accept emails from him.  Thanks again.
> > ------------------------------------------------------------------------
> >
> >
> ------------------------------------------------------------------------------
> > Let Crystal Reports handle the reporting - Free Crystal Reports 2008
> 30-Day
> > trial. Simplify your report design, integration and deployment - and
> focus on
> > what you do best, core application coding. Discover what's new with
> > Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Quickfix-developers mailing list
> > Qui...@li...
> > https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>