Re: [Quickfix-developers] Architecture Question

[K. F. <kfr...@gm...>]

Hello Arup -


On Fri, Aug 13, 2010 at 11:18 PM, Sarkar, Arup <sar...@gm...> wrote:
>
> Hi,
>
> I have a basic architecture question. The following is what I am doing currently.

At a rough level, I see three ways to go here, outlined by
increasing modularity.  All could work well, and the which
of these could be good choices for you will depend on your
business logic and technical requirements.

1)  A single, integrated "trading engine" (that runs as a single
application / process).  It would both listen to the pricing data
from the exchange and send order flow through QuickFIX to
your broker (or the exchange or other execution venue).

2)  Your current scheme, where you have a "trading engine"
that listens to pricing data from the exchange, implements
your trading logic, and sends order flow through a separate
QuickFIX-based FIX engine that runs as a separate
application / process.

3)  You have a separate "price-feed server" and QuickFIX-based
FIX engine that both run as separate applications / processes,
and your trading engine is a client of both.  It this case, your
system would consist of three cooperating processes.

> a) I have a c++ client program (program A)  which connects to exchange to get high frequency data, in the same program it is formatted and fed into strategy. The output of my strategy is symbol, price, quantity, limit order and relevant data entities for NewOrderSingle, CancelOrder, or ReplaceOrder.

So far, so good.

> b) I have implemented QuickFix (program B) with which I am connecting to broker to execute my trades. All the above three operation which I mentioned is working fine and  I am also capturing the Execution Report.

Do I assume correctly that you are using C++ Quickfix?  If so, going
with plan 1, and integrating your programs A and B together into a
single, integrated trading engine shouldn't be too hard.

If you are using the Java QuickFIX/J, then you would almost certainly
want to keep your FIX engine as a separate process (either plan 2 or 3).

> BUT, the 2 programs are running independently, I am manually executing the trades. What I want to do is something like sending a pipe delimited message from program A to program B, parse the message and execute it.

How relevant is latency to your trading strategy?  If you can survive
executing your trades manually, then even a high-latency (i.e. slow)
automated solution should be more than adequate.

> I have tried using named pipe where I have made the QuickFix (program B) as the pipe server and c++ client (program A), but when I initiate the pipe server it is blocking run() method of quickfix. I am thinking of creating a threadpool and use pipe server.

In this scheme, how do you get your fills back to your trading engine
from your FIX engine?

If you do go this route, I would think you would want just a single
additional thread in your FIX engine (program B) that services
the pipe, rather than a thread pool.

> If any of you implemented a automated execution of trades using quickfix reducing latency and can share some ideas, I would be grateful.

The big questions that will affect your choice of architecture are:

a)  What kind of stability to you need?  How error-free do your
applications need to be?

b)  What kind of recoverability do you need?  What happens
if your application crashes?  What happens if the disk drive
supporting your application dies?

c)  Does a database figure into any of this?  For example,
are you writing your fills to either a light-weight or industrial-
strength database?

d)  How low-latency does your processing need to be?

e)  What is your "budget"?  I.e., what kind of development
man-hours (or hired resources) can you devote to building
this?

f)  What operating system hosts your trading engine?  Do
I assume correctly that you are using Linux?  (You mention
named pipes.)

If the answer to "e" is low budget, and the answer to "b" is
no real support for recovery (Note, these go together: adding
robust recovery will increase your budget.), then scheme "1,"
a single, integrated trading engine, is probably the cheapest,
shortest path to getting something up and running (but not
necessarily the most solid platform o build on in the future).

In this scheme you might have your main application "run"
loop (event loop) wait on an event queue (maybe a condition
variable, assuming Linux and pthreads).  You would have a
separate price-feed thread listening to pricing information
coming from the exchange.  For the sake of argument, let's
say your trading strategy is such that a new quote from the
exchange causes your trading logic to decide to issue a
new order.  Your price-feed thread could then post this new
order to the queue processed by your main application "run"
loop.

This is a simple scheme in which you choose to have your main
run loop be the place where you integrate your non-QuickFIX
events (e.g., price feed, console or gui input) with the QuickFIX
side of your application.

You avoid the complication of running multiple processes and
interprocess communication.  (But you lose modularity and
flexibility.)  It's doesn't introduce much extra latency.  (You do
introduce one additional queue, which I think it should be possible
to get rid of, but it's in-process, so it doesn't add much.)

But again, the good choices for your architecture (there is rarely
a single best choice) will depend a lot on your specific requirements.

> Regards,
> Arup

Good luck.


K. Frank