Re: [Quickfix-developers] new hardware crashes my quickfix client

[Oren M. <or...@qu...>]

The MySQLStore shouldn't interfere with receiving messages.  Only=20
messages that are sent out are added to storage.  It does need to=20
update the sequence number, so that may have some affect.  If you are=20
using MySQL to log messages, that could slow you down when logging the=20=

incoming messages.

More then likely however it is whatever you implemented in fromApp that=20=

is doing most of the processing.  If you are using the SocketInitiator,=20=

everything is happening in one thread, and that thread needs to pull=20
messages off the socket, parse them, and do all of your applications=20
processing.  This can leave the socket neglected for some time.

I'm guessing if you are using 50-70% of your CPU normally, then a=20
resend request for any significant period range would likely peg you to=20=

100% meaning you can't process it fast enough so you would eventually=20
fall behind.  You should probably benchmark your application to find=20
out where your bottlenecks are.  Running some tests with a FileStore=20
should also help you indicate if the MySQLStore is an issue (yes,=20
FileStore is *much* faster).

If you are expecting high traffic (like with these resend requests) and=20=

have significant processing, the ThreadedSocketInitiator would be a=20
better choice.  It will dedicate an entire thread to pulling messages=20
off the socket, decreasing the likely hood that it buffer will=20
overload.  All the session management (such as storage) and application=20=

logic occur in a different thread.  It is for these high=20
traffic/processing scenarios that the ThreadedSocketInitiator was=20
specifically designed.  If you wrap your application in a=20
SynchronizedApplication, you shouldn't have to alter any of your code=20
to make it thread safe.

BTW, how large are these messages and at what rate are you receiving=20
them?

--oren

On Oct 12, 2004, at 3:17 PM, Clark Sims wrote:

> I am using quickfix on a linux box. The box is a 1.8 gigahrtz pentium.=20=

> I am using debian linux, "Woody".
>  =A0
> SLK "upgraded" their hardware last night. On the new hardware my fix=20=

> client crashes everytime I send a resend request. The new SLK boxes=20
> spit out messages too fast for me to read them all.
> =A0
> What is a convient fix ?
> =A0
> All I can of is hacking into the quickfix code, and looking at the=20
> socket open statements. I could use setsockopt, (or whatever that=20
> fucntion is) to increase the buffer size.
> =A0
> In the mean time I told them to switch me back to the old hardware.=20
> Fortunately they kept some of the old boxes, and are using them as=20
> backup servers.
> =A0
> I also wondered what about mysql? Is that slowing me down? I am using=20=

> mysql to store all messages. I am typically using about 50 - 70% of my=20=

> cpu intraday. This is a lot higher than I expected. Or perhaps, I just=20=

> need to optimize the mysql database? LOL I haven't found the answer=20
> for that yet.
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com