quickfix-developers — QuickFIX developers forum

Re: [Quickfix-developers] python api on windows?

[Oren M. <or...@qu...>]

Well, when we first generated it we tried compiling under windows and 
saw similar problems.  Instead of holding up the release we labeled it 
linux only.  No one (until now) has really inquired about windows 
support so we just haven't gotten around to it.

--oren

On Feb 24, 2005, at 1:34 PM, russ wrote:

> QuickFIX Documentation: 
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ: 
> http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
>
> Hello All,  I noticed that the Python api doesn't seem to be compiled 
> for the windows binary packages, and the release notes for 1.8 mention 
> that it works for linux, but nothing about windows.  In my preliminary 
> attempts to compile the swig file under VC++6.0, I'm getting a lot of 
> errors like the following:
>  error C2872: 'INT' : ambiguous symbol
>  error C2872: 'CHAR' : ambiguous symbol
>
> Does anyone have any idea what's happening?  Are there any reasons why 
> the swig interface wouldn't work under windows?
> Thanks for the help, -Russ
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real 
> users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>

Re: [Quickfix-developers] Session::next/nextQueued deep recursion

[Oren M. <or...@qu...>]

Yeah definitely.  It is in fact the very next thing I was going to 
begin working on.  I'll post to the list when a fix is available.  
Shouldn't take too long I think/hope.

--oren

On Feb 25, 2005, at 3:34 PM, Caleb Epstein wrote:

> On Fri, 21 Jan 2005 00:17:35 -0600, Oren Miller 
> <or...@qu...> wrote:
>> Yeah, next is a recursive call (which in turn calls nextQueued, which
>> can in turn call next again) so I can see how this could happen.  We
>> may need to redesign this in a non-recursive way so it can better
>> handle arbitrarily large queues.
>
> We've had this happen again in production.  I don't see any progress
> towards fixing this in CVS.  Is it still on the radar screen?
>
> -- 
> Caleb Epstein
> caleb dot epstein at gmail dot com
>

Re: [Quickfix-developers] Session::next/nextQueued deep recursion

[Caleb E. <cal...@gm...>]

On Fri, 21 Jan 2005 00:17:35 -0600, Oren Miller <or...@qu...> wrote:
> Yeah, next is a recursive call (which in turn calls nextQueued, which
> can in turn call next again) so I can see how this could happen.  We
> may need to redesign this in a non-recursive way so it can better
> handle arbitrarily large queues.

We've had this happen again in production.  I don't see any progress
towards fixing this in CVS.  Is it still on the radar screen?

-- 
Caleb Epstein
caleb dot epstein at gmail dot com

[Quickfix-developers] RE: Infinite loop when receiving bad message in threaded acceptor

[Yihu F. <Yih...@re...>]

This works.

=20

In case you get a message with negative body length like 9=3D-10, it
should have if (length > 0) instead.

=20

Thanks,

=20

-Yihu

=20

  _____ =20

From: Oren Miller [mailto:or...@qu...]=20
Sent: Thursday, February 24, 2005 2:18 PM
To: Yihu Fang; qui...@li...
Subject: Re: Infinite loop when receiving bad message in threaded
acceptor

=20

I ended up writing the catch block as follows:

=20

if( length )

    m_buffer.erase( 0, pos + length )

else

  m_buffer.erase();

=20

I added a unit test to verify that passing the example string you
supplied throws an exception the first time, then returns false on the
second pass.  The reason I do the check for length is that it could save
some resend requests in case there are other messages in the buffer that
might be cleared.  Could be a significant factor in a high frequency
system if the buffer builds up. Please let me know if your tests
indicate that this code satisfies the error case you reported.

=20

--oren

	----- Original Message -----=20

	From: Yihu Fang <mailto:Yih...@re...> =20

	To: Oren Miller <mailto:or...@qu...>  ;
qui...@li...=20

	Sent: Tuesday, February 22, 2005 7:01 PM

	Subject: RE: Infinite loop when receiving bad message in
threaded acceptor

	=20

	Oren,

	=20

	Yes, after the fix to empty m_buffer correctly, QF can continue
to read the next message without disconnect the session.

	=20

	The only annoying thing is that
ThreadedSocketConnection::readMessage() gets an empty string so the log
could not record the garbled incoming message. It would record an event
error "BodyLength or CheckSum missing" because of the empty string. This
could be misleading.

	=20

	Thanks,

	=20

	-Yihu

	=20

=09
  _____ =20


	From: Oren Miller [mailto:or...@qu...]=20
	Sent: Tuesday, February 22, 2005 5:54 PM
	To: Yihu Fang; qui...@li...
	Subject: Re: Infinite loop when receiving bad message in
threaded acceptor

	=20

	Garbled messages should be ignored, so it shouldn't cause a
termination of the session. If it goes on to continue reading the
buffer, then that is the correct behavior according to the FPL test
cases.

	=20

	--oren

		----- Original Message -----=20

		From: Yihu Fang <mailto:Yih...@re...> =20

		To: Yihu Fang <mailto:Yih...@re...>  ; Oren
Miller <mailto:or...@qu...>  ;
qui...@li...=20

		Sent: Tuesday, February 22, 2005 2:45 PM

		Subject: RE: Infinite loop when receiving bad message in
threaded acceptor

		=20

		Hi,

		=20

		To stop the infinite loop, in catch(MessageParseError
&e) of Parser::readFixMessage(), the m_buffer should be emptied. Because
pos and length are zero, the bad message in m_buffer is reused which
causes the infinite loop.

		=20

		QuickFIX 1.9.4

		117,118c117

		<     // m_buffer.erase( 0, pos + length );

		<         m_buffer.erase();

		---

		>     m_buffer.erase( 0, pos + length );

		=20

		Whether QF is going to decide to terminate the session
or not should be done in the ThreadedSocketConnection.

		=20

		Thanks,

		=20

		-Yihu

		=20

	=09
  _____ =20


		From: Yihu Fang=20
		Sent: Tuesday, February 22, 2005 2:09 PM
		To: 'Oren Miller';
qui...@li...
		Subject: Infinite loop when receiving bad message in
threaded acceptor

		=20

		Oren,

		=20

		When a threaded acceptor receives a garbled message,
e.g. invalid length field, during an established FIX session, the
QuickFIX engine runs into an infinite loop.

		=20

		An example garbles message is like this,
8=3DTEST9=3DTEST35=3DTEST49=3DSS156=3DRORE34=3D352=3D20050222-16:45:5310=3D=
TEST

		=20

		Parser::extractLength() throws a MessageParserError
exception because of field 9=3DTEST.

		=20

		ThreadSocketConnection::readMessage() catches the
exception but it does nothing and returns true.

		=20

		This causes an infinite loop in the
ThreadedSocketConnection::readQueue(). An empty string is passed to
Session::next(), an InvalidMessage exception is thrown. However, because
the session is in a logon on state, the session is not disconnected, but
keeps going.

		=20

		The session should be disconnected in the exception
catch of either MessageParserError or InvalidMessage.

		=20

		Thank you,

		=20

		-Yihu

	=09
	=09
=09
-----------------------------------------------------------------
		Visit our Internet site at http://www.reuters.com
	=09
		Get closer to the financial markets with Reuters
Messaging - for more
		information and to register, visit
http://www.reuters.com/messaging
	=09
		Any views expressed in this message are those of the
individual
		sender, except where the sender specifically states them
to be
		the views of Reuters Ltd.

=09
=09
=09
-----------------------------------------------------------------
	Visit our Internet site at http://www.reuters.com
=09
	Get closer to the financial markets with Reuters Messaging - for
more
	information and to register, visit
http://www.reuters.com/messaging
=09
	Any views expressed in this message are those of the individual
	sender, except where the sender specifically states them to be
	the views of Reuters Ltd.



-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

Get closer to the financial markets with Reuters Messaging - for more
information and to register, visit http://www.reuters.com/messaging

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

[Quickfix-developers] Re: Infinite loop when receiving bad message in threaded acceptor

[Oren M. <or...@qu...>]

I ended up writing the catch block as follows:

if( length )
    m_buffer.erase( 0, pos + length )
else
  m_buffer.erase();

I added a unit test to verify that passing the example string you =
supplied throws an exception the first time, then returns false on the =
second pass.  The reason I do the check for length is that it could save =
some resend requests in case there are other messages in the buffer that =
might be cleared.  Could be a significant factor in a high frequency =
system if the buffer builds up. Please let me know if your tests =
indicate that this code satisfies the error case you reported.

--oren
  ----- Original Message -----=20
  From: Yihu Fang=20
  To: Oren Miller ; qui...@li...=20
  Sent: Tuesday, February 22, 2005 7:01 PM
  Subject: RE: Infinite loop when receiving bad message in threaded =
acceptor


  Oren,

  =20

  Yes, after the fix to empty m_buffer correctly, QF can continue to =
read the next message without disconnect the session.

  =20

  The only annoying thing is that =
ThreadedSocketConnection::readMessage() gets an empty string so the log =
could not record the garbled incoming message. It would record an event =
error "BodyLength or CheckSum missing" because of the empty string. This =
could be misleading.

  =20

  Thanks,

  =20

  -Yihu

  =20


-------------------------------------------------------------------------=
-----

  From: Oren Miller [mailto:or...@qu...]=20
  Sent: Tuesday, February 22, 2005 5:54 PM
  To: Yihu Fang; qui...@li...
  Subject: Re: Infinite loop when receiving bad message in threaded =
acceptor

  =20

  Garbled messages should be ignored, so it shouldn't cause a =
termination of the session. If it goes on to continue reading the =
buffer, then that is the correct behavior according to the FPL test =
cases.

  =20

  --oren

    ----- Original Message -----=20

    From: Yihu Fang=20

    To: Yihu Fang ; Oren Miller ; =
qui...@li...=20

    Sent: Tuesday, February 22, 2005 2:45 PM

    Subject: RE: Infinite loop when receiving bad message in threaded =
acceptor

    =20

    Hi,

    =20

    To stop the infinite loop, in catch(MessageParseError &e) of =
Parser::readFixMessage(), the m_buffer should be emptied. Because pos =
and length are zero, the bad message in m_buffer is reused which causes =
the infinite loop.

    =20

    QuickFIX 1.9.4

    117,118c117

    <     // m_buffer.erase( 0, pos + length );

    <         m_buffer.erase();

    ---

    >     m_buffer.erase( 0, pos + length );

    =20

    Whether QF is going to decide to terminate the session or not should =
be done in the ThreadedSocketConnection.

    =20

    Thanks,

    =20

    -Yihu

    =20


-------------------------------------------------------------------------=
---

    From: Yihu Fang=20
    Sent: Tuesday, February 22, 2005 2:09 PM
    To: 'Oren Miller'; qui...@li...
    Subject: Infinite loop when receiving bad message in threaded =
acceptor

    =20

    Oren,

    =20

    When a threaded acceptor receives a garbled message, e.g. invalid =
length field, during an established FIX session, the QuickFIX engine =
runs into an infinite loop.

    =20

    An example garbles message is like this, =
8=3DTEST9=3DTEST35=3DTEST49=3DSS156=3DRORE34=3D352=3D20050222-16:45:5310=3D=
TEST

    =20

    Parser::extractLength() throws a MessageParserError exception =
because of field 9=3DTEST.

    =20

    ThreadSocketConnection::readMessage() catches the exception but it =
does nothing and returns true.

    =20

    This causes an infinite loop in the =
ThreadedSocketConnection::readQueue(). An empty string is passed to =
Session::next(), an InvalidMessage exception is thrown. However, because =
the session is in a logon on state, the session is not disconnected, but =
keeps going.

    =20

    The session should be disconnected in the exception catch of either =
MessageParserError or InvalidMessage.

    =20

    Thank you,

    =20

    -Yihu



    -----------------------------------------------------------------
    Visit our Internet site at http://www.reuters.com

    Get closer to the financial markets with Reuters Messaging - for =
more
    information and to register, visit http://www.reuters.com/messaging

    Any views expressed in this message are those of the individual
    sender, except where the sender specifically states them to be
    the views of Reuters Ltd.



  -----------------------------------------------------------------
  Visit our Internet site at http://www.reuters.com

  Get closer to the financial markets with Reuters Messaging - for more
  information and to register, visit http://www.reuters.com/messaging

  Any views expressed in this message are those of the individual
  sender, except where the sender specifically states them to be
  the views of Reuters Ltd.

[Quickfix-developers] python api on windows?

[russ <ru...@ag...>]

Hello All,  I noticed that the Python api doesn't seem to be compiled 
for the windows binary packages, and the release notes for 1.8 mention 
that it works for linux, but nothing about windows.  In my preliminary 
attempts to compile the swig file under VC++6.0, I'm getting a lot of 
errors like the following:
  error C2872: 'INT' : ambiguous symbol
  error C2872: 'CHAR' : ambiguous symbol

Does anyone have any idea what's happening?  Are there any reasons why 
the swig interface wouldn't work under windows? 

Thanks for the help, -Russ

[Quickfix-developers] SIMC Conference

[Oren M. <or...@qu...>]

Just a note that Jim Downs will be doing a presentation on QuickFIX at the 
SIMC Conference on Open Source in the Securities Industry.  I will not be be 
personally present this year, but Jim Downs will be there and I will be on 
speaker phone during his presentation.  The conference is being held in New 
York.

Thank you for everyone who volunteered to share their experiences 
implementing QuickFIX.  It was the SIMC, not us, who made the final decision 
on who would present.  Last year it was Reuters. This year Jon Dahl will be 
speaking on implementing QuickFIX at the Chicago Mercantile Exchange.  For 
anyone interested in attending, you can find more information at the website 
listed below.

http://www.simc-inc.org/archive0405/opensource/index.htm

--oren 

RE: [Quickfix-developers] No Java Please! (gij: unrecognized option -- `-cp')

[Howard E. <Ho...@Pi...>]

Yes.  My JAVA_HOME is unset.  This does not appear to have any effect.=20

=20

________________________________

From: Oren Miller [mailto:or...@qu...]=20
Sent: Wednesday, February 23, 2005 12:22 PM
To: Howard Engelhart; qui...@li...
Subject: Re: [Quickfix-developers] No Java Please! (gij: unrecognized
option -- `-cp')

=20

Have you tried unsetting your JAVA_HOME environment variable?

=20

--oren

	----- Original Message -----=20

	From: Howard Engelhart <mailto:Ho...@Pi...> =20

	To: qui...@li...=20

	Sent: Wednesday, February 23, 2005 10:56 AM

	Subject: [Quickfix-developers] No Java Please! (gij:
unrecognized option -- `-cp')

	=20

	Going back to version 1.7.0 we have been unable to get through
basic builds of quickfix due to various java related problems.  Since we
do not use java in any capacity, it would be useful if there was a way
to build quickfix without any java support.  Our latest build error is
listed below.

	=20

	Our environment

	--------------

	=20

	Dell 1850 server running RedHat Linux ES 3.0

	g++ (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-49)

	stlport 4.6.2

	=20

	Our build

	--------------

	./bootstrap

	./configure --with-stlport=3D/usr/local --disable-callstack

	make

	=20

	=20

	I have attempted to modify the Makefile under quickfix/src to
remove java portions of the build myself, however, either way, the error
below occurs.

	=20

	107c107

	< JAVA_DIR =3D java

	---

	> JAVA_DIR =3D=20

	177c177

	< DIST_SUBDIRS =3D C++ java python

	---

	> DIST_SUBDIRS =3D C++ python

	476d475

	<       bash ./build.sh

	479d477

	<       bash ./build.sh clean=20

	=20

	=20

	=20

	Screen scrape of error:

	=20

	bash ./build.sh

	libgcj-java-placeholder.sh

	=20

	This script is a placeholder for the /usr/bin/java

	master link required by jpackage.org conventions.  libgcj's

	rmiregistry, rmic and jar tools are now slave symlinks to these

	masters, and are managed by the alternatives(8) system.

	=20

	This change was necessary because the rmiregistry, rmic and jar
tools

	installed by previous versions of libgcj conflicted with
symlinks

	installed by jpackage.org JVM packages.

	=20

	This script was designed to be overridden by the supported RHEL3
JRE

	packages, java-1.4.2-bea and java-1.4.2-ibm.  It is installed as
an

	alternative symlink as /usr/bin/java.  It will override a
third-party

	(non-RHEL3) JRE's java command if the JRE's bin directory is
listed

	after /usr/bin in PATH.  In that case, it is recommended that
the

	third-party JRE's bin directory be listed first in PATH instead.

	=20

	gij: unrecognized option -- `-cp'

	Try `gij --help' for more information.

	make[3]: *** [all-local] Error 1

	make[3]: Leaving directory `/usr/local/src/quickfix/src'

	make[2]: *** [all-recursive] Error 1

	make[2]: Leaving directory `/usr/local/src/quickfix/src'

	make[1]: *** [all-recursive] Error 1

	make[1]: Leaving directory `/usr/local/src/quickfix'

	make: *** [all] Error 2

	[root@dvsrcs01 quickfix]# gij -help

	=20

	=20

	=20

	=20

Re: [Quickfix-developers] No Java Please! (gij: unrecognized option -- `-cp')

[Oren M. <or...@qu...>]

Have you tried unsetting your JAVA_HOME environment variable?

--oren
  ----- Original Message -----=20
  From: Howard Engelhart=20
  To: qui...@li...=20
  Sent: Wednesday, February 23, 2005 10:56 AM
  Subject: [Quickfix-developers] No Java Please! (gij: unrecognized =
option -- `-cp')


  Going back to version 1.7.0 we have been unable to get through basic =
builds of quickfix due to various java related problems.  Since we do =
not use java in any capacity, it would be useful if there was a way to =
build quickfix without any java support.  Our latest build error is =
listed below.

  =20

  Our environment

  --------------

  =20

  Dell 1850 server running RedHat Linux ES 3.0

  g++ (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-49)

  stlport 4.6.2

  =20

  Our build

  --------------

  ./bootstrap

  ./configure --with-stlport=3D/usr/local --disable-callstack

  make

  =20

  =20

  I have attempted to modify the Makefile under quickfix/src to remove =
java portions of the build myself, however, either way, the error below =
occurs.

  =20

  107c107

  < JAVA_DIR =3D java

  ---

  > JAVA_DIR =3D=20

  177c177

  < DIST_SUBDIRS =3D C++ java python

  ---

  > DIST_SUBDIRS =3D C++ python

  476d475

  <       bash ./build.sh

  479d477

  <       bash ./build.sh clean=20

  =20

  =20

  =20

  Screen scrape of error:

  =20

  bash ./build.sh

  libgcj-java-placeholder.sh

  =20

  This script is a placeholder for the /usr/bin/java

  master link required by jpackage.org conventions.  libgcj's

  rmiregistry, rmic and jar tools are now slave symlinks to these

  masters, and are managed by the alternatives(8) system.

  =20

  This change was necessary because the rmiregistry, rmic and jar tools

  installed by previous versions of libgcj conflicted with symlinks

  installed by jpackage.org JVM packages.

  =20

  This script was designed to be overridden by the supported RHEL3 JRE

  packages, java-1.4.2-bea and java-1.4.2-ibm.  It is installed as an

  alternative symlink as /usr/bin/java.  It will override a third-party

  (non-RHEL3) JRE's java command if the JRE's bin directory is listed

  after /usr/bin in PATH.  In that case, it is recommended that the

  third-party JRE's bin directory be listed first in PATH instead.

  =20

  gij: unrecognized option -- `-cp'

  Try `gij --help' for more information.

  make[3]: *** [all-local] Error 1

  make[3]: Leaving directory `/usr/local/src/quickfix/src'

  make[2]: *** [all-recursive] Error 1

  make[2]: Leaving directory `/usr/local/src/quickfix/src'

  make[1]: *** [all-recursive] Error 1

  make[1]: Leaving directory `/usr/local/src/quickfix'

  make: *** [all] Error 2

  [root@dvsrcs01 quickfix]# gij -help

  =20

  =20

  =20

  =20

[Quickfix-developers] No Java Please! (gij: unrecognized option -- `-cp')

[Howard E. <Ho...@Pi...>]

Going back to version 1.7.0 we have been unable to get through basic
builds of quickfix due to various java related problems.  Since we do
not use java in any capacity, it would be useful if there was a way to
build quickfix without any java support.  Our latest build error is
listed below.

=20

Our environment

--------------

=20

Dell 1850 server running RedHat Linux ES 3.0

g++ (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-49)

stlport 4.6.2

=20

Our build

--------------

./bootstrap

./configure --with-stlport=3D/usr/local --disable-callstack

make

=20

=20

I have attempted to modify the Makefile under quickfix/src to remove
java portions of the build myself, however, either way, the error below
occurs.

=20

107c107

< JAVA_DIR =3D java

---

> JAVA_DIR =3D=20

177c177

< DIST_SUBDIRS =3D C++ java python

---

> DIST_SUBDIRS =3D C++ python

476d475

<       bash ./build.sh

479d477

<       bash ./build.sh clean=20

=20

=20

=20

Screen scrape of error:

=20

bash ./build.sh

libgcj-java-placeholder.sh

=20

This script is a placeholder for the /usr/bin/java

master link required by jpackage.org conventions.  libgcj's

rmiregistry, rmic and jar tools are now slave symlinks to these

masters, and are managed by the alternatives(8) system.

=20

This change was necessary because the rmiregistry, rmic and jar tools

installed by previous versions of libgcj conflicted with symlinks

installed by jpackage.org JVM packages.

=20

This script was designed to be overridden by the supported RHEL3 JRE

packages, java-1.4.2-bea and java-1.4.2-ibm.  It is installed as an

alternative symlink as /usr/bin/java.  It will override a third-party

(non-RHEL3) JRE's java command if the JRE's bin directory is listed

after /usr/bin in PATH.  In that case, it is recommended that the

third-party JRE's bin directory be listed first in PATH instead.

=20

gij: unrecognized option -- `-cp'

Try `gij --help' for more information.

make[3]: *** [all-local] Error 1

make[3]: Leaving directory `/usr/local/src/quickfix/src'

make[2]: *** [all-recursive] Error 1

make[2]: Leaving directory `/usr/local/src/quickfix/src'

make[1]: *** [all-recursive] Error 1

make[1]: Leaving directory `/usr/local/src/quickfix'

make: *** [all] Error 2

[root@dvsrcs01 quickfix]# gij -help

=20

=20

=20

=20

[Quickfix-developers] RE: Infinite loop when receiving bad message in threaded acceptor

[Yihu F. <Yih...@re...>]

Oren,

=20

Yes, after the fix to empty m_buffer correctly, QF can continue to read
the next message without disconnect the session.

=20

The only annoying thing is that ThreadedSocketConnection::readMessage()
gets an empty string so the log could not record the garbled incoming
message. It would record an event error "BodyLength or CheckSum missing"
because of the empty string. This could be misleading.

=20

Thanks,

=20

-Yihu

=20

  _____ =20

From: Oren Miller [mailto:or...@qu...]=20
Sent: Tuesday, February 22, 2005 5:54 PM
To: Yihu Fang; qui...@li...
Subject: Re: Infinite loop when receiving bad message in threaded
acceptor

=20

Garbled messages should be ignored, so it shouldn't cause a termination
of the session. If it goes on to continue reading the buffer, then that
is the correct behavior according to the FPL test cases.

=20

--oren

	----- Original Message -----=20

	From: Yihu Fang <mailto:Yih...@re...> =20

	To: Yihu Fang <mailto:Yih...@re...>  ; Oren Miller
<mailto:or...@qu...>  ;
qui...@li...=20

	Sent: Tuesday, February 22, 2005 2:45 PM

	Subject: RE: Infinite loop when receiving bad message in
threaded acceptor

	=20

	Hi,

	=20

	To stop the infinite loop, in catch(MessageParseError &e) of
Parser::readFixMessage(), the m_buffer should be emptied. Because pos
and length are zero, the bad message in m_buffer is reused which causes
the infinite loop.

	=20

	QuickFIX 1.9.4

	117,118c117

	<     // m_buffer.erase( 0, pos + length );

	<         m_buffer.erase();

	---

	>     m_buffer.erase( 0, pos + length );

	=20

	Whether QF is going to decide to terminate the session or not
should be done in the ThreadedSocketConnection.

	=20

	Thanks,

	=20

	-Yihu

	=20

=09
  _____ =20


	From: Yihu Fang=20
	Sent: Tuesday, February 22, 2005 2:09 PM
	To: 'Oren Miller'; qui...@li...
	Subject: Infinite loop when receiving bad message in threaded
acceptor

	=20

	Oren,

	=20

	When a threaded acceptor receives a garbled message, e.g.
invalid length field, during an established FIX session, the QuickFIX
engine runs into an infinite loop.

	=20

	An example garbles message is like this,
8=3DTEST9=3DTEST35=3DTEST49=3DSS156=3DRORE34=3D352=3D20050222-16:45:5310=3D=
TEST

	=20

	Parser::extractLength() throws a MessageParserError exception
because of field 9=3DTEST.

	=20

	ThreadSocketConnection::readMessage() catches the exception but
it does nothing and returns true.

	=20

	This causes an infinite loop in the
ThreadedSocketConnection::readQueue(). An empty string is passed to
Session::next(), an InvalidMessage exception is thrown. However, because
the session is in a logon on state, the session is not disconnected, but
keeps going.

	=20

	The session should be disconnected in the exception catch of
either MessageParserError or InvalidMessage.

	=20

	Thank you,

	=20

	-Yihu

=09
=09
=09
-----------------------------------------------------------------
	Visit our Internet site at http://www.reuters.com
=09
	Get closer to the financial markets with Reuters Messaging - for
more
	information and to register, visit
http://www.reuters.com/messaging
=09
	Any views expressed in this message are those of the individual
	sender, except where the sender specifically states them to be
	the views of Reuters Ltd.



-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

Get closer to the financial markets with Reuters Messaging - for more
information and to register, visit http://www.reuters.com/messaging

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

[Quickfix-developers] Re: Infinite loop when receiving bad message in threaded acceptor

[Oren M. <or...@qu...>]

Garbled messages should be ignored, so it shouldn't cause a termination =
of the session. If it goes on to continue reading the buffer, then that =
is the correct behavior according to the FPL test cases.

--oren
  ----- Original Message -----=20
  From: Yihu Fang=20
  To: Yihu Fang ; Oren Miller ; =
qui...@li...=20
  Sent: Tuesday, February 22, 2005 2:45 PM
  Subject: RE: Infinite loop when receiving bad message in threaded =
acceptor


  Hi,

  =20

  To stop the infinite loop, in catch(MessageParseError &e) of =
Parser::readFixMessage(), the m_buffer should be emptied. Because pos =
and length are zero, the bad message in m_buffer is reused which causes =
the infinite loop.

  =20

  QuickFIX 1.9.4

  117,118c117

  <     // m_buffer.erase( 0, pos + length );

  <         m_buffer.erase();

  ---

  >     m_buffer.erase( 0, pos + length );

  =20

  Whether QF is going to decide to terminate the session or not should =
be done in the ThreadedSocketConnection.

  =20

  Thanks,

  =20

  -Yihu

  =20


-------------------------------------------------------------------------=
-----

  From: Yihu Fang=20
  Sent: Tuesday, February 22, 2005 2:09 PM
  To: 'Oren Miller'; qui...@li...
  Subject: Infinite loop when receiving bad message in threaded acceptor

  =20

  Oren,

  =20

  When a threaded acceptor receives a garbled message, e.g. invalid =
length field, during an established FIX session, the QuickFIX engine =
runs into an infinite loop.

  =20

  An example garbles message is like this, =
8=3DTEST9=3DTEST35=3DTEST49=3DSS156=3DRORE34=3D352=3D20050222-16:45:5310=3D=
TEST

  =20

  Parser::extractLength() throws a MessageParserError exception because =
of field 9=3DTEST.

  =20

  ThreadSocketConnection::readMessage() catches the exception but it =
does nothing and returns true.

  =20

  This causes an infinite loop in the =
ThreadedSocketConnection::readQueue(). An empty string is passed to =
Session::next(), an InvalidMessage exception is thrown. However, because =
the session is in a logon on state, the session is not disconnected, but =
keeps going.

  =20

  The session should be disconnected in the exception catch of either =
MessageParserError or InvalidMessage.

  =20

  Thank you,

  =20

  -Yihu



  -----------------------------------------------------------------
  Visit our Internet site at http://www.reuters.com

  Get closer to the financial markets with Reuters Messaging - for more
  information and to register, visit http://www.reuters.com/messaging

  Any views expressed in this message are those of the individual
  sender, except where the sender specifically states them to be
  the views of Reuters Ltd.

[Quickfix-developers] RE: Infinite loop when receiving bad message in threaded acceptor

[Yihu F. <Yih...@re...>]

Hi,

=20

To stop the infinite loop, in catch(MessageParseError &e) of
Parser::readFixMessage(), the m_buffer should be emptied. Because pos
and length are zero, the bad message in m_buffer is reused which causes
the infinite loop.

=20

QuickFIX 1.9.4

117,118c117

<     // m_buffer.erase( 0, pos + length );

<         m_buffer.erase();

---

>     m_buffer.erase( 0, pos + length );

=20

Whether QF is going to decide to terminate the session or not should be
done in the ThreadedSocketConnection.

=20

Thanks,

=20

-Yihu

=20

  _____ =20

From: Yihu Fang=20
Sent: Tuesday, February 22, 2005 2:09 PM
To: 'Oren Miller'; qui...@li...
Subject: Infinite loop when receiving bad message in threaded acceptor

=20

Oren,

=20

When a threaded acceptor receives a garbled message, e.g. invalid length
field, during an established FIX session, the QuickFIX engine runs into
an infinite loop.

=20

An example garbles message is like this,
8=3DTEST9=3DTEST35=3DTEST49=3DSS156=3DRORE34=3D352=3D20050222-16:45:5310=3D=
TEST

=20

Parser::extractLength() throws a MessageParserError exception because of
field 9=3DTEST.

=20

ThreadSocketConnection::readMessage() catches the exception but it does
nothing and returns true.

=20

This causes an infinite loop in the
ThreadedSocketConnection::readQueue(). An empty string is passed to
Session::next(), an InvalidMessage exception is thrown. However, because
the session is in a logon on state, the session is not disconnected, but
keeps going.

=20

The session should be disconnected in the exception catch of either
MessageParserError or InvalidMessage.

=20

Thank you,

=20

-Yihu



-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

Get closer to the financial markets with Reuters Messaging - for more
information and to register, visit http://www.reuters.com/messaging

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

[Quickfix-developers] Infinite loop when receiving bad message in threaded acceptor

[Yihu F. <Yih...@re...>]

Oren,

=20

When a threaded acceptor receives a garbled message, e.g. invalid length
field, during an established FIX session, the QuickFIX engine runs into
an infinite loop.

=20

An example garbles message is like this,
8=3DTEST9=3DTEST35=3DTEST49=3DSS156=3DRORE34=3D352=3D20050222-16:45:5310=3D=
TEST

=20

Parser::extractLength() throws a MessageParserError exception because of
field 9=3DTEST.

=20

ThreadSocketConnection::readMessage() catches the exception but it does
nothing and returns true.

=20

This causes an infinite loop in the
ThreadedSocketConnection::readQueue(). An empty string is passed to
Session::next(), an InvalidMessage exception is thrown. However, because
the session is in a logon on state, the session is not disconnected, but
keeps going.

=20

The session should be disconnected in the exception catch of either
MessageParserError or InvalidMessage.

=20

Thank you,

=20

-Yihu



-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

Get closer to the financial markets with Reuters Messaging - for more
information and to register, visit http://www.reuters.com/messaging

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

RE: [Quickfix-developers] "Invalid tag number" error for proprietary field

[Yihu F. <Yih...@re...>]

If we are going to have a configuration option to turn on and off validatio=
n of user defined field, it would be perfect.

But until then, this is your short term solution.

Thanks,

-Yihu



-----Original Message-----
From: Joerg Thoennes [mailto:Joe...@ma...]=20
Sent: Wednesday, February 16, 2005 10:15 AM
To: Yihu Fang
Cc: dav...@ma...; Alvin Wang; qui...@li...urceforg=
e.net
Subject: Re: [Quickfix-developers] "Invalid tag number" error for proprieta=
ry field

> If you always want to pass through user defined fields without using
> data dictionary for validation, change the line 154 of
> DataDictionary.cpp of QuickFIX 1.9.4 as following
>=20
> 154c154
> <     if ( m_beginString.getValue().length() && field.getField() <
> FIELD::UserMin)

Instead of changing the code this could get a configuration option, e.g.

	ValidateUserDefinedFields=3DY (default)
	ValidateUserDefinedMessages=3DY (default)

I wonder what Oren thinks about this.

Cheers, J=F6rg

--=20
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen



-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

Get closer to the financial markets with Reuters Messaging - for more
information and to register, visit http://www.reuters.com/messaging

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

Re: [Quickfix-developers] "Invalid tag number" error for proprietary field

[Alvin W. <AW...@FF...>]

This will be a lot easier! Sometimes, it is hard to maintain counterparts'=
=20
proprietary fields since we do not need all of them. And if 2 counterparts=
=20
use the same field by coincidence, I have to create 2 XML files.=20






Joerg Thoennes <Joe...@ma...>
02/16/2005 10:14 AM

=20
        To:     Yihu Fang <Yih...@re...>
        cc:     dav...@ma..., Alvin Wang <AW...@FF...>,=20
qui...@li...
        bcc:=20
        Subject:        Re: [Quickfix-developers] "Invalid tag number" erro=
r for proprietary field


> If you always want to pass through user defined fields without using
> data dictionary for validation, change the line 154 of
> DataDictionary.cpp of QuickFIX 1.9.4 as following
>=20
> 154c154
> <     if ( m_beginString.getValue().length() && field.getField() <
> FIELD::UserMin)

Instead of changing the code this could get a configuration option, e.g.

                 ValidateUserDefinedFields=3DY (default)
                 ValidateUserDefinedMessages=3DY (default)

I wonder what Oren thinks about this.

Cheers, J=F6rg

--=20
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen



**********************************************************************
This e-mail message is intended solely for the use of the addressee. The me=
ssage
 may contain information that is privileged and confidential.  Disclosure to
anyone other than the intended recipient is prohibited. If you are not the
intended recipient, please do not disseminate, distribute or copy this
communication, by e-mail or otherwise. Instead, please notify us immediatel=
y by
return e-mail (including the original message with your reply) and then del=
ete
and discard all copies of the message.
We have taken precautions to minimize the risk of transmitting software vir=
uses
 but nevertheless advise you to carry out your own virus checks on any
attachment to  this  message.  We  accept  no  liability  for any loss or d=
amage
 caused by software viruses.
**********************************************************************

Re: [Quickfix-developers] "Invalid tag number" error for proprietary field

[Joerg T. <Joe...@ma...>]

> If you always want to pass through user defined fields without using
> data dictionary for validation, change the line 154 of
> DataDictionary.cpp of QuickFIX 1.9.4 as following
> 
> 154c154
> <     if ( m_beginString.getValue().length() && field.getField() <
> FIELD::UserMin)

Instead of changing the code this could get a configuration option, e.g.

	ValidateUserDefinedFields=Y (default)
	ValidateUserDefinedMessages=Y (default)

I wonder what Oren thinks about this.

Cheers, Jörg

-- 
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen

RE: [Quickfix-developers] "Invalid tag number" error for proprietary field

[Yihu F. <Yih...@re...>]

Hi,

=20

If you always want to pass through user defined fields without using
data dictionary for validation, change the line 154 of
DataDictionary.cpp of QuickFIX 1.9.4 as following

=20

154c154

<     if ( m_beginString.getValue().length() && field.getField() <
FIELD::UserMin)

---

>     if ( m_beginString.getValue().length() )

=20

Regards,

=20

-Yihu

=20

=20

=20

  _____ =20

From: qui...@li...
[mailto:qui...@li...] On Behalf Of
Dave Linaker
Sent: Wednesday, February 16, 2005 9:29 AM
To: 'Alvin Wang'; qui...@li...
Subject: RE: [Quickfix-developers] "Invalid tag number" error for
proprietary field

=20

Hi Alvin,

=20

You need to add the user defined tag (i.e.6602) to your DataDictionary.
Remember to add the definition for field and message.

=20

See http://www.quickfixengine.org/quickfix/doc/html/xmlschema.html

=20

Kind regards

=20

=20

Dave

	-----Original Message-----
	From: qui...@li...
[mailto:qui...@li...] On Behalf Of
Alvin Wang
	Sent: 16 February 2005 14:22
	To: qui...@li...
	Subject: [Quickfix-developers] "Invalid tag number" error for
proprietary field

Hi All,

A incoming FIX msg from counterparty contains their proprietary tag
(6602). QF rejected the message as
below:=20
(Message 107 Rejected: Invalid tag number:6602)

I use dictionary for repeating group. I wonder if this
is related to the rejection? How to fix the problem?

Thanks a lot!
Alvin

=09
=09
**********************************************************************
This e-mail message is intended solely for the use of the addressee. The
message may contain information that is privileged and confidential.
Disclosure to anyone other than the intended recipient is prohibited. If
you are not the intended recipient, please do not disseminate,
distribute or copy this communication, by e-mail or otherwise. Instead,
please notify us immediately by return e-mail (including the original
message with your reply) and then delete and discard all copies of the
message. We have taken precautions to minimize the risk of transmitting
software viruses but nevertheless advise you to carry out your own virus
checks on any attachment to this message. We accept no liability for any
loss or damage caused by software viruses.
**********************************************************************=20



-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

Get closer to the financial markets with Reuters Messaging - for more
information and to register, visit http://www.reuters.com/messaging

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

RE: [Quickfix-developers] "Invalid tag number" error for proprietary field

[Dave L. <dav...@ma...>]

Hi Alvin,
 
You need to add the user defined tag (i.e.6602) to your DataDictionary.
Remember to add the definition for field and message.
 
See http://www.quickfixengine.org/quickfix/doc/html/xmlschema.html
 
Kind regards
 
 
Dave

-----Original Message-----
From: qui...@li...
[mailto:qui...@li...] On Behalf Of
Alvin Wang
Sent: 16 February 2005 14:22
To: qui...@li...
Subject: [Quickfix-developers] "Invalid tag number" error for
proprietary field



Hi All,

A incoming FIX msg from counterparty contains their proprietary tag
(6602). QF rejected the message as
below: 
(Message 107 Rejected: Invalid tag number:6602)

I use dictionary for repeating group. I wonder if this
is related to the rejection? How to fix the problem?

Thanks a lot!
Alvin



**********************************************************************
This e-mail message is intended solely for the use of the addressee. The
message may contain information that is privileged and confidential.
Disclosure to anyone other than the intended recipient is prohibited. If
you are not the intended recipient, please do not disseminate,
distribute or copy this communication, by e-mail or otherwise. Instead,
please notify us immediately by return e-mail (including the original
message with your reply) and then delete and discard all copies of the
message. We have taken precautions to minimize the risk of transmitting
software viruses but nevertheless advise you to carry out your own virus
checks on any attachment to this message. We accept no liability for any
loss or damage caused by software viruses.
********************************************************************** 

[Quickfix-developers] "Invalid tag number" error for proprietary field

[Alvin W. <AW...@FF...>]

Hi All,
 
A incoming FIX msg from counterparty contains their proprietary tag 
(6602). QF rejected the message as
below:
(Message 107 Rejected: Invalid tag number:6602)
 
I use dictionary for repeating group. I wonder if this
is related to the rejection? How to fix the problem?
 
Thanks a lot!
Alvin



**********************************************************************
This e-mail message is intended solely for the use of the addressee. The message
 may contain information that is privileged and confidential.  Disclosure to
anyone other than the intended recipient is prohibited. If you are not the
intended recipient, please do not disseminate, distribute or copy this
communication, by e-mail or otherwise. Instead, please notify us immediately by
return e-mail (including the original message with your reply) and then delete
and discard all copies of the message.
We have taken precautions to minimize the risk of transmitting software viruses
 but nevertheless advise you to carry out your own virus checks on any
attachment to  this  message.  We  accept  no  liability  for any loss or damage
 caused by software viruses.
**********************************************************************

Re: [Quickfix-developers] fragmentation?

[Joerg T. <Joe...@ma...>]

Alvin Wang wrote:
> Oren, thanks a lot for your reply. 
> 
> As Joerg suggested, it would be nicer if QF can handle this transparently 
> (the user can configure the max message size in conf file of course). This 
> should be mechanic and will save developers a lot of time.

Alvin, this would not be part of the core QF API since we want to have it as lean as 
possible. But it would be a good idea to have an extra library to deal with common 
application level problems, e.g. ClOrdID chaining, fragmentation, etc.

I would like to encourage everyone to put suggestion for such a utility library into WikiFIX.

Cheers, Jörg

-- 
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen

Re: [Quickfix-developers] fragmentation?

[Alvin W. <AW...@FF...>]

Oren, thanks a lot for your reply.=20

As Joerg suggested, it would be nicer if QF can handle this transparently=20
(the user can configure the max message size in conf file of course). This =

should be mechanic and will save developers a lot of time.

Thanks again
Alvin






"Oren Miller" <or...@qu...>
02/15/2005 12:03 PM

=20
        To:     "Joerg Thoennes" <Joe...@ma...>, "Alvin Wang" <A=
Wa...@FF...>
        cc:     <qui...@li...>
        bcc:=20
        Subject:        Re: [Quickfix-developers] fragmentation?


Alvin,
=20
You would need to do this at the application level.  The session is not=20
going to be able to take a single message and break them up into multiple=20
messages.  You will need to prepare them before being sent.  Same goes for =

receiving messages.  You're application will need to hold on to the=20
received messages and process them when the final one comes through.  QF=20
itself doesn't have a restriction on the size of sent or received=20
messages, so the only time you should have to worry about this is if your=20
counterparty has some sort of restriction.
=20
--oren
----- Original Message -----=20
From: Alvin Wang=20
To: Joerg Thoennes=20
Cc: qui...@li...=20
Sent: Tuesday, February 15, 2005 10:34 AM
Subject: Re: [Quickfix-developers] fragmentation?


Excerpt from FIX44 vol5:=20
Fragmentation of Allocation Messages=20
FIX Allocation messages support fragmentation in a way similar to=20
MassQuote and the List Order messages.  If there are too many entries=20
within a repeating group to fit into one physical message, the entries can =

be continued in subsequent messages by repeating the principal message=20
reference and other required fields, then continuing with the repeating=20
group.  This is achieved by using an optional TotNoAllocs field (giving the=
 total number of AllocAccount details across the entire=20
allocation) that supplements the NoAllocs field (giving the number of Alloc=
Account details in a particular message=20
fragment).  The TotNoAllocs field is repeated with the same value in all fr=
agments of the batch.  For=20
example, an Allocation Instruction with 200 allocation account instances=20
could be fragmented across three messages - the first two containing=20
TotNoAllocs=3D200, NoAllocs=3D80 and the third TotNoAllocs=3D200, NoAllocs=
=3D40.=20
To help the receiver reconstitute the batch the Boolean field LastFragment =
is sent with a "Y" value in the last fragment.=20
For fragmented allocation events the receiving application must persist=20
state between messages to determine whether all instances of the repeating =

group have been received before acting on the instruction or processing=20
the report.=20
For this to work some key rules must be enforced:=20
1)        The sender must supply a consistent value for TotNoAllocs in all =

related fragments and must use the same primary message reference in all=20
fragments of the batch, e.g. AllocID in AllocationInstruction.=20
2)        The sender must ensure that fragments are transmitted in order=20
without intervening traffic.=20
3)        The NoAllocs group must reach capacity only in the last=20
fragment, and that message must contain LastFragment=3DY.=20
4)        The receiver must acknowledge every fragment received=20
(AllocationInstructionAck with AllocStatus=3D"received") and never reject a=
=20
non-last fragment; acknowledgment of the final fragment accepts or rejects =

the entire set.=20
There are a number of design suggestions for implementing fragmentation:=20
1)        Optional block-level fields supplied in early fragments need not =

be repeated in subsequent fragments.  If they are repeated and the values=20
are different, the receiver may choose to reject (on receiving the last=20
fragment) or to apply the last received value to the event.=20
2)        If a message supports multiple "Number of" groups, e.g.=20
NoOrders, NoExecs, and NoAllocs in AllocationInstruction, the sender may=20
distribute the array instances over any and all fragments, as long as the=20
NoAllocs group is not filled before the last fragment.=20
3)        The receiver must be able to abort collecting an incomplete=20
array ? either on expiration of a timer or the receipt of an unrelated=20
message from the same counterparty.=20

FIX Message=20
<Total number of> field=20
related <Number of> field=20
Prinicipal message reference=20
AllocationInstruction=20
TotNoAllocs=20
NoAllocs (78)=20
AllocID (70)=20
AllocationReport=20
TotNoAllocs=20
NoAllocs (78)=20
AllocReportID (755)
Maximum message size for fragmentation purposes can be determined by using =

the optional MaxMessageSize field in the Logon message or by mutual=20
agreement between counterparties.=20




Joerg Thoennes <Joe...@ma...>=20
02/15/2005 11:32 AM=20
       =20
        To:        Alvin Wang <AW...@FF...>=20
        cc:        qui...@li...=20
        bcc:        =20
        Subject:        Re: [Quickfix-developers] fragmentation?



Hi Alvin,

> How does quickfix handle fragmentation? Do I have to do it myself? Or=20
> quickfix will decide and handle for me? Thanks!

What do you mean? If a socket read returns an incomplete FIX message, and=20
the next socket=20
read returns the next part and whether QF handles this?

Cheers, J=F6rg

--=20
Joerg Thoennes
                               http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen


**********************************************************************=20
This e-mail message is intended solely for the use of the addressee. The=20
message may contain information that is privileged and confidential.=20
Disclosure to anyone other than the intended recipient is prohibited. If=20
you are not the intended recipient, please do not disseminate, distribute=20
or copy this communication, by e-mail or otherwise. Instead, please notify =

us immediately by return e-mail (including the original message with your=20
reply) and then delete and discard all copies of the message. We have=20
taken precautions to minimize the risk of transmitting software viruses=20
but nevertheless advise you to carry out your own virus checks on any=20
attachment to this message. We accept no liability for any loss or damage=20
caused by software viruses.=20
**********************************************************************=20

Re: [Quickfix-developers] fragmentation?

[Oren M. <or...@qu...>]

Alvin,

You would need to do this at the application level.  The session is not =
going to be able to take a single message and break them up into =
multiple messages.  You will need to prepare them before being sent.  =
Same goes for receiving messages.  You're application will need to hold =
on to the received messages and process them when the final one comes =
through.  QF itself doesn't have a restriction on the size of sent or =
received messages, so the only time you should have to worry about this =
is if your counterparty has some sort of restriction.

--oren
  ----- Original Message -----=20
  From: Alvin Wang=20
  To: Joerg Thoennes=20
  Cc: qui...@li...=20
  Sent: Tuesday, February 15, 2005 10:34 AM
  Subject: Re: [Quickfix-developers] fragmentation?



  Excerpt from FIX44 vol5:=20

  Fragmentation of Allocation Messages=20


  FIX Allocation messages support fragmentation in a way similar to =
MassQuote and the List Order messages.  If there are too many entries =
within a repeating group to fit into one physical message, the entries =
can be continued in subsequent messages by repeating the principal =
message reference and other required fields, then continuing with the =
repeating group.  This is achieved by using an optional TotNoAllocs =
field (giving the total number of AllocAccount details across the entire =
allocation) that supplements the NoAllocs field (giving the number of =
AllocAccount details in a particular message fragment).  The TotNoAllocs =
field is repeated with the same value in all fragments of the batch.  =
For example, an Allocation Instruction with 200 allocation account =
instances could be fragmented across three messages - the first two =
containing TotNoAllocs=3D200, NoAllocs=3D80 and the third =
TotNoAllocs=3D200, NoAllocs=3D40.  To help the receiver reconstitute the =
batch the Boolean field LastFragment is sent with a "Y" value in the =
last fragment.=20

  For fragmented allocation events the receiving application must =
persist state between messages to determine whether all instances of the =
repeating group have been received before acting on the instruction or =
processing the report.=20

  For this to work some key rules must be enforced:=20
  1)        The sender must supply a consistent value for TotNoAllocs in =
all related fragments and must use the same primary message reference in =
all fragments of the batch, e.g. AllocID in AllocationInstruction.=20
  2)        The sender must ensure that fragments are transmitted in =
order without intervening traffic.=20
  3)        The NoAllocs group must reach capacity only in the last =
fragment, and that message must contain LastFragment=3DY.=20
  4)        The receiver must acknowledge every fragment received =
(AllocationInstructionAck with AllocStatus=3D"received") and never =
reject a non-last fragment; acknowledgment of the final fragment accepts =
or rejects the entire set.=20


  There are a number of design suggestions for implementing =
fragmentation:=20
  1)        Optional block-level fields supplied in early fragments need =
not be repeated in subsequent fragments.  If they are repeated and the =
values are different, the receiver may choose to reject (on receiving =
the last fragment) or to apply the last received value to the event.=20
  2)        If a message supports multiple "Number of" groups, e.g. =
NoOrders, NoExecs, and NoAllocs in AllocationInstruction, the sender may =
distribute the array instances over any and all fragments, as long as =
the NoAllocs group is not filled before the last fragment.=20
  3)        The receiver must be able to abort collecting an incomplete =
array - either on expiration of a timer or the receipt of an unrelated =
message from the same counterparty.=20

        FIX Message  <Total number of> field  related <Number of> field  =
Prinicipal message reference =20
        AllocationInstruction  TotNoAllocs  NoAllocs (78)  AllocID (70)  =

        AllocationReport  TotNoAllocs  NoAllocs (78)  AllocReportID =
(755)=20



  Maximum message size for fragmentation purposes can be determined by =
using the optional MaxMessageSize field in the Logon message or by =
mutual agreement between counterparties.=20





       Joerg Thoennes <Joe...@ma...>=20
        02/15/2005 11:32 AM=20

              =20
                To:        Alvin Wang <AW...@FF...>=20
                cc:        qui...@li...=20
                bcc:        =20
                Subject:        Re: [Quickfix-developers] fragmentation? =




  Hi Alvin,

  > How does quickfix handle fragmentation? Do I have to do it myself? =
Or=20
  > quickfix will decide and handle for me? Thanks!

  What do you mean? If a socket read returns an incomplete FIX message, =
and the next socket=20
  read returns the next part and whether QF handles this?

  Cheers, J=F6rg

  --=20
  Joerg Thoennes
                                 http://macd.com
  Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
  Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen


  ********************************************************************** =
This e-mail message is intended solely for the use of the addressee. The =
message may contain information that is privileged and confidential. =
Disclosure to anyone other than the intended recipient is prohibited. If =
you are not the intended recipient, please do not disseminate, =
distribute or copy this communication, by e-mail or otherwise. Instead, =
please notify us immediately by return e-mail (including the original =
message with your reply) and then delete and discard all copies of the =
message. We have taken precautions to minimize the risk of transmitting =
software viruses but nevertheless advise you to carry out your own virus =
checks on any attachment to this message. We accept no liability for any =
loss or damage caused by software viruses. =
**********************************************************************=20

Re: [Quickfix-developers] fragmentation?

[Joerg T. <Joe...@ma...>]

Alvin Wang wrote:
> Excerpt from FIX44 vol5:
> 
> Fragmentation of Allocation Messages
> [...]

Ok, was not aware of that. Seems like an application level construct and is therefore not 
supported by the QuickFIX engine.

But is probably worth being added to some utility library.

Cheers, Jörg

-- 
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen

Re: [Quickfix-developers] fragmentation?

[Alvin W. <AW...@FF...>]

Excerpt from FIX44 vol5:

Fragmentation of Allocation Messages

FIX Allocation messages support fragmentation in a way similar to=20
MassQuote and the List Order messages.  If there are too many entries=20
within a repeating group to fit into one physical message, the entries can=
=20
be continued in subsequent messages by repeating the principal message=20
reference and other required fields, then continuing with the repeating=20
group.  This is achieved by using an optional TotNoAllocs field (giving the=
 total number of AllocAccount details across the entire=20
allocation) that supplements the NoAllocs field (giving the number of Alloc=
Account details in a particular message=20
fragment).  The TotNoAllocs field is repeated with the same value in all fr=
agments of the batch.  For=20
example, an Allocation Instruction with 200 allocation account instances=20
could be fragmented across three messages - the first two containing=20
TotNoAllocs=3D200, NoAllocs=3D80 and the third TotNoAllocs=3D200, NoAllocs=
=3D40.=20
To help the receiver reconstitute the batch the Boolean field LastFragment =
is sent with a "Y" value in the last fragment.
For fragmented allocation events the receiving application must persist=20
state between messages to determine whether all instances of the repeating=
=20
group have been received before acting on the instruction or processing=20
the report.=20
For this to work some key rules must be enforced:
1)      The sender must supply a consistent value for TotNoAllocs in all=20
related fragments and must use the same primary message reference in all=20
fragments of the batch, e.g. AllocID in AllocationInstruction.
2)      The sender must ensure that fragments are transmitted in order=20
without intervening traffic.
3)      The NoAllocs group must reach capacity only in the last fragment,=
=20
and that message must contain LastFragment=3DY.
4)      The receiver must acknowledge every fragment received=20
(AllocationInstructionAck with AllocStatus=3D"received") and never reject a=
=20
non-last fragment; acknowledgment of the final fragment accepts or rejects=
=20
the entire set.

There are a number of design suggestions for implementing fragmentation:
1)      Optional block-level fields supplied in early fragments need not=20
be repeated in subsequent fragments.  If they are repeated and the values=
=20
are different, the receiver may choose to reject (on receiving the last=20
fragment) or to apply the last received value to the event.
2)      If a message supports multiple "Number of" groups, e.g. NoOrders,=
=20
NoExecs, and NoAllocs in AllocationInstruction, the sender may distribute=
=20
the array instances over any and all fragments, as long as the NoAllocs=20
group is not filled before the last fragment.
3)      The receiver must be able to abort collecting an incomplete array=
=20
? either on expiration of a timer or the receipt of an unrelated message=20
from the same counterparty.

FIX Message
<Total number of> field
related <Number of> field
Prinicipal message reference
AllocationInstruction
TotNoAllocs
NoAllocs (78)
AllocID (70)
AllocationReport
TotNoAllocs
NoAllocs (78)
AllocReportID (755)

Maximum message size for fragmentation purposes can be determined by using=
=20
the optional MaxMessageSize field in the Logon message or by mutual=20
agreement between counterparties.






Joerg Thoennes <Joe...@ma...>
02/15/2005 11:32 AM

=20
        To:     Alvin Wang <AW...@FF...>
        cc:     qui...@li...
        bcc:=20
        Subject:        Re: [Quickfix-developers] fragmentation?


Hi Alvin,

> How does quickfix handle fragmentation? Do I have to do it myself? Or=20
> quickfix will decide and handle for me? Thanks!

What do you mean? If a socket read returns an incomplete FIX message, and=
=20
the next socket=20
read returns the next part and whether QF handles this?

Cheers, J=F6rg

--=20
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen



**********************************************************************
This e-mail message is intended solely for the use of the addressee. The me=
ssage
 may contain information that is privileged and confidential.  Disclosure to
anyone other than the intended recipient is prohibited. If you are not the
intended recipient, please do not disseminate, distribute or copy this
communication, by e-mail or otherwise. Instead, please notify us immediatel=
y by
return e-mail (including the original message with your reply) and then del=
ete
and discard all copies of the message.
We have taken precautions to minimize the risk of transmitting software vir=
uses
 but nevertheless advise you to carry out your own virus checks on any
attachment to  this  message.  We  accept  no  liability  for any loss or d=
amage
 caused by software viruses.
**********************************************************************