Thread: Re: [Quickfix-developers] fragmentation?
Brought to you by:
orenmnero
Menu
▾
▴
quickfix-developers
|
From: Alvin W. <AW...@FF...> - 2005-02-15 16:34:20
|
Excerpt from FIX44 vol5:
Fragmentation of Allocation Messages
FIX Allocation messages support fragmentation in a way similar to=20
MassQuote and the List Order messages. If there are too many entries=20
within a repeating group to fit into one physical message, the entries can=
=20
be continued in subsequent messages by repeating the principal message=20
reference and other required fields, then continuing with the repeating=20
group. This is achieved by using an optional TotNoAllocs field (giving the=
total number of AllocAccount details across the entire=20
allocation) that supplements the NoAllocs field (giving the number of Alloc=
Account details in a particular message=20
fragment). The TotNoAllocs field is repeated with the same value in all fr=
agments of the batch. For=20
example, an Allocation Instruction with 200 allocation account instances=20
could be fragmented across three messages - the first two containing=20
TotNoAllocs=3D200, NoAllocs=3D80 and the third TotNoAllocs=3D200, NoAllocs=
=3D40.=20
To help the receiver reconstitute the batch the Boolean field LastFragment =
is sent with a "Y" value in the last fragment.
For fragmented allocation events the receiving application must persist=20
state between messages to determine whether all instances of the repeating=
=20
group have been received before acting on the instruction or processing=20
the report.=20
For this to work some key rules must be enforced:
1) The sender must supply a consistent value for TotNoAllocs in all=20
related fragments and must use the same primary message reference in all=20
fragments of the batch, e.g. AllocID in AllocationInstruction.
2) The sender must ensure that fragments are transmitted in order=20
without intervening traffic.
3) The NoAllocs group must reach capacity only in the last fragment,=
=20
and that message must contain LastFragment=3DY.
4) The receiver must acknowledge every fragment received=20
(AllocationInstructionAck with AllocStatus=3D"received") and never reject a=
=20
non-last fragment; acknowledgment of the final fragment accepts or rejects=
=20
the entire set.
There are a number of design suggestions for implementing fragmentation:
1) Optional block-level fields supplied in early fragments need not=20
be repeated in subsequent fragments. If they are repeated and the values=
=20
are different, the receiver may choose to reject (on receiving the last=20
fragment) or to apply the last received value to the event.
2) If a message supports multiple "Number of" groups, e.g. NoOrders,=
=20
NoExecs, and NoAllocs in AllocationInstruction, the sender may distribute=
=20
the array instances over any and all fragments, as long as the NoAllocs=20
group is not filled before the last fragment.
3) The receiver must be able to abort collecting an incomplete array=
=20
? either on expiration of a timer or the receipt of an unrelated message=20
from the same counterparty.
FIX Message
<Total number of> field
related <Number of> field
Prinicipal message reference
AllocationInstruction
TotNoAllocs
NoAllocs (78)
AllocID (70)
AllocationReport
TotNoAllocs
NoAllocs (78)
AllocReportID (755)
Maximum message size for fragmentation purposes can be determined by using=
=20
the optional MaxMessageSize field in the Logon message or by mutual=20
agreement between counterparties.
Joerg Thoennes <Joe...@ma...>
02/15/2005 11:32 AM
=20
To: Alvin Wang <AW...@FF...>
cc: qui...@li...
bcc:=20
Subject: Re: [Quickfix-developers] fragmentation?
Hi Alvin,
> How does quickfix handle fragmentation? Do I have to do it myself? Or=20
> quickfix will decide and handle for me? Thanks!
What do you mean? If a socket read returns an incomplete FIX message, and=
=20
the next socket=20
read returns the next part and whether QF handles this?
Cheers, J=F6rg
--=20
Joerg Thoennes
http://macd.com
Tel.: +49 (0)241 44597-24 Macdonald Associates GmbH
Fax : +49 (0)241 44597-10 Lothringer Str. 52, D-52070 Aachen
**********************************************************************
This e-mail message is intended solely for the use of the addressee. The me=
ssage
may contain information that is privileged and confidential. Disclosure to
anyone other than the intended recipient is prohibited. If you are not the
intended recipient, please do not disseminate, distribute or copy this
communication, by e-mail or otherwise. Instead, please notify us immediatel=
y by
return e-mail (including the original message with your reply) and then del=
ete
and discard all copies of the message.
We have taken precautions to minimize the risk of transmitting software vir=
uses
but nevertheless advise you to carry out your own virus checks on any
attachment to this message. We accept no liability for any loss or d=
amage
caused by software viruses.
**********************************************************************
|
|
From: Alvin W. <AW...@FF...> - 2005-02-15 17:08:42
|
Oren, thanks a lot for your reply.=20
As Joerg suggested, it would be nicer if QF can handle this transparently=20
(the user can configure the max message size in conf file of course). This =
should be mechanic and will save developers a lot of time.
Thanks again
Alvin
"Oren Miller" <or...@qu...>
02/15/2005 12:03 PM
=20
To: "Joerg Thoennes" <Joe...@ma...>, "Alvin Wang" <A=
Wa...@FF...>
cc: <qui...@li...>
bcc:=20
Subject: Re: [Quickfix-developers] fragmentation?
Alvin,
=20
You would need to do this at the application level. The session is not=20
going to be able to take a single message and break them up into multiple=20
messages. You will need to prepare them before being sent. Same goes for =
receiving messages. You're application will need to hold on to the=20
received messages and process them when the final one comes through. QF=20
itself doesn't have a restriction on the size of sent or received=20
messages, so the only time you should have to worry about this is if your=20
counterparty has some sort of restriction.
=20
--oren
----- Original Message -----=20
From: Alvin Wang=20
To: Joerg Thoennes=20
Cc: qui...@li...=20
Sent: Tuesday, February 15, 2005 10:34 AM
Subject: Re: [Quickfix-developers] fragmentation?
Excerpt from FIX44 vol5:=20
Fragmentation of Allocation Messages=20
FIX Allocation messages support fragmentation in a way similar to=20
MassQuote and the List Order messages. If there are too many entries=20
within a repeating group to fit into one physical message, the entries can =
be continued in subsequent messages by repeating the principal message=20
reference and other required fields, then continuing with the repeating=20
group. This is achieved by using an optional TotNoAllocs field (giving the=
total number of AllocAccount details across the entire=20
allocation) that supplements the NoAllocs field (giving the number of Alloc=
Account details in a particular message=20
fragment). The TotNoAllocs field is repeated with the same value in all fr=
agments of the batch. For=20
example, an Allocation Instruction with 200 allocation account instances=20
could be fragmented across three messages - the first two containing=20
TotNoAllocs=3D200, NoAllocs=3D80 and the third TotNoAllocs=3D200, NoAllocs=
=3D40.=20
To help the receiver reconstitute the batch the Boolean field LastFragment =
is sent with a "Y" value in the last fragment.=20
For fragmented allocation events the receiving application must persist=20
state between messages to determine whether all instances of the repeating =
group have been received before acting on the instruction or processing=20
the report.=20
For this to work some key rules must be enforced:=20
1) The sender must supply a consistent value for TotNoAllocs in all =
related fragments and must use the same primary message reference in all=20
fragments of the batch, e.g. AllocID in AllocationInstruction.=20
2) The sender must ensure that fragments are transmitted in order=20
without intervening traffic.=20
3) The NoAllocs group must reach capacity only in the last=20
fragment, and that message must contain LastFragment=3DY.=20
4) The receiver must acknowledge every fragment received=20
(AllocationInstructionAck with AllocStatus=3D"received") and never reject a=
=20
non-last fragment; acknowledgment of the final fragment accepts or rejects =
the entire set.=20
There are a number of design suggestions for implementing fragmentation:=20
1) Optional block-level fields supplied in early fragments need not =
be repeated in subsequent fragments. If they are repeated and the values=20
are different, the receiver may choose to reject (on receiving the last=20
fragment) or to apply the last received value to the event.=20
2) If a message supports multiple "Number of" groups, e.g.=20
NoOrders, NoExecs, and NoAllocs in AllocationInstruction, the sender may=20
distribute the array instances over any and all fragments, as long as the=20
NoAllocs group is not filled before the last fragment.=20
3) The receiver must be able to abort collecting an incomplete=20
array ? either on expiration of a timer or the receipt of an unrelated=20
message from the same counterparty.=20
FIX Message=20
<Total number of> field=20
related <Number of> field=20
Prinicipal message reference=20
AllocationInstruction=20
TotNoAllocs=20
NoAllocs (78)=20
AllocID (70)=20
AllocationReport=20
TotNoAllocs=20
NoAllocs (78)=20
AllocReportID (755)
Maximum message size for fragmentation purposes can be determined by using =
the optional MaxMessageSize field in the Logon message or by mutual=20
agreement between counterparties.=20
Joerg Thoennes <Joe...@ma...>=20
02/15/2005 11:32 AM=20
=20
To: Alvin Wang <AW...@FF...>=20
cc: qui...@li...=20
bcc: =20
Subject: Re: [Quickfix-developers] fragmentation?
Hi Alvin,
> How does quickfix handle fragmentation? Do I have to do it myself? Or=20
> quickfix will decide and handle for me? Thanks!
What do you mean? If a socket read returns an incomplete FIX message, and=20
the next socket=20
read returns the next part and whether QF handles this?
Cheers, J=F6rg
--=20
Joerg Thoennes
http://macd.com
Tel.: +49 (0)241 44597-24 Macdonald Associates GmbH
Fax : +49 (0)241 44597-10 Lothringer Str. 52, D-52070 Aachen
**********************************************************************=20
This e-mail message is intended solely for the use of the addressee. The=20
message may contain information that is privileged and confidential.=20
Disclosure to anyone other than the intended recipient is prohibited. If=20
you are not the intended recipient, please do not disseminate, distribute=20
or copy this communication, by e-mail or otherwise. Instead, please notify =
us immediately by return e-mail (including the original message with your=20
reply) and then delete and discard all copies of the message. We have=20
taken precautions to minimize the risk of transmitting software viruses=20
but nevertheless advise you to carry out your own virus checks on any=20
attachment to this message. We accept no liability for any loss or damage=20
caused by software viruses.=20
**********************************************************************=20
|
|
From: Joerg T. <Joe...@ma...> - 2005-02-16 09:39:27
|
Alvin Wang wrote:
> Oren, thanks a lot for your reply.
>
> As Joerg suggested, it would be nicer if QF can handle this transparently
> (the user can configure the max message size in conf file of course). This
> should be mechanic and will save developers a lot of time.
Alvin, this would not be part of the core QF API since we want to have it as lean as
possible. But it would be a good idea to have an extra library to deal with common
application level problems, e.g. ClOrdID chaining, fragmentation, etc.
I would like to encourage everyone to put suggestion for such a utility library into WikiFIX.
Cheers, Jörg
--
Joerg Thoennes
http://macd.com
Tel.: +49 (0)241 44597-24 Macdonald Associates GmbH
Fax : +49 (0)241 44597-10 Lothringer Str. 52, D-52070 Aachen
|
|
From: Joerg T. <Joe...@ma...> - 2005-02-15 16:58:14
|
Alvin Wang wrote:
> Excerpt from FIX44 vol5:
>
> Fragmentation of Allocation Messages
> [...]
Ok, was not aware of that. Seems like an application level construct and is therefore not
supported by the QuickFIX engine.
But is probably worth being added to some utility library.
Cheers, Jörg
--
Joerg Thoennes
http://macd.com
Tel.: +49 (0)241 44597-24 Macdonald Associates GmbH
Fax : +49 (0)241 44597-10 Lothringer Str. 52, D-52070 Aachen
|
|
From: Oren M. <or...@qu...> - 2005-02-15 17:03:33
|
Alvin,
You would need to do this at the application level. The session is not =
going to be able to take a single message and break them up into =
multiple messages. You will need to prepare them before being sent. =
Same goes for receiving messages. You're application will need to hold =
on to the received messages and process them when the final one comes =
through. QF itself doesn't have a restriction on the size of sent or =
received messages, so the only time you should have to worry about this =
is if your counterparty has some sort of restriction.
--oren
----- Original Message -----=20
From: Alvin Wang=20
To: Joerg Thoennes=20
Cc: qui...@li...=20
Sent: Tuesday, February 15, 2005 10:34 AM
Subject: Re: [Quickfix-developers] fragmentation?
Excerpt from FIX44 vol5:=20
Fragmentation of Allocation Messages=20
FIX Allocation messages support fragmentation in a way similar to =
MassQuote and the List Order messages. If there are too many entries =
within a repeating group to fit into one physical message, the entries =
can be continued in subsequent messages by repeating the principal =
message reference and other required fields, then continuing with the =
repeating group. This is achieved by using an optional TotNoAllocs =
field (giving the total number of AllocAccount details across the entire =
allocation) that supplements the NoAllocs field (giving the number of =
AllocAccount details in a particular message fragment). The TotNoAllocs =
field is repeated with the same value in all fragments of the batch. =
For example, an Allocation Instruction with 200 allocation account =
instances could be fragmented across three messages - the first two =
containing TotNoAllocs=3D200, NoAllocs=3D80 and the third =
TotNoAllocs=3D200, NoAllocs=3D40. To help the receiver reconstitute the =
batch the Boolean field LastFragment is sent with a "Y" value in the =
last fragment.=20
For fragmented allocation events the receiving application must =
persist state between messages to determine whether all instances of the =
repeating group have been received before acting on the instruction or =
processing the report.=20
For this to work some key rules must be enforced:=20
1) The sender must supply a consistent value for TotNoAllocs in =
all related fragments and must use the same primary message reference in =
all fragments of the batch, e.g. AllocID in AllocationInstruction.=20
2) The sender must ensure that fragments are transmitted in =
order without intervening traffic.=20
3) The NoAllocs group must reach capacity only in the last =
fragment, and that message must contain LastFragment=3DY.=20
4) The receiver must acknowledge every fragment received =
(AllocationInstructionAck with AllocStatus=3D"received") and never =
reject a non-last fragment; acknowledgment of the final fragment accepts =
or rejects the entire set.=20
There are a number of design suggestions for implementing =
fragmentation:=20
1) Optional block-level fields supplied in early fragments need =
not be repeated in subsequent fragments. If they are repeated and the =
values are different, the receiver may choose to reject (on receiving =
the last fragment) or to apply the last received value to the event.=20
2) If a message supports multiple "Number of" groups, e.g. =
NoOrders, NoExecs, and NoAllocs in AllocationInstruction, the sender may =
distribute the array instances over any and all fragments, as long as =
the NoAllocs group is not filled before the last fragment.=20
3) The receiver must be able to abort collecting an incomplete =
array - either on expiration of a timer or the receipt of an unrelated =
message from the same counterparty.=20
FIX Message <Total number of> field related <Number of> field =
Prinicipal message reference =20
AllocationInstruction TotNoAllocs NoAllocs (78) AllocID (70) =
AllocationReport TotNoAllocs NoAllocs (78) AllocReportID =
(755)=20
Maximum message size for fragmentation purposes can be determined by =
using the optional MaxMessageSize field in the Logon message or by =
mutual agreement between counterparties.=20
Joerg Thoennes <Joe...@ma...>=20
02/15/2005 11:32 AM=20
=20
To: Alvin Wang <AW...@FF...>=20
cc: qui...@li...=20
bcc: =20
Subject: Re: [Quickfix-developers] fragmentation? =
Hi Alvin,
> How does quickfix handle fragmentation? Do I have to do it myself? =
Or=20
> quickfix will decide and handle for me? Thanks!
What do you mean? If a socket read returns an incomplete FIX message, =
and the next socket=20
read returns the next part and whether QF handles this?
Cheers, J=F6rg
--=20
Joerg Thoennes
http://macd.com
Tel.: +49 (0)241 44597-24 Macdonald Associates GmbH
Fax : +49 (0)241 44597-10 Lothringer Str. 52, D-52070 Aachen
********************************************************************** =
This e-mail message is intended solely for the use of the addressee. The =
message may contain information that is privileged and confidential. =
Disclosure to anyone other than the intended recipient is prohibited. If =
you are not the intended recipient, please do not disseminate, =
distribute or copy this communication, by e-mail or otherwise. Instead, =
please notify us immediately by return e-mail (including the original =
message with your reply) and then delete and discard all copies of the =
message. We have taken precautions to minimize the risk of transmitting =
software viruses but nevertheless advise you to carry out your own virus =
checks on any attachment to this message. We accept no liability for any =
loss or damage caused by software viruses. =
**********************************************************************=20
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X