quickfixj-users

[Quickfixj-users] EncodedText field and round-tripping messages that include other messages

[Toli K. <to...@ma...>]

Hey All,

Have a question for people using Text or EncodedText fields.
I need to include one FIX message inside another (for example, i want
to include the "offending" message into the reject message to show
where the error is, such as missing field).

I notice that if i include a FIX message inside another, it doesn't
"roundtrip" well - we can do this, for example

 buy = new NewOrderSingle(.....)
 reject = new ExecutionReport(rejectReason, bla bla bla)
 reject.setField(new EncodedText(buy.toString())
 reject.setField(new EncodedTextLen(buy.toString().length)
 reject.setField(new MessageEncoding(MessageEncoding.UTF_8)
 dataDictionary.validate(reject)

 roundTripped = new Message(reject.toString())

both the buy and the reject messsages validate with the given DataDictionary.
However, when i try to create a "round-tripped" message directly from
the text of the message containing another one, the Message parser
fails (which may be a bug)

I have the following questions:
1. Is this behavior even "legal"? Does FIX support embedding one
message inside another? should it be allowed?
2. If that's the case, shouldn't EncodedText field take in bytes
instead of a string, since you don't know which encoding is being
used?
3. Should Message be able to parse such an "embedded" message back in?

If the answer to all of this is true, then i'll file a bug about the
Message.extractField() code.

The reason it fails is that you end up with 2 SOH fields right after
another in the message, and the parser can't parse it correctly (the
first one form the end of the embedded message, and the 2nd SOH from
the end of the EncodedText field). Here's the example of the last few
fields of the embedded message followed by a trailer of the 'parent'
message:
....60=20070404-15:15:51\00110=021\001\00160=20070404-23:16:21\001102=1\001434=1\00110=002\001";

note how you end up having ....\001\00160=...., which currently breaks
the Java message parser (and i think the C++ QF parser as well)

-- 
Toli Kuznets
http://www.marketcetera.com: Open-Source Trading Platform
download.run.trade.

Re: [Quickfixj-users] EncodedText field and round-tripping messagesthat include other messages

[Steve B. <st...@te...>]

> Have a question for people using Text or EncodedText fields.
> I need to include one FIX message inside another (for example, i want
> to include the "offending" message into the reject message to show
> where the error is, such as missing field).

Hi Toli,

I added an embedded message test to the MessageTest suite in
the trunk. It embeds a NewOrderSingle in an ExecutionReport
EncodedText field. I create the ExecutionReport message string
and then parse it. I am able to retrieve the embedded message
and parse that. Can you take a look at the test? Maybe I'm
doing something different than you are, but the test passes.

Text fields with an SOH in them would be a problem.

Steve

Re: [Quickfixj-users] EncodedText field and round-tripping messagesthat include other messages

[Toli K. <to...@ma...>]

Ah!

I was creating the message without passing a dictionary in:
        Message msg = new Message(report.toString());
while you do this:
        Message msg = new Message(report.toString(),
DataDictionaryTest.getDictionary());

removing the dictionary causes the test to fail.

So I guess there's no bug here, I just need to be using a dictionary
to create a message.

What's the reason that it fails w/out having a dictionary? i tried
tracing through the code, but couldn't see why not having a dictionary
makes it fail.

Thanks for figuring this out!

On 4/12/07, Steve Bate <st...@te...> wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> > Have a question for people using Text or EncodedText fields.
> > I need to include one FIX message inside another (for example, i want
> > to include the "offending" message into the reject message to show
> > where the error is, such as missing field).
>
> Hi Toli,
>
> I added an embedded message test to the MessageTest suite in
> the trunk. It embeds a NewOrderSingle in an ExecutionReport
> EncodedText field. I create the ExecutionReport message string
> and then parse it. I am able to retrieve the embedded message
> and parse that. Can you take a look at the test? Maybe I'm
> doing something different than you are, but the test passes.
>
> Text fields with an SOH in them would be a problem.
>
> Steve
>
>
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>


-- 
Toli Kuznets
http://www.marketcetera.com: Open-Source Trading Platform
download.run.trade.

Re: [Quickfixj-users] EncodedText field and round-trippingmessagesthat include other messages

[Steve B. <st...@te...>]

The dictionary is needed to know that the field is of
type 'DATA'. The parser treats these fields specially. It looks 
for the corresponding length field and then extracts the specified
number of bytes. If it doesn't know it's a data field it will
become confused by embedded field delimiters in the data field
content. Take a look at line 652 in Message.java (rev 579).

> -----Original Message-----
> From: qui...@li... [mailto:quickfixj-
> use...@li...] On Behalf Of Toli Kuznets
> Sent: Thursday, April 12, 2007 9:51 PM
> To: qui...@li...
> Subject: Re: [Quickfixj-users] EncodedText field and round-
> trippingmessagesthat include other messages
> 
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> Ah!
> 
> I was creating the message without passing a dictionary in:
>         Message msg = new Message(report.toString());
> while you do this:
>         Message msg = new Message(report.toString(),
> DataDictionaryTest.getDictionary());
> 
> removing the dictionary causes the test to fail.
> 
> So I guess there's no bug here, I just need to be using a dictionary
> to create a message.
> 
> What's the reason that it fails w/out having a dictionary? i tried
> tracing through the code, but couldn't see why not having a dictionary
> makes it fail.
> 
> Thanks for figuring this out!
> 
> On 4/12/07, Steve Bate <st...@te...> wrote:
> > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> > QuickFIX/J Support: http://www.quickfixj.org/support/
> > > Have a question for people using Text or EncodedText fields.
> > > I need to include one FIX message inside another (for example, i want
> > > to include the "offending" message into the reject message to show
> > > where the error is, such as missing field).
> >
> > Hi Toli,
> >
> > I added an embedded message test to the MessageTest suite in
> > the trunk. It embeds a NewOrderSingle in an ExecutionReport
> > EncodedText field. I create the ExecutionReport message string
> > and then parse it. I am able to retrieve the embedded message
> > and parse that. Can you take a look at the test? Maybe I'm
> > doing something different than you are, but the test passes.
> >
> > Text fields with an SOH in them would be a problem.
> >
> > Steve
> >
> >
> >
> >
> > ------------------------------------------------------------------------
> -
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> > opinions on IT & business topics through brief surveys-and earn cash
> >
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > _______________________________________________
> > Quickfixj-users mailing list
> > Qui...@li...
> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
> >
> 
> 
> --
> Toli Kuznets
> http://www.marketcetera.com: Open-Source Trading Platform
> download.run.trade.
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

Re: [Quickfixj-users] EncodedText field and round-trippingmessagesthat include other messages

[Toli K. <to...@ma...>]

Makes sense.

thanks for the explanation!

On 4/12/07, Steve Bate <st...@te...> wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> The dictionary is needed to know that the field is of
> type 'DATA'. The parser treats these fields specially. It looks
> for the corresponding length field and then extracts the specified
> number of bytes. If it doesn't know it's a data field it will
> become confused by embedded field delimiters in the data field
> content. Take a look at line 652 in Message.java (rev 579).
>
> > -----Original Message-----
> > From: qui...@li... [mailto:quickfixj-
> > use...@li...] On Behalf Of Toli Kuznets
> > Sent: Thursday, April 12, 2007 9:51 PM
> > To: qui...@li...
> > Subject: Re: [Quickfixj-users] EncodedText field and round-
> > trippingmessagesthat include other messages
> >
> > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> > QuickFIX/J Support: http://www.quickfixj.org/support/
> > Ah!
> >
> > I was creating the message without passing a dictionary in:
> >         Message msg = new Message(report.toString());
> > while you do this:
> >         Message msg = new Message(report.toString(),
> > DataDictionaryTest.getDictionary());
> >
> > removing the dictionary causes the test to fail.
> >
> > So I guess there's no bug here, I just need to be using a dictionary
> > to create a message.
> >
> > What's the reason that it fails w/out having a dictionary? i tried
> > tracing through the code, but couldn't see why not having a dictionary
> > makes it fail.
> >
> > Thanks for figuring this out!
> >
> > On 4/12/07, Steve Bate <st...@te...> wrote:
> > > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> > > QuickFIX/J Support: http://www.quickfixj.org/support/
> > > > Have a question for people using Text or EncodedText fields.
> > > > I need to include one FIX message inside another (for example, i want
> > > > to include the "offending" message into the reject message to show
> > > > where the error is, such as missing field).
> > >
> > > Hi Toli,
> > >
> > > I added an embedded message test to the MessageTest suite in
> > > the trunk. It embeds a NewOrderSingle in an ExecutionReport
> > > EncodedText field. I create the ExecutionReport message string
> > > and then parse it. I am able to retrieve the embedded message
> > > and parse that. Can you take a look at the test? Maybe I'm
> > > doing something different than you are, but the test passes.
> > >
> > > Text fields with an SOH in them would be a problem.
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > > ------------------------------------------------------------------------
> > -
> > > Take Surveys. Earn Cash. Influence the Future of IT
> > > Join SourceForge.net's Techsay panel and you'll get the chance to share
> > your
> > > opinions on IT & business topics through brief surveys-and earn cash
> > >
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > > _______________________________________________
> > > Quickfixj-users mailing list
> > > Qui...@li...
> > > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
> > >
> >
> >
> > --
> > Toli Kuznets
> > http://www.marketcetera.com: Open-Source Trading Platform
> > download.run.trade.
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share
> > your
> > opinions on IT & business topics through brief surveys-and earn cash
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > _______________________________________________
> > Quickfixj-users mailing list
> > Qui...@li...
> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>


-- 
Toli Kuznets
http://www.marketcetera.com: Open-Source Trading Platform
download.run.trade.