Re: [Quickfixj-users] Tag not defined for this message type, field=9610"

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

"A possible additional clue is I also see this reject on ExecutionReports
  Rejecting invalid message: quickfix.FieldException: Tag appears more than
once, field=2669"

Oh, man, investigate that one first.  To me, this says that there is likely
an error in Bloomberg's DD regarding a repeating group.  The parser is
terminating the group early because it found a field that didn't belong in
it, and it screws up everything it parses after.

I think your only choice is to take that message and manually parse it
against your modified DD to find what's wrong.

By the way, this is normal.  Don't blindly trust counterparty-supplied
DDs.  Always suspect them of having errors.

On Thu, Jul 11, 2019 at 2:05 PM Andrew Munn <mun...@gm...> wrote:

> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support:
> http://www.quickfixj.org/support/
>
>
> I have confirmed the user running the FIX initiator has read permission on
> the FIX44.modified.xml file
>
> Does FIX44.modified.xml completely replace FIX44.xml so I can delete
> FIX44.xml or is FIX44.modified.xml just overrideing FIX44.xml?
>
> I renamed a few fields in the DD and rebuild:
>
> <message name="ExecutionReport" msgtype="8"
> msgcat="SingleGeneralOrderHandling">
>    ...
>    <component name="NotesSpecial" required="N"/>
>   ...
> </message>
>
> <component name="NotesSpecial">
>   <group name="NoNotes2" required="N">
>      <field name="NoteType" required="N"/>
>      <field name="NoteLabel" required="N"/>
>      <field name="NoteText" required="N"/>
>   </group>
> </component>
>
> <field number="9610" name="NoNotes2" type="NUMINGROUP"/>
> <field number="9611" name="NoteType" type="STRING">
>   <value enum="C" description="CustomerNote"/>
>   <value enum="D" description="DealerNote"/>
>   <value enum="I" description="PrivateOrInternalNote"/>
> </field>
> <field number="9612" name="NoteLabel" type="STRING"/>
> <field number="9613" name="NoteText" type="STRING"/>
>
> I believe my FIX44.modified.xml DD was found in the expected location
> during compilation because when
> decompiling c:\java\qfj-2.1.1-bloomberg-map\quickfixj-core\target\classes\quickfix\fix44\ExecutionReport.class
> I see these methods:
>
>     import quickfix.fix44.component.NotesSpecial;
>     public void set(NotesSpecial component) {
>         this.setComponent(component);
>     }
>
>     public NotesSpecial get(NotesSpecial component) throws FieldNotFound {
>         this.getComponent(component);
>         return component;
>     }
>
>     public NotesSpecial getNotesSpecial() throws FieldNotFound {
>         return this.get(new NotesSpecial());
>     }
>
>     public void set(quickfix.field.NoNotes2 value) {
>         this.setField(value);
>     }
>
>     public quickfix.field.NoNotes2 get(quickfix.field.NoNotes2 value)
> throws FieldNotFound {
>         this.getField(value);
>         return value;
>     }
>
>     public quickfix.field.NoNotes2 getNoNotes2() throws FieldNotFound {
>         return this.get(new quickfix.field.NoNotes2());
>     }
>
>     public boolean isSet(quickfix.field.NoNotes2 field) {
>         return this.isSetField(field);
>     }
>
>     public boolean isSetNoNotes2() {
>         return this.isSetField(9610);
>     }
>
> And this class:
>
> public static class NoNotes2 extends Group {
>         static final long serialVersionUID = 20050617L;
>         private static final int[] ORDER = new int[]{9611, 9612, 9613, 0};
>
>         public NoNotes2() {
>             super(9610, 9611, ORDER);
>         }
>        ...
> }
>
> Does all that look correct?
>
> It feels like my initiator is not reading my DD but it must or I would not
> see the ConfigError exception when I remove the DD and restart, right?
>
> A possible additional clue is I also see this reject on ExecutionReports
>
>   Rejecting invalid message: quickfix.FieldException: Tag appears more
> than once, field=2669
>
> DD has:
>
> <message name="ExecutionReport" msgtype="8"
> msgcat="SingleGeneralOrderHandling">
>   ...
>   <component name="TrdRegPublicationGrp" required="N"/>
>   ...
> </message>
>
> <field number="2668" name="NoTrdRegPublications" type="NUMINGROUP"/>
> <field number="2669" name="TrdRegPublicationType" type="INT">
>   <value enum="0" description="PreTradeTransparencyWaiver"/>
>   <value enum="1" description="PostTradeDeferral"/>
>   <value enum="2" description="ExemptedFromPublication"/>
> </field>
>
> <component name="TrdRegPublicationGrp">
>   <group name="NoTrdRegPublications" required="N">
>     <field name="TrdRegPublicationType" required="N"/>
>     <field name="TrdRegPublicationReason" required="N"/>
>   </group>
> </component>
>
>
>
>
>
> On Thu, Jul 11, 2019 at 1:46 PM Andrew Munn <mun...@gm...> wrote:
>
>> The DD is the one being used.  If I rename it temporarily I get
>> Exception in thread "main" quickfix.ConfigError: Could not find data
>> dictionary: ./etc/FIX44.modified.xml
>>
>> tags are present:
>>
>> $ grep NoNotes2 FIX44.modified.xml
>>       <group name="NoNotes2" required="N">
>>     <field number="9610" name="NoNotes2" type="NUMINGROUP"/>
>>
>> I tried renaming the group to NoNotes2 in case there was some naming
>> collision.  Could this error be caused by some other field or group naming
>> collision elsewhere in the DD I was given?
>>
>> Everything looks ok but I must be overlooking something.
>>
>> Can I just in-line the Group into the ExecutionReport msg definition or
>> must Groups live in Components?
>>
>> On Thu, Jul 11, 2019 at 12:35 PM Grant Birchmeier <
>> gbi...@co...> wrote:
>>
>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support:
>>> http://www.quickfixj.org/support/
>>>
>>>
>>> Regarding your question about components, please read my SO answer here:
>>> https://stackoverflow.com/a/29774713/650475
>>> I think that will clarify the concept.
>>>
>>> I don't see anything obviously wrong in your message or DD definitions.
>>> 9610 is ok to appear in any top-level (not in another group) part of the
>>> message, and must be immediately followed by 9611 and then optionally
>>> 9612/13 (I'm sure you know all this already).
>>>
>>> My gut says your engine isn't reading the right config.  I mean, what
>>> you've posted all looks right.  Are you sure you copied the new DD to /etc/FIX44.modified.xml
>>> ?
>>>
>>>
>>> On Thu, Jul 11, 2019 at 10:22 AM Andrew Munn <mun...@gm...> wrote:
>>>
>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J>
>>>> Support: http://www.quickfixj.org/support/
>>>>
>>>>
>>>> Adding subject
>>>>
>>>>
>>>> On Thu, Jul 11, 2019 at 11:05 AM Andrew Munn <mun...@gm...>
>>>> wrote:
>>>>
>>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>>> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J>
>>>>> Support: http://www.quickfixj.org/support/
>>>>>
>>>>>
>>>>> I'm receiving FIX 4.4 ExecutionReport msgs from Bloomberg and
>>>>> rejecting them with:
>>>>>
>>>>> "Rejecting invalid message: quickfix.FieldException: Tag not defined
>>>>> for this message type, field=9610"
>>>>>
>>>>> I'm using the DataDictionary Bloomberg provided.
>>>>>
>>>>> I've rebuilt QF/J 2.1.1 on Windows using the provided DD.  My process
>>>>> to rebuild is:
>>>>> - delete all instances of FIX44.modified.xml in the QF directory tree
>>>>> - copy the provided DD to
>>>>> c:\java\qfj-2.1.1-bloomberg-map\quickfixj-messages\quickfixj-messages-fix44\src\main\resources\FIX44.modified.xml
>>>>> - do not modify FIX44.xml
>>>>> - mvn clean package -Dmaven.javadoc.skip=true -DskipTests
>>>>> -PskipBundlePlugin -DskipAT=true
>>>>>
>>>>>  Directory of
>>>>> c:\java\qfj-2.1.1-bloomberg-map\quickfixj-messages\quickfixj-messages-fix44\src\main\resources
>>>>> 2019-07-11  09:16 AM           287,770 FIX44.modified.xml
>>>>> 2018-11-12  02:11 PM           336,023 FIX44.xml
>>>>>
>>>>> I get warnings like these.  In the past I was able to ignore then when
>>>>> building for other counterparties.
>>>>>
>>>>> [WARNING] quickfixj-all-2.1.1.jar, quickfixj-messages-all-2.1.1.jar
>>>>> define 7190 overlapping classes:
>>>>> [WARNING] quickfixj-all-2.1.1.jar, quickfixj-codegenerator-2.1.1.jar
>>>>> define 5 overlapping classes:
>>>>> [WARNING] quickfixj-core-2.1.1.jar, quickfixj-all-2.1.1.jar define 238
>>>>> overlapping classes:
>>>>> [WARNING] quickfixj-dictgenerator-2.1.1.jar, quickfixj-all-2.1.1.jar
>>>>> define 13 overlapping classes:
>>>>> [WARNING] quickfixj-core-2.1.1.jar, quickfixj-all-2.1.1.jar,
>>>>> quickfixj-messages-all-2.1.1.jar define 2530 overlapping classes:
>>>>> [WARNING] maven-shade-plugin has detected that some class files are
>>>>> [WARNING] present in two or more JARs. When this happens, only one
>>>>> [WARNING] single version of the class is copied to the uber jar.
>>>>> [WARNING] Usually this is not harmful and you can skip these warnings,
>>>>> [WARNING] otherwise try to manually exclude artifacts based on
>>>>> [WARNING] mvn dependency:tree -Ddetail=true and the above output.
>>>>> [WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
>>>>>
>>>>> FIX44.modified.xml has:
>>>>>
>>>>> <message name="ExecutionReport" msgtype="8"
>>>>> msgcat="SingleGeneralOrderHandling">
>>>>>   ...
>>>>>   <component name="Notes" required="N"/>
>>>>> </message>
>>>>>
>>>>> <component name="Notes">
>>>>>   <group name="NoNotes" required="N">
>>>>>     <field name="NoteType" required="N"/>
>>>>>     <field name="NoteLabel" required="N"/>
>>>>>     <field name="NoteText" required="N"/>
>>>>>   </group>
>>>>> </component>
>>>>>
>>>>> <field number="9610" name="NoNotes" type="NUMINGROUP"/>
>>>>> <field number="9611" name="NoteType" type="STRING">
>>>>>   <value enum="C" description="CustomerNote"/>
>>>>>   <value enum="D" description="DealerNote"/>
>>>>>   <value enum="I" description="PrivateOrInternalNote"/>
>>>>> </field>
>>>>> <field number="9612" name="NoteLabel" type="STRING"/>
>>>>> <field number="9613" name="NoteText" type="STRING"/>
>>>>>
>>>>> (cfg file)
>>>>> [DEFAULT]
>>>>> AllowUnknownMsgFields=Y
>>>>> UseDataDictionary=Y
>>>>> AppDataDictionary=./etc/FIX44.modified.xml
>>>>> ValidateFieldsOutOfOrder=Y
>>>>>
>>>>> I can successfully parse the ExecutionReport String they send to XML
>>>>> like this:
>>>>>
>>>>> void parseSingleBlbExec(String data) throws InvalidMessage,
>>>>> ConfigError {
>>>>>         DataDictionary sessionDictionary = new
>>>>> DataDictionary("c:\\temp\\FIX44.modified.xml");
>>>>>         appDictionary = new
>>>>> DataDictionary("c:\\temp\\FIX44.modified.xml");
>>>>>         appDictionary.setCheckFieldsOutOfOrder(true);
>>>>>         appDictionary.setAllowUnknownMessageFields(false);
>>>>>         appDictionary.setCheckUserDefinedFields(true);
>>>>>         appDictionary.setCheckUnorderedGroupFields(true);
>>>>>         quickfix.fix44.ExecutionReport message = new
>>>>> quickfix.fix44.ExecutionReport();
>>>>>         message.fromString(data, sessionDictionary, appDictionary,
>>>>> true);
>>>>>         System.out.println(message.toXML(appDictionary));
>>>>> }
>>>>>
>>>>> But when the message arrives on a live session I generate a reject.
>>>>>
>>>>> Messages are like this:
>>>>>
>>>>> 8=FIX.4.4|9=1400|35=8|49=BLP_MAP_PROD3|56=XXX_PROD3|34=780|128=914171|347=UTF-8|142=Voice|144=FI|115=DCSVC_278_4|43=Y|122=20190711-11:18:25|52=20190711-14:10:12|30=XOFF|60=20190710-21:38:48.671|150=F|1950=8|31=123.456|151=0|541=20290515|32=99999|423=1|64=20190712|6=123.456|37=VCON:20190710:9999:9|157=58|38=99999|39=2|159=99999.99|669=123.456|460=6|223=0.123|14=999999|854=0|15=USD|75=20190710|106=US
>>>>> TREASURY
>>>>> N/B|17=VCON:20190710:xxxx:x:xx|167=XXXXXX|48=XXXXX86T2|198=3739:20190710:XXXXX:X|470=US|1430=V|381=123456.31|22=1|1913=0|54=2|7014=21|55=[N/A]|236=0.99|118=9999999.49|453=6|448=GS|447=D|452=1|802=2|523=3788|803=4014|523=N|803=4069|448=XXXXXXXXXXX:99999999|447=D|452=11|802=4|523=14|803=4|523=XXX
>>>>> XXXXX|803=9|523=NEW
>>>>> YORK|803=34|523=US|803=38|448=XXXXXXXXX:99999999|447=D|452=12|802=3|523=XXX
>>>>> XX|803=9|523=NEW
>>>>> YORK|803=34|523=US|803=38|448=XXXX|447=D|452=13|802=2|523=99999|803=9999|523=N|803=9999|448=BXT|447=D|452=16|448=XXXXXX:9999999|447=D|452=36|802=3|523=XXXX
>>>>> XXXX|803=9|523=NEW
>>>>> YORK|803=34|523=US|803=38|9610=3|9611=C|9612=Group|9613=XXX.XXX|9611=C|9612=Spread|9613=0|9611=C|9612=Prime|9613=XXX|454=4|455=USXXXXXXXXX|456=4|455=999999999|456=1|455=999999999|456=A|455=T|456=8|768=3|769=20190710-21:38:48.671|770=1|769=20190710-21:39:54.335|770=2|769=20190711-11:18:25.539|770=113|2529=1|2530=100|2531=9.9999999|10=210
>>>>>
>>>>> 8=FIX.4.4|9=195|35=3|34=4|49=XXX_PROD3|52=20190711-14:10:12.422|56=BLP_MAP_PROD3|115=914171|128=DCSVC_278_4|143=Voice|145=FI|45=780|58=Tag
>>>>> not defined for this message type, field=9610|371=9610|372=8|373=2|10=237|
>>>>>
>>>>> Must <component> blocks in the dictionary's message definition appear
>>>>> in the same order that the repeating groups arrive in?
>>>>>
>>>>> What am I overlooking here?
>>>>>
>>>>> Thanks
>>>>> _______________________________________________
>>>>> Quickfixj-users mailing list
>>>>> Qui...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>>>
>>>> _______________________________________________
>>>> Quickfixj-users mailing list
>>>> Qui...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>>
>>>
>>>
>>> --
>>> Grant Birchmeier
>>> *Connamara Systems, LLC*
>>> *Made-To-Measure Trading Solutions.*
>>> Exactly what you need. No more. No less.
>>> http://connamara.com
>>> _______________________________________________
>>> Quickfixj-users mailing list
>>> Qui...@li...
>>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>
>> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

-- 
Grant Birchmeier
*Connamara Systems, LLC*
*Made-To-Measure Trading Solutions.*
Exactly what you need. No more. No less.
http://connamara.com