HowTo deserialize nested structures

Help
2010-06-29
2013-09-19
  • BuddyButterfly

    BuddyButterfly - 2010-06-29

    Hi,

    I am not able to deserialize nested strucutes. The structures are as follows:

                /*
                 * OPCHDA_TIME structure
                 * 
                 * typedef struct tagOPCHDA_TIME {
                 *      BOOL bString;
                 *      [string] LPWSTR szTime;
                 *      FILETIME ftTime;
                 * } OPCHDA_TIME;
                 */
    
                /*
                 * OPCHDA_ITEM structure
                 * 
                 * typedef struct tagOPCHDA_ITEM {
                 *      OPCHANDLE   hClient;
                 *      DWORD       haAggregate;
                 *      DWORD       dwCount;
                 *      [size_is(dwCount)] FILETIME *pftTimeStamps;
                 *      [size_is(dwCount)] DWORD *pdwQualities;
                 *      [size_is(dwCount)] VARIANT *pvDataValues;
                 *  } OPCHDA_ITEM;
                 */
    
            /*
                 * ReadRaw method signature
                 * 
                 * HRESULT ReadRaw (
                 *      [in, out] OPCHDA_TIME *htStartTime,
                 *      [in, out] OPCHDA_TIME *htEndTime,
                 *      [in] DWORD dwNumValues,
                 *      [in] BOOL bBounds,
                 *      [in] DWORD dwNumItems,
                 *      [in, size_is(dwNumItems)] OPCHANDLE * phServer,
                 *      [out, size_is(,dwNumItems)] OPCHDA_ITEM ** ppItemValues,
                 *      [out, size_is(,dwNumItems)] HRESULT ** ppErrors
                 *  );
                 */
    

    The NDR represenation is:

    OUT LONG [#36] [proc#3]
    (   IN OUT MS FC_PSTRUCT [#4], 
        IN OUT MS FC_PSTRUCT [#8], 
        IN LONG [#12], 
        IN LONG [#16], 
        IN LONG [#20], 
        IN MS FC_C[V]ARRAY [#24], 
        OUT MS PTR_TO_TYPE{PTR_TO_TYPE{FC_BOGUS_ARRAY}} [#28], 
        OUT MS PTR_TO_TYPE{PTR_TO_TYPE{FC_C[V]ARRAY}} [#32] 
    )
    

    I have tried lots of things but either get a NullPointerException or an ArrayIndexOutOfBounds.

    The Wireshark trace is (only Stubs Data):

    00000000h: 00 00 00 00 00 00 00 00 F9 8E A6 01 00 00 00 00 ; ........ùŽ¦.....
    00000010h: 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 ; ................
    00000020h: 90 9F 8D DF 80 10 CB 01 00 00 00 00 00 00 00 00 ; Ÿß€.Ë.........
    00000030h: 01 10 04 C0                                     ; ...À
    

    Any help would be very much appreciated!

     
  • BuddyButterfly

    BuddyButterfly - 2010-07-28

    Still not working even with the thread listed.

    Any help available?

     
  • BuddyButterfly

    BuddyButterfly - 2010-07-29

    Hi Virkram,

    I see you replying to other posts. Do you have any clue how to get the ReadRaw to work? It would be nice to get a tip.

    Thanks in advance,
    Matt

     
  • Vikram Roopchand

    Hi Matt,
                  Theirs are relatively easy Qs to answer. Yours on the other hand will take sometime. Have you seen the UtGard project http://openscada.org/index.php?option=com_content&task=view&id=26&Itemid=46 . They have done work in the same segment as you and this could be of great help in marshaling OPC objects.

    Of course, I sincerely apologize if I have made you uncomfortable in anyway by not answering.

    best regards,
    Vikram

     
  • BuddyButterfly

    BuddyButterfly - 2010-07-29

    Hi Vikram,

    thanks a lot for comming back to me. Yes, I am already in contact with one of the guys. What do you think? How much time would it need? 1 or 2 hours? I am willing to pay for it if it needs to. A 2 hour websession would be a great thing :-)

    Best regards,
    Matt

     
  • Vikram Roopchand

    Hi,
         I think you are almost there. Let me go over this on Monday and I think we should be able to sort it out.

    best regards,
    Vikram

     
  • BuddyButterfly

    BuddyButterfly - 2010-08-02

    Hi Vikram,

    here the code snipped for validation. Setting isRef in JIPointer to true like erlend suggested gives an received bad stub data.
    Start and end time are just the appropriate JIStruct with the two 32 bit values set. I guess it has something to do how
    I create the empty structs createOpcTimeStruct() and createOpcItemStruct().

                callObject.addInParamAsObject(new JIPointer(hdaStartTime), JIFlags.FLAG_NULL);
                callObject.addInParamAsObject(new JIPointer(hdaEndTime), JIFlags.FLAG_NULL);
                callObject.addOutParamAsObject(new JIPointer(createOpcTimeStruct()), JIFlags.FLAG_NULL);
                callObject.addOutParamAsObject(new JIPointer(createOpcTimeStruct()), JIFlags.FLAG_NULL);
                callObject.addInParamAsInt(0, JIFlags.FLAG_NULL);
                callObject.addInParamAsInt(0, JIFlags.FLAG_NULL);
                callObject.addInParamAsInt(itemServerHandles.length, JIFlags.FLAG_NULL);
                callObject.addInParamAsObject(new JIArray(itemServerHandles, true), JIFlags.FLAG_NULL);
    
                callObject.addOutParamAsObject(new JIPointer(new JIArray(createOpcItemStruct(), null, 1, true)), JIFlags.FLAG_NULL);
                callObject.addOutParamAsObject(new JIPointer(new JIArray(Integer.class, null, 1, true)), JIFlags.FLAG_NULL);
    
        /*
         * OPCHDA_TIME structure
         * 
         * typedef struct tagOPCHDA_TIME {
         *      BOOL bString;
         *      [string] LPWSTR szTime;
         *      FILETIME ftTime;
         * } OPCHDA_TIME;
         */ 
        public static JIStruct createOpcTimeStruct()
        {
            JIStruct hdaTime = new JIStruct();
            try {
                hdaTime.addMember(Integer.class);
                hdaTime.addMember(new JIPointer(new JIString(JIFlags.FLAG_REPRESENTATION_STRING_LPWSTR),true));
                hdaTime.addMember(createFileTimeStruct());
            } catch (JIException e) {
                e.printStackTrace();
            }
    
            return hdaTime;
        }
    
        /*
         * OPCHDA_ITEM structure
         * 
         * typedef struct tagOPCHDA_ITEM {
         *      OPCHANDLE   hClient;
         *      DWORD       haAggregate;
         *      DWORD       dwCount;
         *      [size_is(dwCount)] FILETIME *pftTimeStamps;
         *      [size_is(dwCount)] DWORD    *pdwQualities;
         *      [size_is(dwCount)] VARIANT  *pvDataValues;
         *  } OPCHDA_ITEM;
         */
        public static JIStruct createOpcItemStruct()
        {
            JIStruct opcHdaItem = new JIStruct();
    
            try {
                opcHdaItem.addMember(Integer.class);
                opcHdaItem.addMember(Integer.class);
                opcHdaItem.addMember(Integer.class);
                opcHdaItem.addMember(new JIArray(createFileTimeStruct(), null, 1, true));
                opcHdaItem.addMember(new JIArray(Integer.class, null, 1, true));
                opcHdaItem.addMember(new JIArray(JIVariant.class, null, 1, true));
            } catch (JIException e) {
                e.printStackTrace();
            }
    
            return opcHdaItem;
        }
    
        public static JIStruct createFileTimeStruct() throws JIException {
            JIStruct struct = new JIStruct();
            struct.addMember(Integer.class);
            struct.addMember(Integer.class);
            return struct;
        }
    

    The code above leads to the following exception:

    Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 52
        at jcifs.util.Encdec.dec_uint32le(Encdec.java:90)
        at ndr.NdrBuffer.dec_ndr_long(NdrBuffer.java:135)
        at ndr.NetworkDataRepresentation.readUnsignedLong(NetworkDataRepresentation.java:64)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper$IntegerImpl.deserializeData(JIMarshalUnMarshalHelper.java:807)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:328)
        at org.jinterop.dcom.core.JIArray.decode(JIArray.java:501)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:271)
        at org.jinterop.dcom.core.JIPointer.decode(JIPointer.java:228)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:281)
        at org.jinterop.dcom.core.JICallBuilder.readPacket(JICallBuilder.java:1009)
        at org.jinterop.dcom.core.JICallBuilder.read(JICallBuilder.java:956)
        at ndr.NdrObject.decode(NdrObject.java:36)
        at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:137)
        at rpc.Stub.call(Stub.java:113)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:901)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:856)
        at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:266)
        at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:153)
        at mwit.HdaJinteropDirect.main(HdaJinteropDirect.java:431)
    
     
  • Vikram Roopchand

    Hi,
         You know this means that the marshalling happened alright and the unmarshalling has issues. We can do this. You can take a wireshark capture for this call (the failed one) and send it across to me , I will try to map out the returned structure. Its the hard way but a sure shot one.

    What do you think ?

    thanks,
    best regards,
    Vikram

     
  • BuddyButterfly

    BuddyButterfly - 2010-08-10

    Hi Vikram,

    so, I am back again. Was also busy. I have done a ws capture. How can I send it to you?

    best regards,
    Matt

     
  • Vikram Roopchand

    Hi,
         Replied to your mail. Send it to professional support.

    thanks,
    best regards,
    Vikram

     
    • Fred Larsen

      Fred Larsen - 2013-08-02

      Hi Vikram and BuddyButterfly,

      I am having exactly the same problem as in this thread.
      (https://sourceforge.net/p/j-interop/discussion/600730/thread/390f67c8/#6c3c)

      I see that the thread is old, but did you find a solution for this?

      Best regards,
      Fred Larsen

       
      • Vikram Roopchand

        Dear Fred,

        I am not sure. It was quite some time back. Could you contact Buddy or may
        be look at the UtGuard project. Its open source and they may have handled
        this.

        thanks,
        best regards,
        Vikram

        On Fri, Aug 2, 2013 at 7:00 PM, Fred Larsen fredlar@users.sf.net wrote:

        Hi Vikram and BuddyButterfly,

        I am having exactly the same problem as in this thread.
        (
        https://sourceforge.net/p/j-interop/discussion/600730/thread/390f67c8/#6c3c
        )

        I see that the thread is old, but did you find a solution for this?

        Best regards,
        Fred Larsen


        HowTo deserialize nested structureshttps://sourceforge.net/p/j-interop/discussion/600730/thread/390f67c8/?limit=25#e08b/32de

        Sent from sourceforge.net because you indicated interest in
        https://sourceforge.net/p/j-interop/discussion/600730/

        To unsubscribe from further messages, please visit
        https://sourceforge.net/auth/subscriptions/

        --
        The Mind is a place of its own. It can make a heaven out of hell or a hell
        out of heaven. Attitude is everything. No matter how adverse conditions
        maybe, one has the capacity to turn things around by one's Determination,
        Perseverance and Hardwork.

        John Milton
        (Paradise Lost)

         
        • Fred Larsen

          Fred Larsen - 2013-08-11

          Hi Vikram, and thanks for the reply.

          Do you have any idea how I can reach BuddyButterfly? Is there some kind of PM system here that I have not found? If you have had an e-mail correspondence previously and you have his e-mail address, is it possible for you to notify him about activity in this thread?
          I am sure this is of interest to many people, and a solution would greatly increase the usage of j-interop.

          As for the Utgard project, it seems that they have stoppet at OPC DA 2.0 which does not include any calls of this complexity. OPC HDA (and possibly OPC DA 3.0) does, and these I have not been able to get working so far.

          Kind Regards,
          Fred Larsen

           
          • Vikram Roopchand

            Dear Fred,

            You can email him directly via the sourceforge system. Btw, Filetime
            variable looks to be a pointer to an array of structs. Have you tried
            building it that way?

            thanks,
            best regards,
            Vikram
            On Aug 11, 2013 5:36 AM, "Fred Larsen" fredlar@users.sf.net wrote:

            Hi Vikram, and thanks for the reply.

            Do you have any idea how I can reach BuddyButterfly? Is there some kind of
            PM system here that I have not found? If you have had an e-mail
            correspondence previously and you have his e-mail address, is it possible
            for you to notify him about activity in this thread?
            I am sure this is of interest to many people, and a solution would greatly
            increase the usage of j-interop.

            As for the Utgard project, it seems that they have stoppet at OPC DA 2.0
            which does not include any calls of this complexity. OPC HDA (and possibly
            OPC DA 3.0) does, and these I have not been able to get working so far.

            Kind Regards,
            Fred Larsen


            HowTo deserialize nested structureshttps://sourceforge.net/p/j-interop/discussion/600730/thread/390f67c8/?limit=25#e08b/32de/55c3/56e7

            Sent from sourceforge.net because you indicated interest in
            https://sourceforge.net/p/j-interop/discussion/600730/

            To unsubscribe from further messages, please visit
            https://sourceforge.net/auth/subscriptions/

             
          • Andrey Romanenko

            Hi Fred,

            We have extended their libraries with OPC AE specs client code. I do not have it handy at the moment, but if you think that OPC AE has similar calls, you can take a look at what we did there. Is your project open source? It would be great to have all those specs bundled up (DA, AE, HDA).

             
            • Fred Larsen

              Fred Larsen - 2013-08-29

              Hi Andrey,

              I think perhaps the problem is with the [in, out] parameters which are not used in any of the AE calls. I am using the OPCHDA_TIME struct in the OPCHDA_ITEMs that are successfully returned by ReadAtTime, so I think the structs are declared correctly. It is only ReadRaw that does not work.

              Our project is not open source, but if I find a solution I will show it here.

               
              • Andrey Romanenko

                Hi Fred,

                Have you solved the issue? If yes, could you please post the solution? This seems to be a general problem with [IN, OUT] parameters... Thanks!

                Andrey

                 
  • Fred Larsen

    Fred Larsen - 2013-08-12

    Hi Vikram,

    I have got another OPC HDA call named ReadAtTime to work. This function does not have the htStartTime and htEndTime parameters as in ReadRaw, but all the others. So I think it has to be the time parameters that cause the trouble.

    Based on the IDL declaration

        HRESULT ReadRaw (
            [in, out] OPCHDA_TIME *htStartTime,
            [in, out] OPCHDA_TIME *htEndTime,
            [in] DWORD dwNumValues,
            [in] BOOL bBounds,
            [in] DWORD dwNumItems,
            [in, size_is(dwNumItems)] OPCHANDLE * phServer,
            [out, size_is(,dwNumItems)] OPCHDA_ITEM ** ppItemValues,
            [out, size_is(,dwNumItems)] HRESULT ** ppErrors
            );
    

    I first sent the time as

    callObject.addInParamAsPointer(new JIPointer(getOpcHdaTime(startTimeMillis));
    

    and I always got a "The stub received bad data.". Changing to:

    callObject.addInParamAsArray(new JIArray(new JIStruct[]{getOpcHdaTime(startTimeMillis)});
    

    got rid of that message and got me ahead to an unmarshaling error. This should indicate that the input parameters are OK but that there are problems with the output?

    As it stands now, this code (showing only the start time parameter):

    callObject.addInParamAsArray(new JIArray(new JIStruct[]{getOpcHdaTime(startTimeMillis)});
    callObject.addOutParamAsObject(new JIPointer(new JIArray(getOpcHdaTime(), null, 1, true)), JIFlags.FLAG_NULL);
    

    gives the following exception:

    java.lang.NegativeArraySizeException
        at java.lang.reflect.Array.newArray(Native Method)
        at java.lang.reflect.Array.newInstance(Array.java:52)
        at org.jinterop.dcom.core.JIArray.recurseDecode(JIArray.java:606)
        at org.jinterop.dcom.core.JIArray.decode(JIArray.java:593)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:271)
        at org.jinterop.dcom.core.JIPointer.decode(JIPointer.java:243)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:281)
        at org.jinterop.dcom.core.JICallBuilder.readPacket(JICallBuilder.java:1021)
        at org.jinterop.dcom.core.JICallBuilder.read(JICallBuilder.java:966)
        at ndr.NdrObject.decode(NdrObject.java:36)
        at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:137)
        at rpc.Stub.call(Stub.java:113)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:990)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:945)
        at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:271)
        at plugins.opc.OpcHdaIdl.readRaw(OpcHdaIdl.java:791)
    

    And if I try this for the out parameter:

    callObject.addOutParamAsObject(new JIArray(getOpcHdaTime(), null, 1, true), JIFlags.FLAG_NULL);
    

    then I get:

    java.lang.IllegalStateException: Internal Library Error, the serializerdeserializer was not found for 0.  Please check the parameters passed to JICallBuilder. [0x0000100F]
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:326)
        at org.jinterop.dcom.core.JIStruct.decode(JIStruct.java:297)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:287)
        at org.jinterop.dcom.core.JIArray.recurseDecode(JIArray.java:629)
        at org.jinterop.dcom.core.JIArray.decode(JIArray.java:593)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:271)
        at org.jinterop.dcom.core.JICallBuilder.readPacket(JICallBuilder.java:1021)
        at org.jinterop.dcom.core.JICallBuilder.read(JICallBuilder.java:966)
        at ndr.NdrObject.decode(NdrObject.java:36)
        at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:137)
        at rpc.Stub.call(Stub.java:113)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:990)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:945)
        at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:271)
        at plugins.opc.OpcHdaIdl.readRaw(OpcHdaIdl.java:791)
    

    I am not sure if the problem is because of the [in, out] nature of the parameters, or because of how I declare the parameters. Do you see any errors in the way that I am doing it?

    I also tried the following variations without success:

    callObject.addOutParamAsObject(new JIArray(new JIPointer(getOpcHdaTime()), null, 1, true), JIFlags.FLAG_NULL);
    
    callObject.addOutParamAsObject(new JIPointer(new JIArray(new JIPointer(getOpcHdaTime()), null, 1, true)), JIFlags.FLAG_NULL);
    

    This is the declaration of the time structure:

        /*
        typedef struct tagOPCHDA_TIME {
            BOOL bString;
            [string] LPWSTR szTime;
            FILETIME ftTime;
        } OPCHDA_TIME;
        */
        public static JIStruct getOpcHdaTime() throws JIException {
            JIStruct struct = new JIStruct();
            struct.addMember(Integer.valueOf(0));
            struct.addMember(new JIString("",JIFlags.FLAG_REPRESENTATION_STRING_LPWSTR));
            struct.addMember(getFileTimeStruct());
            return struct;
        }
        public static JIStruct getFileTimeStruct() throws JIException {
          JIStruct struct = new JIStruct();
          struct.addMember(Integer.class);
          struct.addMember(Integer.class);
          return struct;
        }
    

    (PS: I got a reply from BuddyButterfly/Matt. He gave up and switched to OPC UA instead, but I do not have the freedom to do so. The alternative to using j-interop is to buy a DLL/wrapper and be limited to the Windows platform.)

    Hopefully,
    Fred Larsen

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks