Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

HowTo deserialize nested structures

Help
2010-06-29
2013-09-19
  • BuddyButterfly
    BuddyButterfly
    2010-06-29

    Hi,

    I am not able to deserialize nested strucutes. The structures are as follows:

                /*
                 * OPCHDA_TIME structure
                 * 
                 * typedef struct tagOPCHDA_TIME {
                 *      BOOL bString;
                 *      [string] LPWSTR szTime;
                 *      FILETIME ftTime;
                 * } OPCHDA_TIME;
                 */
    
                /*
                 * OPCHDA_ITEM structure
                 * 
                 * typedef struct tagOPCHDA_ITEM {
                 *      OPCHANDLE   hClient;
                 *      DWORD       haAggregate;
                 *      DWORD       dwCount;
                 *      [size_is(dwCount)] FILETIME *pftTimeStamps;
                 *      [size_is(dwCount)] DWORD *pdwQualities;
                 *      [size_is(dwCount)] VARIANT *pvDataValues;
                 *  } OPCHDA_ITEM;
                 */
    
            /*
                 * ReadRaw method signature
                 * 
                 * HRESULT ReadRaw (
                 *      [in, out] OPCHDA_TIME *htStartTime,
                 *      [in, out] OPCHDA_TIME *htEndTime,
                 *      [in] DWORD dwNumValues,
                 *      [in] BOOL bBounds,
                 *      [in] DWORD dwNumItems,
                 *      [in, size_is(dwNumItems)] OPCHANDLE * phServer,
                 *      [out, size_is(,dwNumItems)] OPCHDA_ITEM ** ppItemValues,
                 *      [out, size_is(,dwNumItems)] HRESULT ** ppErrors
                 *  );
                 */
    

    The NDR represenation is:

    OUT LONG [#36] [proc#3]
    (   IN OUT MS FC_PSTRUCT [#4], 
        IN OUT MS FC_PSTRUCT [#8], 
        IN LONG [#12], 
        IN LONG [#16], 
        IN LONG [#20], 
        IN MS FC_C[V]ARRAY [#24], 
        OUT MS PTR_TO_TYPE{PTR_TO_TYPE{FC_BOGUS_ARRAY}} [#28], 
        OUT MS PTR_TO_TYPE{PTR_TO_TYPE{FC_C[V]ARRAY}} [#32] 
    )
    

    I have tried lots of things but either get a NullPointerException or an ArrayIndexOutOfBounds.

    The Wireshark trace is (only Stubs Data):

    00000000h: 00 00 00 00 00 00 00 00 F9 8E A6 01 00 00 00 00 ; ........ùŽ¦.....
    00000010h: 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 ; ................
    00000020h: 90 9F 8D DF 80 10 CB 01 00 00 00 00 00 00 00 00 ; Ÿß€.Ë.........
    00000030h: 01 10 04 C0                                     ; ...À
    

    Any help would be very much appreciated!

     
  • BuddyButterfly
    BuddyButterfly
    2010-07-28

    Still not working even with the thread listed.

    Any help available?

     
  • BuddyButterfly
    BuddyButterfly
    2010-07-29

    Hi Virkram,

    I see you replying to other posts. Do you have any clue how to get the ReadRaw to work? It would be nice to get a tip.

    Thanks in advance,
    Matt

     
  • Hi Matt,
                  Theirs are relatively easy Qs to answer. Yours on the other hand will take sometime. Have you seen the UtGard project http://openscada.org/index.php?option=com_content&task=view&id=26&Itemid=46 . They have done work in the same segment as you and this could be of great help in marshaling OPC objects.

    Of course, I sincerely apologize if I have made you uncomfortable in anyway by not answering.

    best regards,
    Vikram

     
  • BuddyButterfly
    BuddyButterfly
    2010-07-29

    Hi Vikram,

    thanks a lot for comming back to me. Yes, I am already in contact with one of the guys. What do you think? How much time would it need? 1 or 2 hours? I am willing to pay for it if it needs to. A 2 hour websession would be a great thing :-)

    Best regards,
    Matt

     
  • Hi,
         I think you are almost there. Let me go over this on Monday and I think we should be able to sort it out.

    best regards,
    Vikram

     
  • BuddyButterfly
    BuddyButterfly
    2010-08-02

    Hi Vikram,

    here the code snipped for validation. Setting isRef in JIPointer to true like erlend suggested gives an received bad stub data.
    Start and end time are just the appropriate JIStruct with the two 32 bit values set. I guess it has something to do how
    I create the empty structs createOpcTimeStruct() and createOpcItemStruct().

                callObject.addInParamAsObject(new JIPointer(hdaStartTime), JIFlags.FLAG_NULL);
                callObject.addInParamAsObject(new JIPointer(hdaEndTime), JIFlags.FLAG_NULL);
                callObject.addOutParamAsObject(new JIPointer(createOpcTimeStruct()), JIFlags.FLAG_NULL);
                callObject.addOutParamAsObject(new JIPointer(createOpcTimeStruct()), JIFlags.FLAG_NULL);
                callObject.addInParamAsInt(0, JIFlags.FLAG_NULL);
                callObject.addInParamAsInt(0, JIFlags.FLAG_NULL);
                callObject.addInParamAsInt(itemServerHandles.length, JIFlags.FLAG_NULL);
                callObject.addInParamAsObject(new JIArray(itemServerHandles, true), JIFlags.FLAG_NULL);
    
                callObject.addOutParamAsObject(new JIPointer(new JIArray(createOpcItemStruct(), null, 1, true)), JIFlags.FLAG_NULL);
                callObject.addOutParamAsObject(new JIPointer(new JIArray(Integer.class, null, 1, true)), JIFlags.FLAG_NULL);
    
        /*
         * OPCHDA_TIME structure
         * 
         * typedef struct tagOPCHDA_TIME {
         *      BOOL bString;
         *      [string] LPWSTR szTime;
         *      FILETIME ftTime;
         * } OPCHDA_TIME;
         */ 
        public static JIStruct createOpcTimeStruct()
        {
            JIStruct hdaTime = new JIStruct();
            try {
                hdaTime.addMember(Integer.class);
                hdaTime.addMember(new JIPointer(new JIString(JIFlags.FLAG_REPRESENTATION_STRING_LPWSTR),true));
                hdaTime.addMember(createFileTimeStruct());
            } catch (JIException e) {
                e.printStackTrace();
            }
    
            return hdaTime;
        }
    
        /*
         * OPCHDA_ITEM structure
         * 
         * typedef struct tagOPCHDA_ITEM {
         *      OPCHANDLE   hClient;
         *      DWORD       haAggregate;
         *      DWORD       dwCount;
         *      [size_is(dwCount)] FILETIME *pftTimeStamps;
         *      [size_is(dwCount)] DWORD    *pdwQualities;
         *      [size_is(dwCount)] VARIANT  *pvDataValues;
         *  } OPCHDA_ITEM;
         */
        public static JIStruct createOpcItemStruct()
        {
            JIStruct opcHdaItem = new JIStruct();
    
            try {
                opcHdaItem.addMember(Integer.class);
                opcHdaItem.addMember(Integer.class);
                opcHdaItem.addMember(Integer.class);
                opcHdaItem.addMember(new JIArray(createFileTimeStruct(), null, 1, true));
                opcHdaItem.addMember(new JIArray(Integer.class, null, 1, true));
                opcHdaItem.addMember(new JIArray(JIVariant.class, null, 1, true));
            } catch (JIException e) {
                e.printStackTrace();
            }
    
            return opcHdaItem;
        }
    
        public static JIStruct createFileTimeStruct() throws JIException {
            JIStruct struct = new JIStruct();
            struct.addMember(Integer.class);
            struct.addMember(Integer.class);
            return struct;
        }
    

    The code above leads to the following exception:

    Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 52
        at jcifs.util.Encdec.dec_uint32le(Encdec.java:90)
        at ndr.NdrBuffer.dec_ndr_long(NdrBuffer.java:135)
        at ndr.NetworkDataRepresentation.readUnsignedLong(NetworkDataRepresentation.java:64)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper$IntegerImpl.deserializeData(JIMarshalUnMarshalHelper.java:807)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:328)
        at org.jinterop.dcom.core.JIArray.decode(JIArray.java:501)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:271)
        at org.jinterop.dcom.core.JIPointer.decode(JIPointer.java:228)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:281)
        at org.jinterop.dcom.core.JICallBuilder.readPacket(JICallBuilder.java:1009)
        at org.jinterop.dcom.core.JICallBuilder.read(JICallBuilder.java:956)
        at ndr.NdrObject.decode(NdrObject.java:36)
        at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:137)
        at rpc.Stub.call(Stub.java:113)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:901)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:856)
        at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:266)
        at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:153)
        at mwit.HdaJinteropDirect.main(HdaJinteropDirect.java:431)
    
     
  • Hi,
         You know this means that the marshalling happened alright and the unmarshalling has issues. We can do this. You can take a wireshark capture for this call (the failed one) and send it across to me , I will try to map out the returned structure. Its the hard way but a sure shot one.

    What do you think ?

    thanks,
    best regards,
    Vikram

     
  • BuddyButterfly
    BuddyButterfly
    2010-08-10

    Hi Vikram,

    so, I am back again. Was also busy. I have done a ws capture. How can I send it to you?

    best regards,
    Matt

     
  • Hi,
         Replied to your mail. Send it to professional support.

    thanks,
    best regards,
    Vikram

     
    • Fred Larsen
      Fred Larsen
      2013-08-02

      Hi Vikram and BuddyButterfly,

      I am having exactly the same problem as in this thread.
      (https://sourceforge.net/p/j-interop/discussion/600730/thread/390f67c8/#6c3c)

      I see that the thread is old, but did you find a solution for this?

      Best regards,
      Fred Larsen

       
      • Dear Fred,

        I am not sure. It was quite some time back. Could you contact Buddy or may
        be look at the UtGuard project. Its open source and they may have handled
        this.

        thanks,
        best regards,
        Vikram

        On Fri, Aug 2, 2013 at 7:00 PM, Fred Larsen fredlar@users.sf.net wrote:

        Hi Vikram and BuddyButterfly,

        I am having exactly the same problem as in this thread.
        (
        https://sourceforge.net/p/j-interop/discussion/600730/thread/390f67c8/#6c3c
        )

        I see that the thread is old, but did you find a solution for this?

        Best regards,
        Fred Larsen


        HowTo deserialize nested structureshttps://sourceforge.net/p/j-interop/discussion/600730/thread/390f67c8/?limit=25#e08b/32de

        Sent from sourceforge.net because you indicated interest in
        https://sourceforge.net/p/j-interop/discussion/600730/

        To unsubscribe from further messages, please visit
        https://sourceforge.net/auth/subscriptions/

        --
        The Mind is a place of its own. It can make a heaven out of hell or a hell
        out of heaven. Attitude is everything. No matter how adverse conditions
        maybe, one has the capacity to turn things around by one's Determination,
        Perseverance and Hardwork.

        John Milton
        (Paradise Lost)

         
        • Fred Larsen
          Fred Larsen
          2013-08-11

          Hi Vikram, and thanks for the reply.

          Do you have any idea how I can reach BuddyButterfly? Is there some kind of PM system here that I have not found? If you have had an e-mail correspondence previously and you have his e-mail address, is it possible for you to notify him about activity in this thread?
          I am sure this is of interest to many people, and a solution would greatly increase the usage of j-interop.

          As for the Utgard project, it seems that they have stoppet at OPC DA 2.0 which does not include any calls of this complexity. OPC HDA (and possibly OPC DA 3.0) does, and these I have not been able to get working so far.

          Kind Regards,
          Fred Larsen

           
          • Dear Fred,

            You can email him directly via the sourceforge system. Btw, Filetime
            variable looks to be a pointer to an array of structs. Have you tried
            building it that way?

            thanks,
            best regards,
            Vikram
            On Aug 11, 2013 5:36 AM, "Fred Larsen" fredlar@users.sf.net wrote:

            Hi Vikram, and thanks for the reply.

            Do you have any idea how I can reach BuddyButterfly? Is there some kind of
            PM system here that I have not found? If you have had an e-mail
            correspondence previously and you have his e-mail address, is it possible
            for you to notify him about activity in this thread?
            I am sure this is of interest to many people, and a solution would greatly
            increase the usage of j-interop.

            As for the Utgard project, it seems that they have stoppet at OPC DA 2.0
            which does not include any calls of this complexity. OPC HDA (and possibly
            OPC DA 3.0) does, and these I have not been able to get working so far.

            Kind Regards,
            Fred Larsen


            HowTo deserialize nested structureshttps://sourceforge.net/p/j-interop/discussion/600730/thread/390f67c8/?limit=25#e08b/32de/55c3/56e7

            Sent from sourceforge.net because you indicated interest in
            https://sourceforge.net/p/j-interop/discussion/600730/

            To unsubscribe from further messages, please visit
            https://sourceforge.net/auth/subscriptions/

             
          • Hi Fred,

            We have extended their libraries with OPC AE specs client code. I do not have it handy at the moment, but if you think that OPC AE has similar calls, you can take a look at what we did there. Is your project open source? It would be great to have all those specs bundled up (DA, AE, HDA).

             
            • Fred Larsen
              Fred Larsen
              2013-08-29

              Hi Andrey,

              I think perhaps the problem is with the [in, out] parameters which are not used in any of the AE calls. I am using the OPCHDA_TIME struct in the OPCHDA_ITEMs that are successfully returned by ReadAtTime, so I think the structs are declared correctly. It is only ReadRaw that does not work.

              Our project is not open source, but if I find a solution I will show it here.

               
              • Hi Fred,

                Have you solved the issue? If yes, could you please post the solution? This seems to be a general problem with [IN, OUT] parameters... Thanks!

                Andrey

                 
  • Fred Larsen
    Fred Larsen
    2013-08-12

    Hi Vikram,

    I have got another OPC HDA call named ReadAtTime to work. This function does not have the htStartTime and htEndTime parameters as in ReadRaw, but all the others. So I think it has to be the time parameters that cause the trouble.

    Based on the IDL declaration

        HRESULT ReadRaw (
            [in, out] OPCHDA_TIME *htStartTime,
            [in, out] OPCHDA_TIME *htEndTime,
            [in] DWORD dwNumValues,
            [in] BOOL bBounds,
            [in] DWORD dwNumItems,
            [in, size_is(dwNumItems)] OPCHANDLE * phServer,
            [out, size_is(,dwNumItems)] OPCHDA_ITEM ** ppItemValues,
            [out, size_is(,dwNumItems)] HRESULT ** ppErrors
            );
    

    I first sent the time as

    callObject.addInParamAsPointer(new JIPointer(getOpcHdaTime(startTimeMillis));
    

    and I always got a "The stub received bad data.". Changing to:

    callObject.addInParamAsArray(new JIArray(new JIStruct[]{getOpcHdaTime(startTimeMillis)});
    

    got rid of that message and got me ahead to an unmarshaling error. This should indicate that the input parameters are OK but that there are problems with the output?

    As it stands now, this code (showing only the start time parameter):

    callObject.addInParamAsArray(new JIArray(new JIStruct[]{getOpcHdaTime(startTimeMillis)});
    callObject.addOutParamAsObject(new JIPointer(new JIArray(getOpcHdaTime(), null, 1, true)), JIFlags.FLAG_NULL);
    

    gives the following exception:

    java.lang.NegativeArraySizeException
        at java.lang.reflect.Array.newArray(Native Method)
        at java.lang.reflect.Array.newInstance(Array.java:52)
        at org.jinterop.dcom.core.JIArray.recurseDecode(JIArray.java:606)
        at org.jinterop.dcom.core.JIArray.decode(JIArray.java:593)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:271)
        at org.jinterop.dcom.core.JIPointer.decode(JIPointer.java:243)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:281)
        at org.jinterop.dcom.core.JICallBuilder.readPacket(JICallBuilder.java:1021)
        at org.jinterop.dcom.core.JICallBuilder.read(JICallBuilder.java:966)
        at ndr.NdrObject.decode(NdrObject.java:36)
        at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:137)
        at rpc.Stub.call(Stub.java:113)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:990)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:945)
        at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:271)
        at plugins.opc.OpcHdaIdl.readRaw(OpcHdaIdl.java:791)
    

    And if I try this for the out parameter:

    callObject.addOutParamAsObject(new JIArray(getOpcHdaTime(), null, 1, true), JIFlags.FLAG_NULL);
    

    then I get:

    java.lang.IllegalStateException: Internal Library Error, the serializerdeserializer was not found for 0.  Please check the parameters passed to JICallBuilder. [0x0000100F]
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:326)
        at org.jinterop.dcom.core.JIStruct.decode(JIStruct.java:297)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:287)
        at org.jinterop.dcom.core.JIArray.recurseDecode(JIArray.java:629)
        at org.jinterop.dcom.core.JIArray.decode(JIArray.java:593)
        at org.jinterop.dcom.core.JIMarshalUnMarshalHelper.deSerialize(JIMarshalUnMarshalHelper.java:271)
        at org.jinterop.dcom.core.JICallBuilder.readPacket(JICallBuilder.java:1021)
        at org.jinterop.dcom.core.JICallBuilder.read(JICallBuilder.java:966)
        at ndr.NdrObject.decode(NdrObject.java:36)
        at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:137)
        at rpc.Stub.call(Stub.java:113)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:990)
        at org.jinterop.dcom.core.JIComServer.call(JIComServer.java:945)
        at org.jinterop.dcom.core.JIComObjectImpl.call(JIComObjectImpl.java:271)
        at plugins.opc.OpcHdaIdl.readRaw(OpcHdaIdl.java:791)
    

    I am not sure if the problem is because of the [in, out] nature of the parameters, or because of how I declare the parameters. Do you see any errors in the way that I am doing it?

    I also tried the following variations without success:

    callObject.addOutParamAsObject(new JIArray(new JIPointer(getOpcHdaTime()), null, 1, true), JIFlags.FLAG_NULL);
    
    callObject.addOutParamAsObject(new JIPointer(new JIArray(new JIPointer(getOpcHdaTime()), null, 1, true)), JIFlags.FLAG_NULL);
    

    This is the declaration of the time structure:

        /*
        typedef struct tagOPCHDA_TIME {
            BOOL bString;
            [string] LPWSTR szTime;
            FILETIME ftTime;
        } OPCHDA_TIME;
        */
        public static JIStruct getOpcHdaTime() throws JIException {
            JIStruct struct = new JIStruct();
            struct.addMember(Integer.valueOf(0));
            struct.addMember(new JIString("",JIFlags.FLAG_REPRESENTATION_STRING_LPWSTR));
            struct.addMember(getFileTimeStruct());
            return struct;
        }
        public static JIStruct getFileTimeStruct() throws JIException {
          JIStruct struct = new JIStruct();
          struct.addMember(Integer.class);
          struct.addMember(Integer.class);
          return struct;
        }
    

    (PS: I got a reply from BuddyButterfly/Matt. He gave up and switched to OPC UA instead, but I do not have the freedom to do so. The alternative to using j-interop is to buy a DLL/wrapper and be limited to the Windows platform.)

    Hopefully,
    Fred Larsen