#459 OLEObject handling of VT_BYREF is incorrect

Classes (182)

In OLE Automation all arguments to a method invocation through IDispatch::Invoke have to be variants. Much of the code in OLEObject is dealing with changing ooRexx variables to and from variants.

The VARIANT datatype has the concept of passing by reference and OLEObject has functions to create the by reference variant and to deference that variant.

This code is incorrect in that it does not handle all the variant types that can be sent by reference in OLE Automation. It compounds this problem by marking variants that it has not changed to a pass by reference variant as VT_BYREF. This causes poorly-written OLE Automation applications to crash and causes IDispatch::Inovke to fail on well-written applications.

This problem was uncovered by Mark Weaver and his use of Chart Director (http://www.advsofteng.com/index.html)

Chart Director's OLE interface is written in Visual Basic. Visual Basic seems to require that VT_ARRAY be passed by reference and this is one of the variant types that the referenceVariant() and dereferenceVariant() functions do not handle correctly.

It seems that this would be a common cause for OLEObject to work inconsistently with OLE applications written in Visual Basic. When passing a variant that is not a safe array, the OLEObject code would work. But then it would fail on a method that used an array as an arguement. This is the case with Chart Director. Many of the method invocations would work, but then one method would inexplicably fail.


  • Mark Miesfeld

    Mark Miesfeld - 2007-12-06

    Logged In: YES
    Originator: YES

    Committed revision 1417.

    This commit cleaned up the VT_BYREF handling in several places. This seems as though it could be a significant fix for OLEObject.

  • Mark Miesfeld

    Mark Miesfeld - 2010-02-22

    Fixed long before the 4.0.0 release, switching from pending to closed.



Cancel  Add attachments

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.

No, thanks