#15 GxInspect TREETEXT - the 64-bit Delphi 13 IDE's OnGetText passes the cell text as WideString, not string

[Ian Branch]

Hi Thomas,

First, thank you for TREETEXT. We have taken the idea into our own IDE automation package
(gllIdeAutomation, as a "tree_text" command), re-implemented rather than copied, and credited to
you and GxInspect in the source and the documentation. It finally lets us read the Local
Variables, Watch and Call Stack panes without screen coordinates, which we had been working around
for weeks.

While doing that we found something that affects GxInspect as well.

WHAT WE FOUND

Before hooking OnGetText, our version reads the event's type from RTTI and checks every parameter.
In the 64-bit Delphi 13.2 IDE (bin64\bds.exe 37.0.60952.8797) it refused all three debugger panes,
because RTTI reports this declaration (quoted from the refusal message, identical for
CallStackTree, LocalsTreeView and WatchTree):

TVSTGetTextEvent = procedure(Sender: TBaseVirtualTree; Node: PVirtualNode;
Column: TColumnIndex; TextType: TVSTTextType; var CellText: WideString) of object

So the IDE's own copy of VirtualTrees still declares the cell text as WideString. Its trees also
publish DefaultText as WideString, which is the same tell. Current VirtualTrees declares both as
string (UnicodeString), and that is the type GxInspect_Server.pas uses for every Unicode compiler:

{$IFDEF UNICODE}
TGxVtText = string;
{$ELSE}
TGxVtText = WideString;
{$ENDIF}

WHY IT MATTERS

A WideString is a COM BSTR and a string is a UnicodeString. The characters are laid out the same,
but the headers are not:

• A BSTR stores its length in BYTES at offset -4. A UnicodeString stores its length in CHARACTERS
  there, so a BSTR read as a string appears twice as long as it is. Anything that reads the full
  length, such as Copy, Pos or a concatenation, runs past the end of the text. The BSTR's own
  terminating #0 is usually what stops the damage. We believe this explains the symptom your code
  comment describes for the Call Stack window, where every row came back padded with the tail of an
  earlier one and had to be cut at the first #0.

• A UnicodeString carries a reference count at offset -8, which a BSTR does not have. Assigning the
  value ("Cell := string(_CellText)") increments whatever happens to sit at that address, and
  finalising the local decrements it again. If that value ever reaches zero, the RTL frees the
  block with the Delphi memory manager, although the OLE allocator owns it. We have not seen that
  happen, but it is a memory write into somebody else's allocation header on every painted cell.

We only have RTTI evidence for the 64-bit IDE. The 32-bit IDE probably carries the same
VirtualTrees, but we have not checked it.

WHAT WE DID

The reader now picks the handler shape from RTTI: one handler declared with "var CellText: string",
one with "var CellText: WideString", chosen by the kind of the fifth parameter (tkUString or
tkWString). The WideString handler converts with string(ACellText), so the BSTR is read as a BSTR.
Any other shape is refused, and nothing is hooked. The check, in outline:

var LType := LContext.GetType(PropInfo^.PropType^) as TRttiMethodType;
var LParams := LType.GetParameters;
// 5 parameters; [0] tkClass, [1] tkPointer, [2] 4-byte tkInteger, [3] 1-byte tkEnumeration,
// [4] pfVar and TypeKind in [tkUString, tkWString]
IsWide := LParams[4].ParamType.TypeKind = tkWString;

With that change and the debugger paused at a breakpoint, the reader matched the IDE row for row:
22 of 22 Call Stack frames, and Local Variables Self, Sender, E and __frame. The IDE stayed stable
throughout.

This needs Delphi 2010 or later for System.Rtti, so it may not suit GxInspect's range as it stands.
Where there is no RTTI, a per-IDE-version constant would do the same job. The package is built per
IDE version anyway, and it is the IDE's VirtualTrees, not the compiler, that decides the type.
Either way, it would probably be worth checking the declaration in each supported IDE rather than
keying it on UNICODE.

ONE MORE THING, IN CASE IT HELPS YOU TOO

Separately, our 64-bit IDE was intermittently hanging on the splash screen ("Initializing .NET
Services"). A dump traced it to Peganza's Pascal Analyzer IDE wizard (palwiz19_x64.dll) freeing
IDE-owned memory with its own private memory manager. That corrupted borlndmm's free lists, and the
IDE's JCL exception hook then deadlocked on borlndmm's lock. We have reported it to Peganza. If a
GExperts user ever reports a hang at that point of startup, that wizard is worth ruling out first.

Kind regards,
Ian

[Thomas Mueller]

Fixed in the GxInspect revision #5813

(Note that GxInspect moved to its own branch in svn: https://svn.code.sf.net/p/gexperts/code/GxInspect/trunk which is included as an external in GExperts)

Thanks for the analysis, it was right, and the problem turned out to be wider than the 64-bit IDE. Additional findings, measured with GxInspect attached to a paused process and reading the Call Stack window:

• Every IDE tested passes the cell text as WideString. The event's type information says var CellText: WideString in the 64-bit Delphi 13.2 IDE, the 32-bit Delphi 13.2 IDE and Delphi 2007. The IDE's copy of VirtualTrees (unit Idevirtualtrees in vclide<ver>.bpl) has kept the old declaration, whatever current VirtualTrees does. Keying the type on UNICODE was therefore wrong for every Unicode IDE, not just the 64-bit one.</ver>
• Your explanation of the padded rows is correct. With the handler declared as WideString, every row came back clean in all three IDEs, including rows of different lengths, without any cutting at #0. That workaround has been removed.
• Your memory concern is real. In the 64-bit IDE we deliberately forced the old string handler once. The IDE answered with "Invalid pointer operation" from System.@FreeMem / System.@UStrClr inside our handler, called from Idevirtualtrees.TCustomVirtualStringTree.DoGetText. That is the string release handing the IDE's WideString to the Delphi memory manager. The IDE had to be killed afterwards.
• You don't need System.Rtti for the check. The classic TypInfo data of a method type already has the parameter list: flags, name and type name for each parameter, in the same layout since Delphi 6. GxInspect now walks GetTypeData(PropInfo^.PropType^)^.ParamList. It requires a procedure with 5 parameters, only the last one var. It then installs a WideString handler if that type name is WideString, or a string handler if it is string or UnicodeString on a Unicode compiler. Anything else is refused with the declaration it found, instead of being hooked. This works for every IDE version we build a package for, so no version-specific constant is needed.

[Thomas Mueller]

Ticket moved from /p/gexperts/bugs/526/

Can't be converted:

• _milestone: Closed
• _priority: 5