#637 Fix: Dockable toolbar drag guide invisible or misplaced under DWM / high-DPI (Windows 8+)

[Hideaki KODATSU]

Problem

When dragging a dockable toolbar (TDockableControlBar) on Windows 8 and
later, the drag guide rectangle malfunctions in two related ways:

1. Invisible under DWM compositing -- The guide is drawn with
   PatBlt/PATINVERT (XOR painting) via TScreenDC. Under DWM desktop
   compositing, XOR operations on the screen DC are not reflected in the
   composition layer, so the guide simply does not appear.

2. Misplaced on high-DPI displays -- Even when the XOR painting is
   visible, the screen coordinates passed to PatBlt are not adjusted for
   DPI scaling. On a high-DPI monitor the guide is drawn at the wrong
   position, shifted to an unrelated area of the screen.

Both issues share the same root cause: TScreenDC + PatBlt/PATINVERT is
fundamentally incompatible with the DWM compositing model used since
Windows 8.

Fix

Replace the XOR-painted guide with a layered popup window
(WS_EX_LAYERED | WS_EX_TOPMOST | WS_EX_NOACTIVATE). The window is
filled with a solid color at 50% transparency and clipped to a hollow
frame shape via SetWindowRgn. Because it is a real window positioned in
screen coordinates, the OS handles DPI and compositing correctly with no
manual adjustment needed. It is created on drag begin and destroyed on
mouse up -- no explicit "erase" pass is needed.

Changes:

• source/owlcore/docking.cpp: Remove TDitherBrush, GetDitherBrush, and
  fastWindowFrame. Add CreateDragGuide, MoveDragGuide, and
  DestroyDragGuide helpers in an anonymous namespace.
• include/owl/docking.h: Add HWND DragGuideHwnd to THarbor. Mark
  TWindowDC* DragDC as [[deprecated]] -- it is kept (and still
  allocated) to avoid breaking derived classes that reference it.

The patch for 7.0.20 is attached.

This patch and report were created with the assistance of Claude Code
(Claude Sonnet 4.6 by Anthropic).

[Ognyan Chernokozhev]

Thanks for the fix. I applied it to the trunk and tested it on machines with both regular and high-DPI screens, and it looks good.

My only concern about applying it to the 7 branch is that due to adding a new member to THarbor, that may break the ABI (Application Binary Interface) - see also https://sourceforge.net/p/owlnext/wiki/Frequently_Asked_Questions/#h-what-is-a-beta-release-and-should-i-use-it-in-production

[Hideaki KODATSU]

Revised patch for 7.0.20 attached -- same fix, no ABI change (docking.h untouched).

The previous patch added HWND DragGuideHwnd to THarbor and marked DragDC [[deprecated]], both of which change the class layout or interface and break ABI.

This version stores the drag-guide HWND in a translation-unit-local map instead:

static std::unordered_map<THarbor*, HWND> s_dragGuideHwnds;

DragDC is left as-is and still allocated, serving only as a non-null sentinel for the existing if (DragDockable && DragDC) guards.

[Ognyan Chernokozhev]

Many thanks, the patch works great!

Path applied in [r8706].

[Vidar Hasfjord]

@jogybl committed the patch by @neige68 in [r8654].

Could the same translucent window technique be used to improve TSplitterIndicator? Like the original toolbar drag indicator, TSplitterIndicator currently uses the old outdated XOR technique which causes flicker. See e.g. FolderSize.