Windows_Message_Dispatch_in_OWLNext

Windows Message Dispatch in OWLNext

To a large extent, the Windows API is based on message passing. Message dispatch concerns all the details from sending a message until it arrives at its destination, the message handler function. This article describes the mechanisms in OWLNext for routing a message on its way to its handler.

This article assumes you have basic knowledge of message handling in OWL applications. If not, it is recommended that you first read the article Understanding the Response Tables in ObjectWindows.

---

• Windows Message Dispatch in OWLNext
  • Message handling in Windows vs OWL
  • Forwarding messages to objects
    • Thunking — a mechanism to forward messages to member functions
      • How exactly do thunks work?
      • The Data Execution Prevention issue
    • Issues involved with subclassing
      • Aliasing and lifetime issues
      • Object reattachment issues
  • Response tables and the type problem
    • Borland's solution — cast away the type
    • The new solution — use templates
    • Notifications
      • Wildcards (UINT_MAX) in the response table
    • Future evolution
  • Using the new dispatch framework
    • Looking up dispatchers
    • User-defined dispatchers
    • Dispatchers for notifications
      • Defining notifications for custom controls
      • Defining DocView notifications
  • Implementing response tables without macros
  • Sending messages
    • Manual encoding
    • Aliasing existing HWND elements
    • Extending the dispatch lookup template framework
      • Sending messages in WM_USER space (e.g. Common Controls messages)
  • References

---

Message handling in Windows vs OWL

The basic GUI element in Windows is a window identified by its HWND handle. The program window, child windows, dialog boxes and controls are all windows. To affect the behaviour and appearance of these elements, the system sends messages through callbacks. There are primarily two kinds of such callbacks, which are called window procedures and dialog box procedures in Windows terminology.

The main difference between these callback procedures is how results are returned. In a dialog box procedure, the return type is BOOL. The procedure should return TRUE if a message is handled and FALSE otherwise. Actual message results are returned by calling SetWindowLongPtr with the index DWLP_MSGRESULT and the result as arguments. In contrast, a window procedure has return type LRESULT through which the result is directly passed, and there is no indication of whether the message was handled or not. It is just assumed to be.

While much of classic Windows message handling occurs within dialog box procedures, message handling in OWL occurs pretty much exclusively within window procedures — even for dialog boxes — as OWL deploys a customized window procedure for every encapsulated HWND handle. As such, OWL has a very regular design.

When consulting the Windows API documentation, it is important for OWL programmers to understand the difference between window procedures and dialog box procedures, and in particular how they differ in terms of message result return. Unfortunately, the Windows API documentation often assumes that messages are handled within a dialog box procedure, while in OWL the message will be handled within a window procedure.

For example, the documentation of the property sheet notification PSN_QUERYINITIALFOCUS shows the following example of how it should be handled, as if in the context of a dialog box procedure:

case PSN_QUERYINITIALFOCUS:
  SetWindowLongPtr(hDlg, DWLP_MSGRESULT, (LPARAM) GetDlgItem(hDlg, IDC_LOCATION));
  return TRUE;

However, an OWL event handler executes within the context of a window procedure, so will return the message result directly:

HWND PsnQueryInitialFocus(const TPshNotify&) // PSN_QUERYINITIALFOCUS
{
  return GetDlgItem(IDC_LOCATION);
}

---

Forwarding messages to objects

Now, let us look at how OWLNext manages to dispatch the message received in the Windows message queue to the associated TWindow-based object. Since the Windows API is plain C, it is a challenge to dispatch messages efficiently to the C++ objects where they should be handled. The obvious mechanism is to use a lookup table associating each window identifier (handle) with the C++ object that handles messages for that particular window. OWLNext originally had a build mode (BI_NOTHUNK) that used this approach, but the primary solution in OWLNext uses a more efficient mechanism based on thunking.

Thunking — a mechanism to forward messages to member functions

Thunks are small chunks of run-time generated code. OWLNext creates a thunk for each TWindow-based object. Each thunk embeds a hard-coded TWindow pointer, pointing to the object it was created for (the client object). By calling the thunk and passing the message, the thunk will forward it to the client object by calling a designated member function (TWindow::ReceiveMessage). This is very efficient since it involves just a function call.

How exactly do thunks work?

To send a message related to a particular interface element, the system sends it to the window procedure assigned to that element. An interface element is an actual Windows UI element identified by its HWND handle. A window procedure is simply a function with a particular signature (WindowProc). Initially, an element gets the window procedure defined by its window class. But it is possible to change the window procedure for an element at any time, and hence redirect messages for a particular element to another window procedure. Usually this window procedure will handle some messages and pass the rest back to the original window procedure. The process of installing a new window procedure for an element in this way is called subclassing in Windows terminology. For more details about the Windows architecture, consult the Microsoft Windows API documentation.

OWLNext uses subclassing and window procedures generated at run-time, i.e. thunks, to redirect messages for a particular interface element to the associated TWindow-based object. When a TWindow-based object is initialized, OWLNext generates a thunk for it. Then, as the object is associated with an interface element, i.e. during creation or aliasing, OWLNext installs the thunk by assigning it as the window procedure for the element. Basically, the thunk consists of just two simple machine instructions:

mov eax, _ptr_  ; Move the TWindow pointer into a register.
jmp _function_  ; Jump to the dispatch function.

Since every subclassed element has its own thunk with the associated TWindow pointer hard-coded within, messages are correctly dispatched. For the full details, consult the OWLNext source code.

The Data Execution Prevention issue

Thunks are generated at run-time and need to be placed in executable memory. For security reasons, modern processors and operating systems have introduced restrictive modes to prevent execution of data. On Windows this feature is called Data Execution Prevention (DEP). If enabled it will prevent OWL and old OWLNext programs from running because it blocks the thunks from executing. Support for DEP was introduced in OWLNext 6.20. See OWLNext and Data Execution Prevention for more details.

Issues involved with subclassing

The replacement of the window procedure for an element introduces aliasing, lifetime and reattachment issues. In some use-cases these issues can create problems, and may need careful consideration.

Aliasing and lifetime issues

Aliasing is a particularly difficult problem, because it allows a many-to-one relationship between several TWindow objects and a single interface element. This introduces complex interaction and lifetime issues. However, the thunking mechanism handles aliasing reasonably well, although with at least one exception outlined below.

An interface element is aliased when a window object is instantiated by passing an existing HWND handle. In this case the window object doesn't create the interface element nor own it. It just subclasses the element to receive its messages and control its behaviour, i.e. the window object installs its own window procedure and takes over message handling. Unhandled messages are passed on to the original window procedure. If the window object dies before the interface element, it restores the original window procedure. If the interface element dies before the window object, TWindow just cleans up and clears the handle.

Problems occur when more than one window object are associated with an interface element. A window object (B) may alias an interface element created by, or already aliased by, another window object (A). The following scenarios may then occur:

1. B dies before A, i.e. the objects have scoped lifetimes. Before destructing itself B should restore the original window procedure, so that A is again associated with the interface element, just as it was before B was created.
2. B outlives A, i.e. the objects have partially overlapping lifetimes. When A dies it should not restore the original window procedure, but notify B and pass the original window procedure, so that it can be restored when B dies.

A failure to handle these scenarios will leave the message dispatch mechanism in an undefined state. The thunking mechanism handles (1) correctly but not (2).

Fortunately (2) should be rare in application code. To run into the issue you would have to create A dynamically, then the B alias, then forcefully delete A before B. Explicit delete statements for window objects are not common in OWL. Window objects are automatically deallocated by OWLNext as a result of the default handling of WM_CLOSE.

Object reattachment issues

When a window object dies the destructor restores the original window procedure for the associated interface element. The destructor then frees the thunk. Note that this logic will fail if you have attached the object to another interface element in the meantime using TWindow::Handle, TWindow::SetHandle or TWindow::AttachHandle.

If you use Handle or SetHandle to attach an object to another interface element, it will leave the previous element with the window procedure installed by the object, i.e. the original window procedure will never be restored. The TWindows destructor will wrongly use the original window procedure for the previous element to try to restore the windows procedure for the new element. Also, messages for the new element will not be dispatched to the object. Instead the object will continue to receive messages for the old element.

If you use AttachHandle to attach an object to another interface element, OWLNext will install a new window procedure for the new element, but will not restore the original window procedure for the old element. OWLNext will continue to dispatch messages for the old element to the object. The result is that the object will receive messages for both elements. The original window procedure of the old element will never be restored.

The upshot is that you need to test well to make sure that your code is free from such issues. Do not reattach a window object!

---

Response tables and the type problem

After a typical message is received in the application message loop, it is routed through the following functions:

1. TApplication::ProcessMsg
2. ::DispatchMessage (calls the registered WindowProc associated with the HWND)
3. WindowProc (this is the thunk )
4. TWindow::ReceiveMessage (non-virtual)
5. TWindow::WindowProc (virtual, may be overridden)

You can think of this as the thunk acting as a postman delivering a letter to a family's letterbox. Now the letter needs to be forwarded to the appropriate family member. This task is handled in TWindow::WindowProc, where the message is forwarded using the response table for the associated TWindow-based class.

An entry in the response table has two duties:

1. Designate a member function to handle a particular message.
2. Designate the appropriate dispatch function for decoding ("cracking") the message arguments.

Both of these duties introduce a type problem, since the type of a member function pointer depends on the class and function signature. Also, since a dispatcher's job is to forward the message to a member function, the obvious solution is to hand it the member function pointer. Thus, the dispatcher signature will also depend on the type of the member function pointer. The resulting lack of common types to use for the response table is a problem.

Borland's solution — cast away the type

In OWL 5, and OWLNext up to and including the 6.30 series, the solution to the response table type problem is to do brute-force casting, and by that force pointers into the same type (a dummy type). In essence, this solution stores the following information in a response table entry:

1. The message identifier (and possibly further discriminating info for notifications)
2. A cast pointer to the designated member function
3. A cast pointer to the appropriate dispatcher (decoder) for this message

Now, when the message arrives, TWindow::WindowProc searches the response table for a matching entry, and if found, calls the looked up dispatcher, passing the designated member function pointer along. The dispatcher then decodes the message arguments and passes them on to the designated member function.

Unfortunately, due to the brute-force casting required, this mechanism invokes undefined behaviour (UB) according to the C++ standard. Whether the casting works as expected depends on implementation detail in the compiler. In other words, it may break as soon as a different compiler, or compiler version, is used, even though that compiler is fully conformant with the standard (see "UB in message dispatch" [bugs:#235]).

The type erasure here creates another problem; how to type-check the signature of a given member function when the user inserts it into the response table. To solve this problem, the member function pointer is, at the point of insertion into the table, passed through a corresponding signature function that does nothing except enforce the type of the pointer. Unfortunately, the signature of the dispatcher is not checked, so there is a big potential for unintended mismatches between dispatcher and member function.

For example, here is the dispatcher and signature function used for WM_HELP:

_OWLFUNC(LRESULT)
v_LPARAM_Dispatch(TGeneric& i, void (TGeneric::*f)(LPARAM), WPARAM, LPARAM p2)
{return (i.*f)(p2), 0;}

template <class T>
void (T::*v_HELPINFO_Sig(void (T::*pmf)(const HELPINFO&)))(const HELPINFO&)
{return pmf;}

Note that here the dispatcher calls an undefined member on an undefined class TGeneric. Also, note that there is a mismatch between the signature specified by the v_HELPINFO_Sig function template (void (T::*) (const HELPINFO&)) and the member function pointer parameter in the dispatcher (void (TGeneric::*) (LPARAM)). This is common in this solution. Dispatchers are rarely written for a particular message, but instead a dispatcher is reused for many messages, provided it just has "wide enough" parameters for the member function signatures. This is hard to review and maintain.

The new solution — use templates

For OWLNext 6.40, we have designed a new solution that does not invoke UB, and which is simpler to review and maintain (in particular, this is important for the new 64-bit support). In this solution, a response table entry essentially consists of the following information:

1. The message identifier (and possibly further discriminating info for notifications)
2. A pointer to a customized dispatch function (for this particular message and the designated member function)

By using a template to generate a dispatch function particular to the designated member function, we do not need to pass the member function to the dispatcher when forwarding the message. The generated dispatcher knows the object class and member function it was instantiated for. Hence we do not need to store the member function pointer in the table. We only need to store a pointer to the dispatcher. And since the dispatcher no longer has the member function pointer as part of its signature (since it is not passed), we can use a common signature for all dispatchers, i.e.

typedef TResult TDispatcher(void* object, TParam1, TParam2);

Note that the pointer to the TWindow-based object is passed to the dispatcher as a void pointer, but this does not invoke a UB problem. The C++ standard allows us to cast an object pointer to void pointer and back without losing information. So when we look up an entry in the response table for a particular object, we cast the object pointer to void (this happens in the class member function Find, which is an override of TEventHandler::Find) and pass it to the dispatcher (this happens in TEventHandler::Dispatch). The dispatcher then casts the void pointer back to its original type. The dispatcher knows the original type, because it was instantiated with this type.

With this solution, the need for the signature function templates used by the Borland solution vanishes. The dispatch function template enforces the correct signature for the designated member function. This removes a big source of potential errors due to unintended mismatches between dispatcher and signature functions in the Borland solution.

What does a dispatch function template look like? Here is the dispatch function template for WM_HELP:

template <class T, void (T::*M)(const HELPINFO&)>
static TResult Decode(void* i, TParam1, TParam2 p2)
{return (static_cast<T*>(i)->*M)(*reinterpret_cast<HELPINFO*>(p2)), TRUE;}

This is the compact style in which dispatch function templates are implemented, but let us break it out for readability:

template <class T>
using THandler = void (T::*)(const HELPINFO&); // Member function signature.

template <class T, THandler<T> M>
static TResult Decode(void* i, TParam1, TParam2 p2)
{
  T* obj = static_cast<T*>(i); // Restore the type of the instance.
  const HELPINFO& helpInfo = *reinterpret_cast<HELPINFO*>(p2); // Decode parameter.
  (obj->*M)(helpInfo); // Call the member function, passing the decoded argument.
  return TRUE; // Should always return TRUE for WM_HELP (see Windows API).
}

Note that the template will only accept a member function pointer (M) of the correct signature, and that since the template knows the class (T), it can safely cast the void pointer (i) back to its original type and perform a type-safe call.

Notifications

What are notifications? Basically, notifications are a set of messages sent through the same shared carrier message. The carrier message includes a notification code as part of its arguments, identifying the particular notification the message represents, and an identifier code which represents the sender. Each notification can specify a set of further parameters encoded in the generic message arguments.

Windows uses primarily two standard carrier messages for notifications: WM_COMMAND and WM_NOTIFY.

OWLNext provides general response table macros for notifications sent through these messages. For example, EV_CHILD_NOTIFY (id, notifyCode, method) creates a response table entry for notifications sent through WM_COMMAND, and EV_CHILD_NOTIFICATION (notificationCode, sourceId, method) creates a response table entry for notifications sent through WM_NOTIFY. In general, since each notification may encode its own set of parameters in the message arguments, it requires its own handler signature and dispatcher (message decoder). The response table macros will look up and validate the correct signature of the handler function (method).

OWLNext also provides dedicated macros for each specific notification code. For example, EV_BN_CLICKED creates a response table entry for the BN_CLICKED notification sent through WM_COMMAND, and EV_NM_CLICK creates a response table entry for the NM_CLICK notification sent through WM_NOTIFY. These macros are in turn implemented in terms of EV_CHILD_NOTIFY (for WM_COMMAND) and EV_CHILD_NOTIFICATION (for WM_NOTIFY).

We want to dispatch a notification based on the sender, so that we can handle notifications from different sources separately. For example, we may only be interested in handling mouse click notifications from a particular control in our dialog box. For this reason, response table entries for notifications include a identifier code for the sender of the message. This is specified as the id or sourceId in the parameter list of the response table macros for notifications. As part of message dispatch and response table lookup, OWLNext matches this identifier with the actual sender of the notification, so that each entry only handles messages from a particular source.

Wildcards (UINT_MAX) in the response table

To allow a single response table entry to handle all notification codes from a single source, you can specify UINT_MAX for the notification code in the response table entry (TResponseTableEntry::NotifyCode). Note that this requires a common handler signature for all the notifications, which is generally not true. Hence this feature is only used for notifications sent through WM_COMMAND, for which all of the notifications have a single and same parameter: the notification code. The response table macro EV_CHILD_NOTIFY_ALL_CODES is the only macro that makes use of this.

It is not possible to have a single entry that handles notifications from multiple sources. If you want to do that, you have to handle the carrier message itself. Alas, there are no response table macros for that (i.e. EV_WM_COMMAND and EV_WM_NOTIFY are not provided), but you can override TWindow::EvCommand and TWindow::EvNotify. Note that the generic EV_MESSAGE response table macro does not work for WM_COMMAND and WM_NOTIFY, as these notification messages are intercepted and treated specially by the dispatch machinery.

Future evolution

In Owlet, our experimental branch, we have made further improvements by simplifying the current response table implementation and increasing type-safety.

For example, we make use of std::function to further improve our solution to the response table type problem. The std::function class is able to encapsulate any callable entity, and by using it as the return type for response table lookup, we can adapt the type of each response table to the containing class and avoid using void pointers in the dispatcher signatures. With this change the signature of a dispatcher becomes a template:

template <class T>
using TDispatcher = auto (T* i, TParam1, TParam2) -> TResult;

Hence now the type of the signature of the dispatcher depends on the class it relates to. As a result we can simplify the implementation of each dispatcher and avoid casting the object pointer. For example, for WM_HELP the dispatcher changes to this:

template <class T, void (T::*M)(const HELPINFO&)>
static TResult Decode(T* i, TParam1, TParam2 p2)
{return (i->*M)(*reinterpret_cast<HELPINFO*>(p2)), TRUE;}

Compare this implementation with the implementation in the last section.

Regarding notifications, we have simplified the response table entry structure and the lookup machinery. Previously, the members Msg and NotifyCode in TResponseTable were in a union. This meant that notifications were assumed to be sent through WM_COMMAND or WM_NOTIFY, and that the dispatch of notifications sent through other (custom) messages were not supported. Now we have separated the message identifier and the notification code, so that they have separate fields in the response table entries. This makes the dispatch logic more regular and simplifies the machinery.

Incidentally, this simplification generalizes the use of wildcards (UINT_MAX) for the notification code. It is no longer a special case for WM_COMMAND and WM_NOTIFY. We also allow UINT_MAX as a wildcard for the identifier code of the sender (source). This makes it possible to handle notifications from multiple sources in a single handler.

Regarding the implementation of the dispatch templates, we have made use of new features in modern C++ to abstract and simplify the implementation further. Currently, there is redundancy in the implementation of a dispatch template specialization due to the need to specify the handler signature three times: In the encoder, in the signature check and in the decoder. For example:

template <> struct TDispatch<WM_ENDSESSION>
{
  template <class F>
  static void Encode(F sendMessage, HWND wnd, bool endSession, uint flags)
  {sendMessage(wnd, WM_ENDSESSION, endSession ? TRUE : FALSE, static_cast<TParam2>(flags));}

#if OWL_EV_SIGNATURE_CHECK

  template <class T> struct THandler {typedef void (T::*type)(bool endSession, uint flags);};

#endif

  template <class T, void (T::*M)(bool endSession, uint flags)>
  static TResult Decode(void* i, TParam1 p1, TParam2 p2)
  {return (static_cast<T*>(i)->*M)(static_cast<bool>(p1), static_cast<uint>(p2)), 0;}
};

We now instead deduce the THandler signature from the Encode function alone using modern C++ meta-programming. This deduction is done in a base class, removing the need to explicitly define the handler signature in the dispatch specialization at all:

template <> struct TDispatch<WM_ENDSESSION> : TDispatchBase<WM_ENDSESSION, void>
{
  template <class F>
  static auto Encode(F sendMessage, HWND wnd, bool endSession, uint flags) -> THandlerResult
  {sendMessage(wnd, MessageId, endSession ? TRUE : FALSE, static_cast<TParam2>(flags));}

  template <class T, THandler<T> M>
  static auto Decode(T* i, TParam1 p1, TParam2 p2) -> TResult
  {return (i->*M)(static_cast<bool>(p1), static_cast<uint>(p2)), 0;}
};

In this case, the THandler signature is deduced to void (T::*)(bool endSession, uint flags) based on THandlerResult and the parameters following HWND wnd in the parameter list of the Encode function. THandler is then used in the specification of the template parameters for Decode, obviating the need to specify the signature explicitly.

MessageId and THandlerResult are also defined in the base class TDispatchBase, based on the passed template arguments (WM_ENDSESSION and void).

---

Using the new dispatch framework

The new framework requires little change in the actual use of response tables, but in some cases, such as with user-defined messages and response table macros, some changes are required.

Looking up dispatchers

The Borland solution requires every response table macro to explicitly specify the dispatcher that should be used for a particular message. For example, here is the response table macro for WM_HELP:

#define EV_WM_HELP\
  {{WM_HELP}, 0, (::owl::TAnyDispatcher) ::owl::v_LPARAM_Dispatch,\
  (TMyPMF)::owl::v_HELPINFO_Sig<TMyClass>(&TMyClass::EvHelp)}

As we can see, it uses the dispatcher and signature function discussed in the previous section (which, by the way, we identified were mismatching; although it magically works). Having this complexity for every response table macro is clearly error-prone. With the new dispatch framework, the corresponding macro is much simpler:

#define EV_WM_HELP OWL_EV_(WM_HELP, EvHelp)

Here we use a generic macro, OWL_EV_, that automatically looks up the correct dispatcher for the message at compile-time. This is achieved using template specialization. All the standard messages have their own specialization of a dispatch lookup template, TDispatch, in which the dispatch function for that particular message is defined. For example, TDispatch <WM_HELP> looks like this:

template <> struct TDispatch<WM_HELP>
{
  template <class T, void (T::*M)(const HELPINFO&)>
  static TResult Decode(void* i, TParam1, TParam2 p2)
  {return (static_cast<T*>(i)->*M)(*reinterpret_cast<HELPINFO*>(p2)), TRUE;}
};

You may recognize the dispatch function discussed previously. By defining it within a template specialization we enable compile-time lookup. This approach is also open to extension in the future by allowing the introduction of further message-specific functionality (such as encoding) within the template specialization for each message.

User-defined dispatchers

You can extend the TDispatch template with specializations for your own custom messages. For example, here is the definition of a dispatcher and response table macro for a custom message MY_WM_FOO, passing an int in TParam1 and returning a bool:

namespace owl {

template <> struct TDispatch<MY_WM_FOO>
{
  template <class T, bool (T::*M)(int)>
  static TResult Decode(void* i, TParam1 p1, TParam2)
  {return (static_cast<T*>(i)->*M)(static_cast<int>(p1)) ? TRUE : FALSE;}
};

} // namespace

#define EV_MY_WM_FOO OWL_EV_(MY_WM_FOO, EvFoo)

Note that the dispatcher is completely encapsulated, and although we here extend the framework, there is no coupling with the dispatchers for other messages. The dispatcher is fully self-contained. As always, you simply need to make sure that the MY_WM_FOO message identifier does not clash with other predefined or custom messages, otherwise you will get a template specialization error for TDispatch <MY_WM_FOO> (duplicate specialization attempt).

Alternatively, you can use free-standing dispatchers for your custom messages, or you can define your own dispatch lookup template. In these cases, you cannot use OWL_EV_ for your response table macro. Instead, you must write out the code for the response table entry manually. For example, if you have a free-standing dispatcher called Decode_MY_WM_FOO, you can define your response table macro as follows:

#define EV_MY_WM_FOO\
  {{MY_WM_FOO}, 0, OWL_DISPATCH(Decode_MY_WM_FOO, EvFoo)}

If you provide your own dispatch lookup template, e.g. TMyDispatch, the response table macro should be defined like this:

#define EV_MY_WM_FOO\
  {{MY_WM_FOO}, 0, OWL_DISPATCH(TMyDispatch<MY_WM_FOO>::Decode, EvFoo)}

Here the OWL_DISPATCH macro simply instantiates the dispatch function template with the current class and the given member function. For more details about the response table macros, see the source code.

Note that, by following the conventions of the dispatch framework, you may be able to reuse functionality planned for the future, such as a SendMessage overload that does automatic argument type-checking and encoding. For this reason you should prefer using the dispatch lookup approach to define your dispatchers. See Extending the dispatch lookup template framework later in this article.

Dispatchers for notifications

Each notification can specify a set of further parameters encoded in the generic message parameters. In general, each notification hence requires its own handler signature and dispatcher (message decoder).

This means that looking up dispatchers based on message identifier alone is not enough. For notifications, we need to further look up the dispatcher based on the notification code. To do this we use a nested dispatch template, TNotificationDispatch, defined within the dispatch template specialization for the carrier message.

For example, the WM_NOTIFY message is used by the Windows Common Controls to send information from the control to its parent. The dispatcher lookup template for WM_NOTIFY is defined as follows:

template <> struct TDispatch<WM_NOTIFY>
{
  //...

  template <uint NotificationCode>
  struct TNotificationDispatch;
};

template <> struct TDispatch<WM_NOTIFY>::TNotificationDispatch<NM_CHAR> /*...*/;
template <> struct TDispatch<WM_NOTIFY>::TNotificationDispatch<NM_MOUSE> /*...*/;
//...

To lookup the dispatcher for a notification, say NM_CHAR, we can now do as follows:

#define EV_NM_CHAR(id, method)\
  {{NM_CHAR}, id, OWL_DISPATCH(TDispatch<WM_NOTIFY>::\
    TNotificationDispatch<NM_CHAR>::Decode, method)}

Other notification carriers, such as WM_OWLNOTIFY, are handled with a similar approach.

Defining notifications for custom controls

If you use WM_NOTIFY to send notifications from your custom controls (e.g. by using TWindow::SendNotification), note that notification codes across controls may not be unique and may hence clash if you try to create a specialization for TDispatch <WM_NOTIFY>::TNotificationDispatch. For this reason, you should define a custom dispatch template for each custom control. This is pretty easy though:

#include <owl/commctrl.h>

template <owl::TMsgId>
struct TCustomDispatch;

template <>
struct TCustomDispatch<WM_NOTIFY>
{
  template <owl::uint NotificationCode>
  struct TNotificationDispatch;
};

template <>
struct TCustomDispatch<WM_NOTIFY>::TNotificationDispatch<NM_CUSTOM_EVENT>
  : owl::TDispatch<WM_NOTIFY>::TNotificationDispatchBase<NM_CUSTOM_EVENT, void, TCustomEventParam&>
{};

#define EV_CUSTOM_NOTIFICATION(notificationCode, sourceId, method)\
  {{notificationCode}, sourceId,\
  OWL_DISPATCH(TCustomDispatch<WM_NOTIFY>::\
    TNotificationDispatch<notificationCode>::Decode, method)}

#define EV_CUSTOM_EVENT(sourceId, method)\
  EV_CUSTOM_NOTIFICATION(NM_CUSTOM_EVENT, sourceId, method)

Here we assume that TCustomEventParam is inherited from NMHDR (or the encapsulation TNotify), as required by WM_NOTIFY.

If you want robust type casting and enhanced signature checking in your response table macro, the implementation gets a little bit more complicated. See the definition of OWL_EV_NOTIFICATION.

In the code above, note that we reuse TDispatch <WM_NOTIFY>::TNotificationDispatchBase, since it already has the code to decode notifications. If you have many notifications, you can simplify your code by redefining this base within TCustomDispatch <WM_NOTIFY>. If you use a compiler with support for C++11 or later, you can use an alias template. If not, you can use inheritance:

template <>
struct TCustomDispatch<WM_NOTIFY>
{
  template <owl::uint NotificationCode, class THandlerResult, class TParam>
  struct TNotificationDispatchBase
    : owl::TDispatch<WM_NOTIFY>::TNotificationDispatchBase<NotificationCode, THandlerResult, TParam>
  {};  

  template <owl::uint NotificationCode>
  struct TNotificationDispatch;
};

Then you can simplify your specializations like so:

template <>
struct TCustomDispatch<WM_NOTIFY>::TNotificationDispatch<NM_CUSTOM_EVENT>
  : TNotificationDispatchBase<NM_CUSTOM_EVENT, void, TCustomEventParam&>
{};

Of course, you don't have to use TNotificationDispatchBase. If you need to do something special, you can provide your own dispatch implementation.

Defining DocView notifications

DocView notifications are sent through the custom WM_OWLNOTIFY message. Since each notification may encode different arguments, we need to look up the dispatcher based on the particular notification code. For the predefined notification codes, OWLNext uses the conventions discussed in the previous section, i.e. a nested dispatch lookup template within TDispatch <WM_OWLNOTIFY>. See the source code for the details.

For user-defined DocView notifications, old code must be rewritten to use the new dispatch framework. For example, to define a notification vnFoo, that passes a TFoo pointer and returns bool, do as follows:

const uint vnFoo = vnCustomBase + 0;

template <TMsgId>
struct TMyDispatch; // This dispatch lookup template may be declared elsewhere.

template <> struct TMyDispatch<WM_OWLNOTIFY>
{
  template <uint NotificationCode>
  struct TNotificationDispatch;
};

template <> struct TMyDispatch<WM_OWLNOTIFY>::TNotificationDispatch<vnFoo>
{
  template <class T, bool (T::*M)(TFoo*)>
  static TResult Decode(void* i, TParam1, TParam2 p2)
  {return (static_cast<T*>(i)->*M)(reinterpret_cast<TFoo*>(p2)) ? TRUE : FALSE;}
};

#define EV_VN_FOO\
  VN_DEFINE(vnFoo, VnFoo, TMyDispatch<WM_OWLNOTIFY>::TNotificationDispatch<vnFoo>::Decode)

By following the conventions of the dispatch framework like this, you may be able to benefit from functionality planned for the future, such as a SendNotification overload that does automatic argument type-checking and encoding. See Extending the dispatch lookup template framework later in this article.

---

Implementing response tables without macros

OWL implements response tables with heavy use of macros. This is unfortunate in many ways. Macros are implemented by the preprocessor and hides the underlying C++ code, making it hard to see what is going on. Macro-free code is much more friendly to both the user and C++ tools, such as the compiler (for error reporting), debuggers (for stepping through code), smart editors (e.g. Microsoft's IntelliSense) and code formatters. So, let's have a look at how we can eliminate much of the response table macros and implement our tables explicitly with mostly C++ code.

For example, consider the following typical implementation of a response table:

class TMyWindow
  : public TWindow
{
public:
  TMyWindow(...);

protected:
  void EvSetFocus(HWND focusLoser);

  DECLARE_RESPONSE_TABLE(TMyWindow);
};

DEFINE_RESPONSE_TABLE1(TMyWindow, TWindow)
  EV_WM_SETFOCUS,
END_RESPONSE_TABLE;

The response table macros here basically implements an override of TEventHandler::Find. We can get rid of most of the macros by implementing this override explicitly:

class TMyWindow
  : public TWindow
{
public:
  TMyWindow(...);

  auto Find(TEventInfo& eventInfo, TEqualOperator op) -> bool override
  {
    // Implement and search response table.
    //
    eventInfo.Object = this; // Important for correct dispatch.
    using TMyClass = TMyWindow; // Alias used by response table macros.
    static const auto responseTable = std::initializer_list<TResponseTableEntry>
    {
      EV_WM_SETFOCUS,
      {} // Sentinel is required.
    };
    return SearchEntries(responseTable.begin(), eventInfo, op) ||
      TWindow::Find(eventInfo, op);
  }

protected:
  void EvSetFocus(HWND focusLoser);
};

For convenience, we still use the familiar macros for the response table entries, since a bit of magic is going on in their implementation (in particular, the macros implement signature checking to improve compiler error messages). Typing out the response table entries would hence be quite a hassle. Also, there is not much code executed in the table entries, so these macros do not matter much for debugging and code understanding.

Note the need to assign eventInfo.object and to declare the type TMyClass. The latter class name is used in the response table entry macros. Also note the need to end the table with an empty entry (sentinel). TEventHandler::SearchEntries expects this sentinel to mark the end of the table.

We have to explicitly specify how the table is searched. First we search our own table by calling SearchEntries, and if that fails to find a handler, we then call Find in the base class. If you have multiple inheritance, then you can control the search order of the bases here. Actually, you can implement any kind of lookup logic here! You can override the plain table search, filter messages, and more.

For information about completely macro-free response tables in Owlet, our experimental branch, see Response tables without macros.

---

Sending messages

Now, let us turn our focus to the other end of dispatch; creating the message and sending it. This has its own set of problems, in particular the need to encode the message arguments correctly. OWL did not provide any general framework for this, but by convention encapsulated many messages as member functions of the related classes.

Manual encoding

For example, TButton::SetImage encapsulates the BM_SETIMAGE message. This is all nice and well, as long as you have a TButton instance. If you just have a raw HWND for a button from somewhere (e.g. GetDlgItem (IDC_BUTTON)), and you need to send BM_SETIMAGE, one solution is to resort to sending the message manually, being careful to encode the arguments correctly:

HWND b = GetDlgItem(IDC_BUTTON);
HANDLE newImage = /*...*/;
TImageType t = /*...*/;
SendMessage(b, BM_SETIMAGE,
  static_cast<TParam1>(t), 
  reinterpret_cast<TParam2>(newImage));

This is error-prone, and while manageable in this example, some messages require far more complex encoding, where several arguments are packed into TParam1 and/or TParam2, and where the encoding may depend on the actual values of the arguments (e.g. for notifications).

Even worse, the correct encoding has to be repeated on every call site for the same message. See, for example, TEdit::SetLeftMargin, SetRightMargin and SetMarginUseFontInfo in the OWLNext source code. All three implementations send the same message EM_SETMARGINS, and all three do manual encoding. This is hard to review and maintain. Ideally, the encoding details should be encapsulated in one place and reused.

Aliasing existing HWND elements

OWL provides a feature it calls aliasing. This feature allows you to instantiate a TWindow object given an existing HWND handle, and thus subclass the associated UI element. You can then use the TWindow member functions to interact with the element, instead of using raw SendMessage calls. OWLNext 6.32 extended this feature to almost all controls. So for our example above, we can now do this:

TButton b(GetDlgItem(IDC_BUTTON));
HANDLE newImage = /*...*/;
TImageType t = /*...*/;
b.SetImage(t, newImage);

Note however that subclassing an element is an expensive operation. OWLNext must instantiate a full-blown TWindow object, create and install a thunk to intercept the message handling for the HWND element, and then route all messages addressed to the element through its complex dispatch framework. So aliasing is highly inefficient, and probably slower by order of several magnitudes, compared to directly calling SendMessage.

Ideally, we should not have to use aliasing for such basic functionality as sending a message. And we should not have to resort to manual and error-prone message encoding and basic SendMessage calls. We need a low-level framework dealing specifically with message encoding.

Extending the dispatch lookup template framework

In Owlet, our experimental branch, we have extended the framework to include both encoding and decoding of messages. Using the example above, this extension of the framework allows you to simply say:

HWND b = GetDlgItem(IDC_BUTTON);
HANDLE newImage = /*...*/;
TImageType t = /*...*/;
SendMessage<BM_SETIMAGE>(b, t, newImage); // Fully type-checked!

Here you pass the message identifier as a template argument, thus allowing the correct parameter list for the message to be looked up at compile-time. The arguments are then type-checked and forwarded to the encoder, which in turn will forward the encoded message to the raw SendMessage function provided by the Windows API. Similar template overloads are provided for the other message functions, such as PostMessage, SendNotification, etc. All of these templates reuse the common new dispatch framework.

This solution is optimally efficient. The code to do the encoding is instantiated and (with high probability) in-lined at the call site, incurring no overhead, compared to manual encoding and a raw SendMessage call.

How does it work? The TDispatch specialization for each message now also has an encoder that does the message encoding and forwards the message to a given function. E.g. for BM_SETIMAGE it looks like this:

template <> struct TDispatch<BM_SETIMAGE> : TDispatchBase<BM_SETIMAGE, HANDLE>
{
  template <class F>
  static THandlerResult Encode(F sendMessage, HWND wnd, int type, HANDLE image)
  {
    return ConvertResult<THandlerResult>(sendMessage(wnd, MessageId, 
      MakeParam1(type), 
      MakeParam2(image)));
  }
};

This function template encapsulates all the details of encoding this message correctly, and it supports forwarding the encoded arguments to any sender function that has a compatible signature. If the signature does not match, a simple adapter (e.g. lambda) may be used. This means that this encoder can be reused for any function that sends messages, such as PostMessage, SendNotification, etc.

The template-based SendMessage overload does the lookup of the encoder and passes the arguments along:

template <TMsgId M, template <TMsgId> class D = TDispatch, class... A>
auto SendMessage(HWND w, A&&... a)
{return D<M>::Encode(&::SendMessage, w, std::forward<A>(a)...);}

As mentioned before, similar overloads are provided for other functions for sending messages.

Sending messages in WM_USER space (e.g. Common Controls messages)

As discussed before, the new framework is based on specializing a dispatch lookup template for each message using the message identifier as a template parameter. The observant reader may have noted that this relies on message identifiers being unique, i.e. that there is a one-to-one mapping between message identifier and message. Unfortunately, only certain ranges of the Windows message identifier type are guaranteed to be unique. Some sub-ranges are free to be used by private window classes and applications. Here is an overview taken from MSDN:

Range	Meaning
0 through 0x03FF	Messages reserved for use by the system.
WM_USER (0x0400) through 0x7FFF	Integer messages for use by private window classes.
WM_APP (0x8000) through 0xBFFF	Messages available for use by applications.
0xC000 through 0xFFFF	String messages for use by applications.
Greater than 0xFFFF	Reserved by the system.

In particular, the Common Controls messages are not defined within the system range, but all live within the WM_USER range, and unfortunately, the use of identifiers in this range is overlapping. The upshot of this is that we cannot rely on a single dispatch lookup template for all messages. Common Controls, custom controls and applications that define messages outside the system range need to define their own dispatch lookup template.

For example, the Header Control uses the dispatch lookup template TDispatch_HDM. When sending Header Control messages (HDM...) we need to explicitly specificy TDispatch_HDM in the call. E.g. to send the HDM_SETHOTDIVIDER message, do as follows:

SendMessage<HDM_SETHOTDIVIDER, TDispatch_HDM>(index);

Here, the SendMessage overload will look up the correct encoder for the HDM_SETHOTDIVIDER message within the TDispatch_HDM dispatch lookup template, and then forward the arguments (the index) to this encoder. If there is a mismatch between the argument list and the signature of the encoder, you will get a compilation error.

---

References

• OWLNext and Data Execution Prevention
• OWLNext Mailing List Discussions
• Ted Neward (1997), "Core OWL 5.0", Manning Publications Co., Section 5.3: "Finding object from widget".
• Microsoft Windows API documentation

[Vidar Hasfjord]

Possible solutions to aliasing issues

by [[User:Vattila|Vattila]] 15:47, 19 March 2010 (UTC)

A solution to the first aliasing issue (1) for the BI_NOTHUNK option requires TWindow to not only store a pointer to the original window procedure, but also a pointer to the window object that owns it, if any. The object pointer is needed to update the look-up table. This is unlike the thunking solution, where the window object pointer is part of the window procedure itself.

A solution to the second issue (2) is harder. It requires a notification protocol between all the aliases and the owner (if any) of an interface element, so that an alias is notified of the death of the original window object whose window procedure it stores, and is given its original window procedure as a replacement, i.e. the window procedure that was active before the dying object subclassed the element. Using the example in the article, A notifies B about its death and passes on the original window procedure that was active when A subclassed the element. When B finally dies it restores this windows procedure. The element is thus left in the state it was before A and B subclassed it.

There is currently no ongoing effort to fix these issues.

Possible solutions to reattachment issues

by [[User:Vattila|Vattila]] 15:47, 19 March 2010 (UTC)

Clean-up must be done while the handle is still valid. Otherwise we have no access to the interface element, which leaves us unable to restore the original window procedure. The BI_NOTHUNK option also needs to remove the window object from the look-up table and replace it by the entry for the original object, if any.

A possible approach to control proper clean-up and reattachment is to handle it in SetHandle. The idea is that clean-up before reattachment can then be ensured by enforcing the rule that all handle assignments are done by calling SetHandle, and that a SetHandle (0) is executed wherever detachment and clean-up is required (mainly in the destructor). The poor encapsulation and the myriad of possible execution paths through the TWindow code makes this tricky to ensure, so this solution may entail some refactoring of the code.

Also, unless OWLNext is compiled with the OWL_STRICT_DATA option, the handle is unfortunately public, so client code may circumvent proper management of clean-up and reattachment by modifying the handle directly. Hence a robust solution requires that OWL_STRICT_DATA becomes the default and only disabled for compatibility modes.