Safe_Transfer_Buffers

Safe Transfer Buffers

OWL provides a mechanism called transfer buffers to help the programmer get data in and out of dialog boxes. Unfortunately, transfer buffers are known to be error prone due to a lack of type safety. Consequently, transfer buffers have been widely criticized, and users have been advised to use alternative solutions [1]. Here we look closer at the problem the transfer buffer mechanism was meant to solve, the flaws in its design and implementation, and the changes made in OWLNext to make transfer buffers safe.

• Safe Transfer Buffers
  • Communicating with dialogs
  • The OWL transfer buffer mechanism and its flaws
    • Usage
      • An aside about string buffers
    • Design flaws
    • Implementation flaws
      • No type checking
      • Associations between field and controls are implicit
      • The buffer is assumed to have standard layout
      • The buffer is assumed to be byte-packed
      • Transfers are prone to buffer underflow and overflow
      • Validator meddling
  • Making transfer buffers safe
    • Usage
      • Safe string handling
      • Alternative ways to bind
      • Specializing field binding for custom control types
    • Rules and limitations
  • Conclusion
  • Example code
  • References

---

Communicating with dialogs

OWLNext provides a solid set of control encapsulation classes to simplify the manipulation and querying of the interface elements of a dialog. While these control classes are powerful they are limited when it comes to getting data into and out of a dialog box. The problem is that the control classes are only proxies for the GUI elements. Hence, they are only useful for data transfer during the lifetime of the dialog box on the screen. In particular, the control classes do not buffer the interface element's data. The following code snippet illustrates the problem:

class TSearchDialog
  : public TDialog
{
public:
  TSearchDialog(TWindow* parent)
    : TDialog(parent, IDD_SEARCH_DIALOG),
    SearchStringEdit(this, IDC_SEARCH_STRING),
    SearchUpCheckBox(this, IDC_SEARCH_UP)
  {}

  TEdit SearchStringEdit;
  TCheckBox SearchUpCheckBox;
};

struct TSearchArguments
{
  tstring SearchString;
  uint Flags;
  enum TFlags {CaseSensitive = 1, WholeWord = 2, SearchUp = 4};
};

void TTransferDemo::CmSearch(TSearchArguments& a)
{
  TSearchDialog dlg(GetMainWindow());
  // ...too early to transfer data...
  if (dlg.Execute() == IDOK)
  {
    // ... too late to transfer data...
    Search(a);
  }
}

Although SearchStringEdit and SearchUpCheckBox are public, they are of no use to us for transferring data to and from the dialog. While the instances of the C++ encapsulation classes exist after construction of the TSearchDialog object, the underlying interface elements do not come into being until Execute is called. The interface elements stay alive until the dialog box is closed. Then they are dead and gone. Since the OWLNext encapsulation classes do not buffer the contents of the interface elements, it is now too late to get data out.

The brute-force solution is to further specialize the dialog class by overriding setup (SetupWindow) and shutdown (CmOk) and handle the data transfer explicitly. The transfer buffer mechanism was invented to save you from this chore.

---

The OWL transfer buffer mechanism and its flaws

The basic idea of transfer buffers is that every control class has a particular buffer field type.

Control type	Field type
TCheckBox	WORD
TCheckList	TCheckListData
TComboBox	TComboBoxData
TComboBoxEx	TComboBoxExData
TDateTimePicker	TDateTimePickerData
TEdit	owl::tchar []
THotKey	owl::uint16
TIPAddress	TIPAddressBits
TListBox	TListBoxData
TMemComboBox	owl::tchar []
TMonthCalendar	TMonthCalendarData
TRadioButton	WORD
TScrollBar	TScrollBarData
TSlider	TScrollBarData
TStatic	owl::tchar []

Given a pointer to such a field the control class instance can transfer data to or from the interface element it encapsulates. The dialog class then simply takes a pointer to a buffer and calls each control to transfer data to or from the buffer. The dialog performs this transfer in one direction at window setup and in the other direction just before the window closes after the OK button is pressed.

Usage

// Traditional OWL transfer buffer 
// NB! Byte packing is required!

#include <pshpack1.h>
struct TTransferBuffer
{
  enum {SearchStringSize = 255};
  char SearchString[SearchStringSize]; // TEdit data
  WORD SearchUp; // TCheckBox data
};
#include <poppack.h>

class TSearchDialog 
  : public TDialog
{
public:
  TSearchDialog(TWindow* parent, TTransferBuffer& b)
    : TDialog(parent, IDD_SEARCH_DIALOG)
  {
    new TEdit(this, IDC_SEARCH_STRING, TTransferBuffer::SearchStringSize);
    new TCheckBox(this, IDC_SEARCH_UP);
    SetTransferBuffer(&b);
  }
};

You can then use the transfer buffer with the dialog as follows. Notice how data is transferred into the transfer buffer before Execute is called and out of the transfer buffer afterwards. In particular, we have to be very careful when copying string buffers.

void TTransferDemo::CmSearch(TSearchArguments& a)
{
  TTransferBuffer b;

  const size_t n = TTransferBuffer::SearchStringSize - 1;
  b.SearchString[0] = '\0';
  strncat(b.SearchString, a.SearchString.c_str(), n); // tstring -> char[]
  WARN(n < a.SearchString.size(), "The search string was truncated!");

  b.SearchUp = (a.Flags & TSearchArguments::SearchUp) ? 
    BST_CHECKED : BST_UNCHECKED; // TFlags -> WORD

  TSearchDialog dlg(GetMainWindow(), b);
  if (dlg.Execute() == IDOK)
  {
    a.SearchString = b.SearchString; // char[] -> tstring
    a.Flags = (b.SearchUp == BST_CHECKED) ? 
      TSearchArguments::SearchUp : 0; // WORD -> TFlags
    Search(a);
  }
}

An aside about string buffers

After decades of C, a standard way to safely copy a null-terminated string is still controversial. Some use strncpy. But, strncpy will not null-terminate the result if the source string is too big. The reason is that it was designed to fill a fixed-length buffer [2]. For the same reason strncpy will unnecessarily pad the buffer with zeroes if the source is shorter than the size of the buffer. An alternative is to use strncat after initializing the buffer with an empty string, as we do here. The strncat function will always null-terminate the result, and it does not do any padding. Just note that we pass the maximum number of characters to copy, excluding the null-terminator, and not the size of the buffer.

Proper handling of string buffer truncation is awkward. In most cases truncation means that the application is in an invalid state. Here we just produce a diagnostic warning, but the proper way to handle truncation may be to alert the user, halt the operation, and/or terminate execution, all depending on the circumstance. Truncation is rarely insignificant.

Design flaws

The transfer buffer mechanism imposes the type of the data source on the client. The transfer buffer that serves as the data source is defined in terms of types and layout dictated by the GUI. In a well designed application, this requires a further transfer between the buffer and application data to separate the GUI front-end from the application back-end (logic). We saw that above; the search arguments; defined by the application; had to be transferred into the dialog transfer buffer; defined by the GUI; and back out again after the dialog was committed. This transfer between application data and transfer buffer is error-prone. Text fields (char arrays) need particular attention to avoid buffer overruns.

To avoid the extra transfer, a novice programmer may be tempted to use transfer buffers as part of the application logic, thus coupling the application logic with the GUI, leading to poor application design. For example, a change from a check box to a drop-down list in a dialog box would affect a change in the application data structures. This is bad. Never use a transfer buffer to implement application logic.

As this flaw is inherent in the transfer buffers design, there is no way to fix this problem other than using a better dialog data transfer mechanism. OWLNext 6.32 introduces such a mechanism, Dialog Data Transfer [3], and new code should use that mechanism in preference to transfer buffers. But, OWLNext is mainly used to maintain old OWL applications, so let us look closer at the implementation of transfer buffers and see how we can improve the safety of existing code.

Implementation flaws

The flaws and pit-falls in the implementation and usage of traditional OWL transfer buffers are grave and numerous. This stems from very low-level usage of void pointers and unchecked assumptions by the implementation about memory layout.

No type checking

The implementation assumes that a field of the correct type has been reserved in the transfer buffer for the corresponding control. If there is a type mismatch the buffer will be corrupted by the transfer.

Associations between field and controls are implicit

The associations between fields and controls are not enforced. The implementation relies on the assumption that the order of the child control list; which depends on creation order; matches the order of the corresponding fields in the buffer. It also assumes that any control that should not participate in data transfer, i.e. has no reserved field in the buffer, has had its data transfer disabled by a call to DisableTransfer.

The buffer is assumed to have standard layout

The assumption that field order matches child order implies that the buffer type must be a standard layout type [4]. The C++ standard has strict rules about standard layout. If a type does not satisify all these rules, the compiler is free to rearrange fields and add hidden fields. While you can get away with breaking these rules with a particular compiler, it is bad practice to rely on non-standard behavior.

The buffer is assumed to be byte-packed

The mechanism assumes that the size of one field determines the offset to the next, i.e. that there are no gaps between fields in the buffer. This is not guaranteed by the C++ standard; not even when the buffer has standard layout. In fact, it is unlikely to be the case, since modern compilers use the preferred alignment of data for modern CPUs. This means that you must ensure that transfer buffers are byte-packed by using compiler pragma statements around the buffer definition.

Transfers are prone to buffer underflow and overflow

The mechanism trusts the child control to return the correct size of its data. Buffer underflows and overflows are possible, even likely, to happen. In particular, this can easily happen with text fields. If there is a mismatch between the size of the text field in the buffer and the setting of the text limit of the corresponding edit control, the transfer will underflow or overflow the field. This then leads to errors in the offset calculation for the following fields. The result is a corrupted buffer.

Validator meddling

There is another complication in the transfer buffer mechanism specific to edit controls. If TEdit is assigned a validator (TValidator derivative), and that validator has the "voTransfer" option activated, then TEdit::Transfer will delegate the transfer job to the validator. This means that calls to TEdit::SetValidator and TValidator::SetOption (voTransfer) may change the expected type of the corresponding field in the transfer buffer.

TRangeValidator is currently the only validator in OWLNext that uses this feature. When activated, TRangeValidator::Transfer will try to transfer a long int to and from the buffer. This assumes that you provide a buffer that has a long int field for that particular edit control. Otherwise, the transfer corrupts the buffer.

---

Making transfer buffers safe

OWLNext 6.32 and later versions incorporate a buffer size check that will detect any potential transfer buffer underflow or overflow. If a size mismatch is detected, an exception is thrown. This ensures that the program does not continue in a corrupt state. In most cases, the check requires no change to user code, but it can only detect a limited number of problems. For example, it will not detect errors in the ordering of fields.

You can make your transfer buffers fully type-safe by using the TTransferBufferWindow template, a new mix-in class template that enforces a mapping between controls and transfer buffer fields of the correct type. TTransferBufferWindow has the following features:

• Checks that all participating controls are bound to a transfer buffer field.
• Checks that controls are not bound to the same field.
• Allows any order and alignment of fields.
• Provides customization hooks for custom controls, e.g. application-specific controls.
• Forbids validator meddling.
• Has support for string class fields to replace error-prone character arrays.

Usage

You have to make some minimal changes to your transfer buffer dialogs. First, your dialog class must inherit from TTransferBufferWindow instantiated with your buffer type. Second, you have to bind each control that participates in data transfer to a field in your buffer. For example,

// Safe transfer buffer
// Any alignment and packing is supported.

struct TTransferBuffer
{
  char SearchString[255]; // TEdit data
  WORD SearchUp; // TCheckBox data
};

class TSearchDialog
  : public TDialog,
  virtual public TTransferBufferWindow<TTransferBuffer>
{
public:
  TSearchDialog(TWindow* parent, TTransferBuffer& b)
    : TDialog(parent, IDD_SEARCH_DIALOG)
  {
    Bind<TEdit>(&TTransferBuffer::SearchString, IDC_SEARCH_STRING);
    Bind<TCheckBox>(&TTransferBuffer::SearchUp, IDC_SEARCH_UP);
    SetTransferBuffer(&b); // type-safe overload
  }
};

That is it! Your transfer buffer dialog is now fully type-safe.

Note that we didn't have to specify the packing of the transfer buffer. Since each control is bound directly to its corresponding field in the buffer, the order and alignment of fields no longer matter.

Also note that we no longer need to worry about the size of the TTransferBuffer::SearchString array. The size is automatically inferred by the Bind function. It ensures that the text limit of the control class is set to the size of the field.

If you forget to bind a child control, or you forget to disable transfer for an intentionally unbound control, then TTransferBufferWindow will throw an exception at run-time. Unless handled, it will cause a helpful error message that sounds something like this:

ObjectWindows Exception: Transfer requested by unbound control #1002 (class owl::TCheckBox) in window class TSearchDialog. Use TTransferBufferWindow::Bind to bind the control to a field in the window's transfer buffer, or disable transfer for this control using TWindow::DisableTransfer.

This should be enough information to track down the dialog and control that caused the problem. You will get similar messages if a control tries to bind to more than one field, if more than one control tries to bind to the same field, and if a validator tries to meddle in the transfer procedure.

Safe string handling

The usage of the dialog is exactly as before. So although the transfer between the dialog and the buffer is now guaranteed to be safe, any transfer in and out of the buffer has exactly the same issues as before. Therefore, you should consider replacing all character array fields in your transfer buffers by string class fields. When you bind a control to a string field, the bind function automatically figures out the correct transfer implementation to use and configures the control accordingly.

// Safe transfer buffer
// Note that we use a string class instead of an array.

struct TTransferBuffer
{
  tstring SearchString; // TEdit data
  WORD SearchUp; // TCheckBox data
};

class TSearchDialog
  : public TDialog,
  virtual public TTransferBufferWindow<TTransferBuffer>
{
public:

  // TTransferBuffer is a typedef provided by the base class.

  TSearchDialog(TWindow* parent, TTransferBuffer& b)
    : TDialog(parent, IDD_SEARCH_DIALOG)
  {
    Bind<TEdit>(&TTransferBuffer::SearchString, IDC_SEARCH_STRING); // string-aware
    Bind<TCheckBox>(&TTransferBuffer::SearchUp, IDC_SEARCH_UP);
    SetTransferBuffer(&b); // type-safe overload
  }
};

void TTransferDemo::CmSearch(TSearchArguments& a)
{
  TSearchDialog::TTransferBuffer b;
  b.SearchString = a.SearchString; // tstring -> tstring, nice!
  b.SearchUp = (a.Flags & TSearchArguments::SearchUp) ?
    BST_CHECKED : BST_UNCHECKED; // TFlags -> WORD

  TSearchDialog dlg(GetMainWindow(), b);
  if (dlg.Execute() == IDOK)
  {
    a.SearchString = b.SearchString;
    a.Flags = (b.SearchUp == BST_CHECKED) ? TSearchArguments::SearchUp : 0;
    Search(a);
  }
}

Now we are freed from the chore and danger of manual character array handling. For the same reason, you should consider using string class fields for TStatic and TMemComboBox. These are the only other control classes in OWLNext that use string fields.

Alternative ways to bind

In the examples above we have created the controls anonymously. What if you want to bind a control that has already been created in a member variable? Well, you just pass the identifier to Bind instead of the constructor arguments:

TStringEdit SearchStringEdit;
TCheckBox SearchUpCheckBox;

TSearchDialog(TWindow* parent, TTransferBuffer& b)
  : TDialog(parent, IDD_SEARCH_DIALOG),
  SearchStringEdit(this, IDC_SEARCH_STRING),
  SearchUpCheckBox(this, IDC_SEARCH_UP)
{
  Bind(&TTransferBuffer::SearchString, SearchStringEdit);
  Bind(&TTransferBuffer::Up, SearchUpCheckBox);
  SetTransferBuffer(&b); // type-safe overload
}

Note that you do not have to specify a template argument to Bind here; it is inferred from the type of the control argument.

If your dialogs are using pointers for the child controls, which is the common OWL idiom, you can take the address of the return value of the constructor version of Bind, or you can Bind the control after assignment of the pointer. Here is an example of both for the controls above:

TStringEdit* SearchStringEdit;
TCheckBox* SearchUpCheckBox;

TSearchDialog(TWindow* parent, TTransferBuffer& b)
  : TDialog(parent, IDD_SEARCH_DIALOG),
  SearchStringEdit(&Bind<TStringEdit>(&TTransferBuffer::SearchString, IDC_SEARCH_STRING))
{
  SearchUpCheckBox = new TCheckBox(this, IDC_SEARCH_UP);
  Bind(&TTransferBuffer::Up, *SearchUpCheckBox);
  SetTransferBuffer(&b); // type-safe overload
}

Specializing field binding for custom control types

To add support for your custom control classes in your transfer buffers, you must add a specialization of TTransferBufferBinder for each control type. To do that you add the following after your control class, or in some header that you include before your dialog class:

namespace owl {

//
// Binder for TMyControl; restricts the field type to TMyControlData.
//
template <>
class TTransferBufferBinder<TMyControl>
{
public:

  template <class TBuffer>
  static void Restrict(TMyControlData TBuffer::* field, TMyControl& c)
  {
    /*... Add checks and enforcements here, if any ...*/
  }

  template <class TBuffer>
  static TMyControl& Create(TMyControlData TBuffer::* field, 
    TWindow* p, int id, /*... constructor args ...*/)
  {
    return *new TMyControl(p, id, /*... constructor args ...*/);
  }

};

} // OWL namespace

Note that you will not use Restrict or Create directly. They are called by TTransferBufferWindow as needed. The essential effect of the code is that the first argument of Restrict and Create is restricted to the correct field type for your control type. If you need to do extra checks and enforcements, e.g. tell the control how much data to transfer, then you can do so in the function body of Restrict. For example, the specialization for TEdit restricts the text limit here.

If you do not have any extra checks and enforcements for your control type, then you can simplify the code above by inheriting a default implementation instead. The base class TTransferBufferBinderImplementation defines Restrict and Create as above for you.

namespace owl {

// 
// Binder for TMyControl; restricts the field type to TMyControlData.
//
template <>
class TTransferBufferBinder<TMyControl>
  : public TTransferBufferBinderImplementation<TMyControl, TMyControlData>
{};

} // OWL namespace

That is all you need to do in most cases!

Rules and limitations

While the TTransferBufferWindow template handles most of the safety issues for you, there are still a couple of issues you need to pay attention to:

• You need to specialize the TTransferBufferBinder template for your custom control types, otherwise Bind will refuse your controls.
• It is your responsibility to uphold type-safety in the Transfer functions of your custom control classes.
• Ideally, safe transfer buffers should have standard layout; as is the requirement for the old transfer buffers as well. Otherwise the C++ standard does not specify the behaviour of offsetof [5], an important component of the implementation. While the use of classes, such as tstring and TListBoxData, as field types is not allowed for standard layout types, in practice it works fine with all the supported compilers.

---

Conclusion

The OWL transfer buffer mechanism is inherently brittle and unsafe. OWLNext can now detect most usage mistakes by the buffer size check it performs. In most cases no changes to your legacy code is required to benefit from this check. With a little more effort you can make your transfer buffers fully type-safe by using the new TTransferBufferWindow mix-in. But for new code you should look at better alternatives, such as the new Dialog Data Transfer framework.

---

Example code

There are two examples in OWLNext accompanying this article:

examples/classes/dialogdatatransfer.cpp

This example demonstrates alternative mechanisms and programming styles for transferring data to and from controls, using a simple search dialog as a case study. This is a complete example based on the code snippets used in this article.

examples/classes/transferbuffer.cpp

This project is the test case for the Safe Transfer Buffers feature. It tests the transfer mechanism with all of the OWLNext control classes in an ordinary dialog as well as in a property sheet.

Ready-made projects for C++Builder and Visual Studio are provided for both examples.

---

References

1. "Core OWL 5.0", Ted Neward, Manning Publications Co.
2. "strncpy", Wikipedia.
3. "Dialog Data Transfer", OWLNext Wiki.
4. "C++ concepts: StandardLayoutType", cppreference.com.
5. "offsetof", cppreference.com.