#261 Hang when attaching stlport.dll to process on Windows

[Roger Sanders]

A bug exists in STLPort that I've encountered causing a hang in my application. In my case I'm not using STLPort at all, but a third party library that I'm loading into my process through a LoadLibrary call at runtime is, and this STLPort bug causes the call to LoadLibrary to hang, because STLPort hangs during its initialization routines.

The bug itself exists in the "src\details\fstream_win32io.cpp" file, in the "is_regular_file" function. This function calls the win32 "GetFileInformationByHandle" function, without knowing if the file handle it was provided actually refers to a file or to another kind of source. The GetFileInformationByHandle can ONLY be called safely for file handles. It's known to hang for pipe handles, and in the case of a "character file" handle, as it will be for a non-redirected console input handle for example, the function will hang while any other threads are waiting for input on the handle, which is what occurred in my application. The chain of calls is basically this:
-ios_base::_S_initialize()
-_Stl_create_filebuf(stdin, ios_base::in)
-_Filebuf_base::_M_open(_Stl_extract_open_param(x), mode)
-is_regular_file(_M_file_id)
-GetFileInformationByHandle(fd, &info)

The "__is_regular_file" function seems to be the only place in STLPort where the GetFileInformationByHandle function is called, and it only needs to return true if the file handle refers to a file on disk that's not a directory, and false otherwise. The easy fix here is to call GetFileType() to check if the handle actually refers to a file on disk first, and return false if it doesn't. This wouldn't alter the result of the existing function, just remove the undefined behaviour, and real-world hangs, that come from calling GetFileInformationByHandle for non-file handles.

I've attached some example code in the form of a "main.cpp" file, which demonstrates the locking problem in a standalone console application. This doesn't use STLPort at all, and was tested in Visual Studio 2010. This code will demonstrate the locking issue in action, and the suggested fix.

For a patch, the "src\details\fstream_win32io.cpp" should have the following current code:

static bool __is_regular_file(_STLP_fd fd) {
  BY_HANDLE_FILE_INFORMATION info;

  // Return true if the file handle isn't a directory.
  return GetFileInformationByHandle(fd, &info) && 
         ((info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0);
}

Changed to this:

static bool __is_regular_file(_STLP_fd fd) {
  BY_HANDLE_FILE_INFORMATION info;

  // Return true if the file handle is a basic file handle that isn't a directory.
  return (GetFileType(fd) == FILE_TYPE_DISK) && GetFileInformationByHandle(fd, &info) && 
         ((info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0);
}

[Petr Ovtchenkov]

Use something like 'git format-patch' and suggest good patch, if you want to see this changes in repo.

[Sergio]

http://www.privateyachtcharter-sxm.com/escpcj/eetleibqlrqgptmpcqxwgawm.nxacdgyfomwcjlqcon

sjordine@yahoo.com.br