I'm currently trying to port a UNIX library to Windows that is designed to launch processes and forward their standard streams and I'm looking for a kind soul to help me out with a couple of problems. I'm hoping you don't mind me asking a few questions here. I figure that there's a good chance you'd know something about this. I haven't had much luck elsewhere :(
There are two things I'm having trouble with.
1. Detecting if a child process has crashed.
I can actually do this using WaitForDebugEvent(), but this adds significant overhead to process creation and slows down the process a little in general. I'd like to find a more light-weight solution.
I've tried using DLL injection (which you know a fair bit about, judging by the source code!) to open and acquire a mutex created in the parent. If a program bails without unlocking the mutex, one should see WAIT_ABANONDED returned by WaitForSingleObject in the parent. However, I can't seem to distinguish between a clean shut-down and a crash in my DLL. It seems DllMain() is always called in either case and so are the destructors of static C++ objects.
At the moment, I have resorted to looking for process return codes greater than 0xC0000000 but of course a process could legitimately return such a large value for reasons other than a crash, though this is somewhat unlikely.
Do you have any tricks that might help here?
2. The other thing I'm interested in is unbuffering the child's stdout and stderr streams. I connect them to named pipes and so the buffering is different to that when they simply write to a console. This can cause trouble with Q&A style apps that don't flush their stdout before waiting for input.
There's a similar problem on UNIX, but there I can use pseudo-terminals. Windows has no similar concept as far as I can tell.
One thing I've tried was to again use DLL injection to make calls to setvbuf() in an attempt to unbuffer stdout and stderr. However this seemed to have no effect at all. If I manually add such calls to a child program of my own creation, it works. But for some reason making these calls via DLL injection does not have the desired effect. Perhaps the buffers are thread-local in some way?
In my CreateProcess() call, I pass CREATE_SUSPENDED, do the injection and then call ResumeThread() on the child's main thread. It may be the case that the buffering for the standard streams will be set after my injected code is executed, thereby undoing my unbuffering.
If you have any ideas in this area, please throw them my way!
Console2 is a very nifty app, by the way -- keep it up! :)
Edd
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> At the moment, I have resorted to looking for process return codes
> greater than 0xC0000000 but of course a process could legitimately
> return such a large value for reasons other than a crash, though
> this is somewhat unlikely.
In fact, the valid exit code must be less than 256 (if not 128, sorry could not find the doc at the moment). It's documented somewhere in MSDN. Also, an indication that it may be true is implied by the existence of STILL_ACTIVE (259) which is an exit code of still running process. Unfortunately, I don't remember the doc about the exit code of a crashed process, but the constant 0xc0000000 rings a bell.
The bottom line: I believe a test of exitCode & 0xC0000000 may actually be good enough [for well written apps].
> 2. The other thing I'm interested in is unbuffering
> the child's stdout and stderr streams.
Unfortunately, I doubt that Marko can answer this one as well, because Console does not really redirect input/output. But... ConsoleHandler::MonitorThread from ConsoleHooke\ConsoleHandler.cpp has some code that creates CONIN$ and CONOUT$ files and triggers an update if CONOUT$ is signalled. So, you might be able to do something along the same lines.
On the other hand, my non-Console experience shows that some Unix utilities are pretty good at figuring out whether stdin/stdout are console in/out or not. That fact somewhat indicates that you might be able to get better results if you unbuffer *your* streams (the ones, you supply as child process' in/out). Have you tried that?
As you can see, I did not actually answer your questions, but hopefully you have some more food for thought.
--
Kirill.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ad 1: when you CreateProcess, you get process and main thread handles.
These handles behave like event handles and can be waited on using WaitForSingleObject/WaitForMultipleObjects and similar functions. When a process/thread exits, its handle is set to a signaled state. This works when process exits normally, or is killed (terminated). I guess it should work for crashing processes as well and maybe after you get that signal, you can try determining if the process in question has crashed...
ad 2: I don't think I can be of much help here, but MSDN states that:
"The setvbuf function allows the program to control both buffering and buffer size for stream. stream must refer to an open file that has not undergone an I/O operation since it was opened."
I'd try the thing Kirill suggested (setvbuf on your side), or trying to read or write something from your injected DLL after calling setvbuf (suspending the main thread might help preventing the application calling setvbuf or reading/writing to streams before your DLL had the chance of doing its thing)
Cheers,
Marko
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
First of all, thanks very much for taking the time to provide some answers. It's really appreciated.
Marko: I should have been more clear about #1. I am waiting on the handle of the child process to detect when it has finished, but I cannot distinguish between the case where it has exited normally or exited due to a crash (without wrapping it in a basic custom debugger).
"and maybe after you get that signal, you can try determining if the process in question has crashed..."
It's exactly this part that I'm having trouble with :) I will stick to checking the exit code of the process for now and perhaps make crash detection optional to accommodate strange applications with huge return values.
As for #2, I hadn't thought of trying to change the buffering of the child handles in the parent process. However, I don't know how I might do this. When creating named pipes, you can pass buffer sizes to the CreateNamedPipe function, but they only act as hints. Also, I don't see anyway of getting a FILE* from a HANDLE in order to call setvbuf, but I'm not sure that would help anyway; the buffer you provide would surely have to reside in the child's address space? Have I misunderstood your suggestion?
I will look further in to the CONOUT$ thing, but am I right in thinking that a HANDLE for this will only be signaled when something is written to a console? This wouldn't happen in my library as stdout and stderr write to pipes. Often stdout and stderr write to different pipes, too. There's no CONERR$ to help distinguish between data written to the different streams, is there?
"or trying to read or write something from your injected DLL after calling setvbuf (suspending the main thread might help preventing the application calling setvbuf or reading/writing to streams before your DLL had the chance of doing its thing) "
This didn't work either, unfortunately :(
Again, I'm very grateful for your help.
Edd
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I had to do this once, and deal with the buffering situation. I remember that instead of a pipe, I wrote to a file instead, and that way you are not limited by the 'suggested buffer size' of the pipe. For performance, I specified that the file is created as temporary which means it is not actually written to disk unless absolutely necessary (FILE_ATTRIBUTE_TEMPORARY).
That's a very interesting idea! May I ask a couple of questions?
1. Am I correct in thinking that I use the handle generated by CreateFile for both reading in the parent and writing in the child i.e. I don't need to do any funny business with DuplicateHandle or similar?
2. I'll certainly try your method, but I'm curious about the circumstances in which actual disk use will occur. If a child program outputs a lot of data, will a write occur regardless of the speed at which I process child output in the parent? For example, if I child spews out data for days on end, perhaps a gigabyte's worth, will I end up with a ~1Gb file on disk?
3. I'm not actually forwarding data to the child's stdin yet. Have you done this and did you use temporary files for this too?
Thanks,
Edd
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1. Yes, you can use the file handle directly without any of the DuplicateHandle calls.
2. I've ran it for several days, but multiple proceses. I've had some that have put out a lot, but not gigs of information. Hereis what MSDN says:
Specifying the FILE_ATTRIBUTE_TEMPORARY attribute causes file systems to avoid writing data back to mass storage if sufficient cache memory is available, because an application deletes a temporary file after a handle is closed. In that case, the system can entirely avoid writing the data. Although it doesn't directly control data caching in the same way as the previously mentioned flags, the FILE_ATTRIBUTE_TEMPORARY attribute does tell the system to hold as much as possible in the system cache without writing and therefore may be of concern for certain applications.
Files
3. I haven't done anything with STDIN yet, sorry, but I assume that as long as you open it up for shared read and write that it will work the same.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for your answers. I was hoping you knew something beyond what MSDN says :)
I guess it's deliberately vague to allow for changes in future.
I've started playing around a bit, but haven't been able to get the temporary file mechanism working yet. I don't suppose you have some example code floating around anywhere? My googling failed in this area.
Looking at the flags you pass to CreateFile, it appears you aren't using overlapped I/O, is that right?
Thanks,
Edd
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am not doing anything outside of the CreateFile call and passing that handle to the STARTUPINFO hStdOut and hStdErr. The &sa that I pass in is a SECURITY_ATTRIBUTES structure that I set as follows:
sa.nLength = sizeof( SECURITY_ATTRIBUTES );
sa.lpSecurityDescriptor = NULL;
sa.bInheritHandle = TRUE;
Thanks
Raymond
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi there,
I'm currently trying to port a UNIX library to Windows that is designed to launch processes and forward their standard streams and I'm looking for a kind soul to help me out with a couple of problems. I'm hoping you don't mind me asking a few questions here. I figure that there's a good chance you'd know something about this. I haven't had much luck elsewhere :(
There are two things I'm having trouble with.
1. Detecting if a child process has crashed.
I can actually do this using WaitForDebugEvent(), but this adds significant overhead to process creation and slows down the process a little in general. I'd like to find a more light-weight solution.
I've tried using DLL injection (which you know a fair bit about, judging by the source code!) to open and acquire a mutex created in the parent. If a program bails without unlocking the mutex, one should see WAIT_ABANONDED returned by WaitForSingleObject in the parent. However, I can't seem to distinguish between a clean shut-down and a crash in my DLL. It seems DllMain() is always called in either case and so are the destructors of static C++ objects.
At the moment, I have resorted to looking for process return codes greater than 0xC0000000 but of course a process could legitimately return such a large value for reasons other than a crash, though this is somewhat unlikely.
Do you have any tricks that might help here?
2. The other thing I'm interested in is unbuffering the child's stdout and stderr streams. I connect them to named pipes and so the buffering is different to that when they simply write to a console. This can cause trouble with Q&A style apps that don't flush their stdout before waiting for input.
There's a similar problem on UNIX, but there I can use pseudo-terminals. Windows has no similar concept as far as I can tell.
One thing I've tried was to again use DLL injection to make calls to setvbuf() in an attempt to unbuffer stdout and stderr. However this seemed to have no effect at all. If I manually add such calls to a child program of my own creation, it works. But for some reason making these calls via DLL injection does not have the desired effect. Perhaps the buffers are thread-local in some way?
In my CreateProcess() call, I pass CREATE_SUSPENDED, do the injection and then call ResumeThread() on the child's main thread. It may be the case that the buffering for the standard streams will be set after my injected code is executed, thereby undoing my unbuffering.
If you have any ideas in this area, please throw them my way!
Console2 is a very nifty app, by the way -- keep it up! :)
Edd
Edd,
> 1. Detecting if a child process has crashed.
> At the moment, I have resorted to looking for process return codes
> greater than 0xC0000000 but of course a process could legitimately
> return such a large value for reasons other than a crash, though
> this is somewhat unlikely.
In fact, the valid exit code must be less than 256 (if not 128, sorry could not find the doc at the moment). It's documented somewhere in MSDN. Also, an indication that it may be true is implied by the existence of STILL_ACTIVE (259) which is an exit code of still running process. Unfortunately, I don't remember the doc about the exit code of a crashed process, but the constant 0xc0000000 rings a bell.
The bottom line: I believe a test of exitCode & 0xC0000000 may actually be good enough [for well written apps].
> 2. The other thing I'm interested in is unbuffering
> the child's stdout and stderr streams.
Unfortunately, I doubt that Marko can answer this one as well, because Console does not really redirect input/output. But... ConsoleHandler::MonitorThread from ConsoleHooke\ConsoleHandler.cpp has some code that creates CONIN$ and CONOUT$ files and triggers an update if CONOUT$ is signalled. So, you might be able to do something along the same lines.
On the other hand, my non-Console experience shows that some Unix utilities are pretty good at figuring out whether stdin/stdout are console in/out or not. That fact somewhat indicates that you might be able to get better results if you unbuffer *your* streams (the ones, you supply as child process' in/out). Have you tried that?
As you can see, I did not actually answer your questions, but hopefully you have some more food for thought.
--
Kirill.
Hi,
ad 1: when you CreateProcess, you get process and main thread handles.
These handles behave like event handles and can be waited on using WaitForSingleObject/WaitForMultipleObjects and similar functions. When a process/thread exits, its handle is set to a signaled state. This works when process exits normally, or is killed (terminated). I guess it should work for crashing processes as well and maybe after you get that signal, you can try determining if the process in question has crashed...
ad 2: I don't think I can be of much help here, but MSDN states that:
"The setvbuf function allows the program to control both buffering and buffer size for stream. stream must refer to an open file that has not undergone an I/O operation since it was opened."
I'd try the thing Kirill suggested (setvbuf on your side), or trying to read or write something from your injected DLL after calling setvbuf (suspending the main thread might help preventing the application calling setvbuf or reading/writing to streams before your DLL had the chance of doing its thing)
Cheers,
Marko
Guys,
First of all, thanks very much for taking the time to provide some answers. It's really appreciated.
Marko: I should have been more clear about #1. I am waiting on the handle of the child process to detect when it has finished, but I cannot distinguish between the case where it has exited normally or exited due to a crash (without wrapping it in a basic custom debugger).
"and maybe after you get that signal, you can try determining if the process in question has crashed..."
It's exactly this part that I'm having trouble with :) I will stick to checking the exit code of the process for now and perhaps make crash detection optional to accommodate strange applications with huge return values.
As for #2, I hadn't thought of trying to change the buffering of the child handles in the parent process. However, I don't know how I might do this. When creating named pipes, you can pass buffer sizes to the CreateNamedPipe function, but they only act as hints. Also, I don't see anyway of getting a FILE* from a HANDLE in order to call setvbuf, but I'm not sure that would help anyway; the buffer you provide would surely have to reside in the child's address space? Have I misunderstood your suggestion?
I will look further in to the CONOUT$ thing, but am I right in thinking that a HANDLE for this will only be signaled when something is written to a console? This wouldn't happen in my library as stdout and stderr write to pipes. Often stdout and stderr write to different pipes, too. There's no CONERR$ to help distinguish between data written to the different streams, is there?
"or trying to read or write something from your injected DLL after calling setvbuf (suspending the main thread might help preventing the application calling setvbuf or reading/writing to streams before your DLL had the chance of doing its thing) "
This didn't work either, unfortunately :(
Again, I'm very grateful for your help.
Edd
Hi Edd,
I had to do this once, and deal with the buffering situation. I remember that instead of a pipe, I wrote to a file instead, and that way you are not limited by the 'suggested buffer size' of the pipe. For performance, I specified that the file is created as temporary which means it is not actually written to disk unless absolutely necessary (FILE_ATTRIBUTE_TEMPORARY).
sout->handle = CreateFileA( sout->filename, GENERIC_WRITE | GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, &sa, CREATE_ALWAYS, FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE, NULL );
Thanks
Hi Raymond,
That's a very interesting idea! May I ask a couple of questions?
1. Am I correct in thinking that I use the handle generated by CreateFile for both reading in the parent and writing in the child i.e. I don't need to do any funny business with DuplicateHandle or similar?
2. I'll certainly try your method, but I'm curious about the circumstances in which actual disk use will occur. If a child program outputs a lot of data, will a write occur regardless of the speed at which I process child output in the parent? For example, if I child spews out data for days on end, perhaps a gigabyte's worth, will I end up with a ~1Gb file on disk?
3. I'm not actually forwarding data to the child's stdin yet. Have you done this and did you use temporary files for this too?
Thanks,
Edd
1. Yes, you can use the file handle directly without any of the DuplicateHandle calls.
2. I've ran it for several days, but multiple proceses. I've had some that have put out a lot, but not gigs of information. Hereis what MSDN says:
Specifying the FILE_ATTRIBUTE_TEMPORARY attribute causes file systems to avoid writing data back to mass storage if sufficient cache memory is available, because an application deletes a temporary file after a handle is closed. In that case, the system can entirely avoid writing the data. Although it doesn't directly control data caching in the same way as the previously mentioned flags, the FILE_ATTRIBUTE_TEMPORARY attribute does tell the system to hold as much as possible in the system cache without writing and therefore may be of concern for certain applications.
Files
3. I haven't done anything with STDIN yet, sorry, but I assume that as long as you open it up for shared read and write that it will work the same.
Hi Raymond,
Thanks for your answers. I was hoping you knew something beyond what MSDN says :)
I guess it's deliberately vague to allow for changes in future.
I've started playing around a bit, but haven't been able to get the temporary file mechanism working yet. I don't suppose you have some example code floating around anywhere? My googling failed in this area.
Looking at the flags you pass to CreateFile, it appears you aren't using overlapped I/O, is that right?
Thanks,
Edd
Edd,
I am not doing anything outside of the CreateFile call and passing that handle to the STARTUPINFO hStdOut and hStdErr. The &sa that I pass in is a SECURITY_ATTRIBUTES structure that I set as follows:
sa.nLength = sizeof( SECURITY_ATTRIBUTES );
sa.lpSecurityDescriptor = NULL;
sa.bInheritHandle = TRUE;
Thanks
Raymond
Ok, thanks Raymond. I guess I should do some smaller experiments first, before going the whole asynchronous I/O route!
Again, thanks to everyone for all their help!
Edd