From: SourceForge.net <no...@so...> - 2008-04-07 22:25:28
|
Feature Requests item #1343161, was opened at 2005-10-30 20:51 Message generated for change (Comment added) made by nobody You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=360894&aid=1343161&group_id=10894 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: 27. Channel Types Group: None Status: Closed Resolution: Works For Me Priority: 3 Private: No Submitted By: Peter Bray (illumino) Assigned to: Nobody/Anonymous (nobody) Summary: exec implemenation fails to close() open file descriptors Initial Comment: Tcl/Tk version : 8.4.6 (exists in 8.4.11) Platform: Sun SPARC Solaris 9 In the child section of TclpCreateProcess() in unix/tclUnixPipe.c, after the "pid = fork(); if ( pid ....)" code, the child process sets up std file descriptors, restores signal handling, and executes the new process. Unfortunately, the child leaves all existing file descriptors open! I believe there should be an option (perhaps the default) to close all open file descriptors before exec() so the children (and grandchildren,...) don't inherit them. If it is an option then "exec" will need an interface to this option, to allow the script writer to close all open file descriptors. Regards, Peter Bray Sydney, Australia PS: In our case a UI, which loads a shared library that connects to a server (TCP/IP), used "exec" to start a backgrounded (daemonised actually) copy of firefox to display the user help. LSOF shows that firefox inherited the socket connections and other file descriptors, and when the UI exited, the firefox process still had references to these file descriptors, meaning that the server did not detect a connection close. Trouble ensued and memory and cpu exited stage left... as the server try to buffer data for a client that was not reading from its socket connection. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2008-04-07 15:25 Message: Logged In: NO Sorry it was the end of a very very long day, and I was not at a system which this application was available, so I didn't test the shell redirection, and it just looked like bash syntax. Sorry again. Anyway I think we have done this to death so lets call an end to it. Many Thanks ---------------------------------------------------------------------- Comment By: Alexandre Ferrieux (ferrieux) Date: 2008-04-07 06:18 Message: Logged In: YES user_id=496139 Originator: NO It is your right to prefer an ad hoc C program that must be recompiled for all platforms over a shell idiom working on all unices ;-) I've just tried the "3>&-" on /bin/sh (which is different from /bin/bash) on a Solaris 5.8: it works. Can you explain what exactly it does on your system ? (use truss for that) ---------------------------------------------------------------------- Comment By: Peter Bray (illumino) Date: 2008-04-07 05:20 Message: Logged In: YES user_id=426773 Originator: YES As I previously stated, I ended up writing a C program called "closeAndExec" which does the right thing and provides a much easier to read TCL for our local TCL developer, also the solution provided looks like BASH not Solaris' /bin/sh (for future readers, and isn't pretty IMHO ;-) ). Bug marked "Works for me", but "won't fix" seems more appropriate. Many Thanks, Peter ---------------------------------------------------------------------- Comment By: Alexandre Ferrieux (ferrieux) Date: 2008-04-06 23:30 Message: Logged In: YES user_id=496139 Originator: NO OK I get it now. Why didn't you say earlier that your inherited descriptors were created inside an extension ? (Maybe you did, sorry for not combing the haystack ;-). Now, I agree this is at *most* a feature request, but I'll even try to requalify this as "out of Tcl's realm". Indeed, any other shell or environment able to (1) load that library and (2) spawn a child will have the very same problem. Moreover, I think it is relatively easy to work around that vendor bug, e.g. in Bourne shell, provided you have an idea on the range of file descriptors to close *and* they are smaller than 10: set f [open "|sh -c {exec 3>&- 4>&- ... 9>&-;exec mycommand myargs} 2>@ stderr" r+] Please tell me if you need additional help to make this work. Otherwise, do you agree we may rightfully close the entry as "works for me" ? ---------------------------------------------------------------------- Comment By: Alexandre Ferrieux (ferrieux) Date: 2008-04-06 23:30 Message: Logged In: YES user_id=496139 Originator: NO OK I get it now. Why didn't you say earlier that your inherited descriptors were created inside an extension ? (Maybe you did, sorry for not combing the haystack ;-). Now, I agree this is at *most* a feature request, but I'll even try to requalify this as "out of Tcl's realm". Indeed, any other shell or environment able to (1) load that library and (2) spawn a child will have the very same problem. Moreover, I think it is relatively easy to work around that vendor bug, e.g. in Bourne shell, provided you have an idea on the range of file descriptors to close *and* they are smaller than 10: set f [open "|sh -c {exec 3>&- 4>&- ... 9>&-;exec mycommand myargs} 2>@ stderr" r+] Please tell me if you need additional help to make this work. Otherwise, do you agree we may rightfully close the entry as "works for me" ? ---------------------------------------------------------------------- Comment By: Peter Bray (illumino) Date: 2008-04-06 21:05 Message: Logged In: YES user_id=426773 Originator: YES Background: A third-party software vendor supplied a TCL package including a shared library which created network connections. The intended TCL/TK application which they also provided, spawned mozilla & acroread processes which survived the execution of the application (occasionally, they were there as online help resources for the application). The problems start when the application goes away and the sub-programs are left holding open sockets, which they are of course, not servicing. The server does not see that the client has exited, the fds are still open, so it buffers data for them, thinking they'll get to it soon. Problems ensue... think memory "leaks" in the server, etc ... Now the TCL/TK application needs those network connections and so the close of all open file descriptors needs to be done after the fork() or at the time of exec(). Ideally they vendor would have used FD_CLOEXEC, but honestly they aren't that bright, and unresponsive would be a polite way to put it. So being perl person (I hope you'll forgive me for that) I thought, well I'll just close them all myself in the child before starting the these helper apps. I understood it was really a vendor issue, but the Perl programmer in me, thought their should be an option on exec to handle such situations given that a TCL programmer can not manually do a fork()/close(ALL)/exec() sequence of commands as would be possible in Perl or C. Sorry I have worded thing badly and went off on a tangent, I now consider that it really is a feature request, because as has been pointed out TCL/TK does handle all its FDs correctly. I'm personally happy if the TCL team, say "well, we should not be responsible for the bad C coding of others" and close the issue. It just seemed (at the time) like a useful feature to have. My thoughts are "Won't Fix" and be done with it, but I'll leave that to the development team, may they might think its useful. ---------------------------------------------------------------------- Comment By: Alexandre Ferrieux (ferrieux) Date: 2008-04-06 15:26 Message: Logged In: YES user_id=496139 Originator: NO I'm not asking to reclassify as a FR, I'm seeking an explanation. Of course the code for [exec] doesn't close the other fds, since they are all marked as close-on-exec. The idea is that they will of course survive the fork(), but not the subsequent execve(). So please provide detailed explanations (like a truss output) showing where these flags disappeared, and also which descriptors exactly got inherited (ie what kind of channels). ---------------------------------------------------------------------- Comment By: Peter Bray (illumino) Date: 2008-04-06 15:05 Message: Logged In: YES user_id=426773 Originator: YES OK, I'll agree to reclassification as a feature request, and I have lowered the priority to 3. We ended up solving the problem by writing a C program we called "closeAndExec" which does close all file descriptors and then exec its command line arguments. Regards, Peter ---------------------------------------------------------------------- Comment By: Alexandre Ferrieux (ferrieux) Date: 2008-04-06 08:54 Message: Logged In: YES user_id=496139 Originator: NO I don't understand the complaints here, since tclUnixChan.c and tclUnixPipe.c have been setting the close-on-exec flag of just about any kind of Tcl channel since the beginning of life on earth (1.1 rjohnson ;-), exactly for this (documented) reason. Could someone provide a modern case reproducing this ? With an strace output ? ---------------------------------------------------------------------- Comment By: Peter Bray (illumino) Date: 2005-11-01 03:00 Message: Logged In: YES user_id=426773 Changed Category (Commands 'e') and AssignedTo (None). Aside: I have to remeber that SourgeForge does not like "back" and "refresh" buttons :-( -- Sorry ---------------------------------------------------------------------- Comment By: Peter Bray (illumino) Date: 2005-11-01 02:53 Message: Logged In: YES user_id=426773 No, I'd call it a BUG or at least an UNDOCUMENTED FEATURE :-), as the language provides no way to access the relevant system calls to do the IMHO correct thing (not give away private file descriptors) to child processes. Without fork() & close() access, how do I stop the pollution of child processes? I don't see it as the child executables' responsibility to cleanup the mess created by the parent process. In C & Perl if I fork() a child process I have the opportunity to cleanup any open file descriptors before issuing and exec() system call, as I should I opened them. While I can see that additional code could be a FEATURE REQUEST, I feel that polluting child processes is a BUG - but each to there own opinion. ---------------------------------------------------------------------- Comment By: Vince Darley (vincentdarley) Date: 2005-10-31 15:27 Message: Logged In: YES user_id=32170 Not really my area of knowledge - it's more about processes than it is about the filesystem. ---------------------------------------------------------------------- Comment By: Peter Bray (illumino) Date: 2005-10-31 14:41 Message: Logged In: YES user_id=426773 No, I'd call it a BUG or at least an UNDOCUMENTED FEATURE :-), as the language provides no way to access the relevant system calls to do the IMHO correct thing (not give away private file descriptors) to child processes. Without fork() & close() access, how do I stop the pollution of child processes? I don't see it as the child executables' responsibility to cleanup the mess created by the parent process. In C & Perl if I fork() a child process I have the opportunity to cleanup any open file descriptors before issuing and exec() system call, as I should I opened them. While I can see that additional code could be a FEATURE REQUEST, I feel that polluting child processes is a BUG - but each to there own opinion. ---------------------------------------------------------------------- Comment By: Don Porter (dgp) Date: 2005-10-31 09:19 Message: Logged In: YES user_id=80530 this looks like a feature request rather than a bug report. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=360894&aid=1343161&group_id=10894 |