Thread: [maildropl] backticks: how to put process into background?
Brought to you by:
mrsam
From: Christian E. <bla...@gm...> - 2011-06-18 13:15:55
|
Hi, Is there a way to force a process inside backticks into background? `&` seems to have no effect. Example: I'm trying to migrate a filter to display (X-)Faces - used as display filter in mutt - from procmail to maildrop. Everything works except putting the viewer in the background: # customize X11 viewer (has to accept stdin) VIEWER="/sw/bin/xv -geometry 100x100 -wait 3 -" # adjust paths of required programs for speed AWK="/usr/bin/awk" REFORMAIL="/usr/local/bin/reformail" OPENSSL="/usr/bin/openssl" PRINTF="/usr/bin/printf" UNCOMPFACE="/sw/bin/uncompface" # check if $DISPLAY is non-empty if ("$DISPLAY") { if (/^Face:\s+(.{16,})/) { # work around broken headers # (re)create lines of width 79, removing all spaces # delete potentially appended X-Face (Claws-Mail) `$PRINTF '%s\n' $MATCH1 | $AWK '{ if ($1 == "X-Face:") exit; OFS = ""; $1 = $1; line = $0; do { print substr(line, 0, 79); line = substr(line, 80); } while (length(line)); }' | $OPENSSL base64 -d | $VIEWER &` xfilter "$REFORMAIL -I'Face:'" } elsif (/^X-Face:\s+(.{16,})/) { # pipe it through uncompface `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER &` # remove the header for mutt xfilter "$REFORMAIL -I'X-Face:'" } } to "| cat" Any ideas someone? c -- theatre - books - texts - movies Black Trash Productions at home: http://www.blacktrash.org Black Trash Productions on Facebook: http://www.facebook.com/blacktrashproductions |
From: Sam V. <mr...@co...> - 2011-06-18 14:11:52
|
Christian Ebert writes: > Hi, > > Is there a way to force a process inside backticks into > background? `&` seems to have no effect. Of course it does. There's no reason why it wouldn't work, as the entire command is passed to the shell for execution, without any interpretation by maildrop. However, maildrop creates a new process group, and the entire process group gets SIGHUPed when maildrop terminates, so that all processes that are involved in mail delivery get terminated, by default, when mail delivery is complete. So, your process most certainly goes into the background, but gets sighupped when maildrop terminates a little bit later. Try nohuping your entire command. |
From: Christian E. <bla...@gm...> - 2011-06-18 20:30:52
|
> Christian Ebert writes: >> Is there a way to force a process inside backticks into >> background? `&` seems to have no effect. > > Of course it does. There's no reason why it wouldn't work, as the > entire command is passed to the shell for execution, without any > interpretation by maildrop. > > However, maildrop creates a new process group, and the entire process > group gets SIGHUPed when maildrop terminates, so that all processes > that are involved in mail delivery get terminated, by default, when > mail delivery is complete. > > So, your process most certainly goes into the background, but gets > sighupped when maildrop terminates a little bit later. Try nohuping > your entire command. Good ... I just have not the slightest clue how to do that. Thanks all the same. c -- \black\trash movie _SAME TIME SAME PLACE_ New York, in the summer of 2001 --->> http://www.blacktrash.org/underdogma/stsp.php |
From: Christian E. <bla...@gm...> - 2011-06-20 09:53:20
|
* Sam Varshavchik on Saturday, June 18, 2011 at 10:11:45 -0400 > Christian Ebert writes: >> Is there a way to force a process inside backticks into >> background? `&` seems to have no effect. > > Of course it does. There's no reason why it wouldn't work, as the > entire command is passed to the shell for execution, without any > interpretation by maildrop. > > However, maildrop creates a new process group, and the entire process > group gets SIGHUPed when maildrop terminates, so that all processes > that are involved in mail delivery get terminated, by default, when > mail delivery is complete. > > So, your process most certainly goes into the background, but gets > sighupped when maildrop terminates a little bit later. Try nohuping > your entire command. Things I've tried: `printf '%s\n' $MATCH1 | uncompface -X | nohup xv -geometry 100x100 -wait 3 - &` `nohup (printf '%s\n' $MATCH1 | uncompface -X | xv -geometry 100x100 -wait 3 - &)` and other combinations: they just do nothing, i.e. no x-face is displayed. When using the variables: PRINTF="/usr/bin/printf" UNCOMPFACE="/sw/bin/uncompface" VIEWER="/sw/bin/xv -geometry 100x100 -wait 3 -" NOHUP="/usr/bin/nohup" `$NOHUP ($PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER &)` again does nothing, whereas: `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $NOHUP $VIEWER &` just exhibits the original behaviour. Omitting `&' does not change anything. c -- theatre - books - texts - movies Black Trash Productions at home: http://www.blacktrash.org Black Trash Productions on Facebook: http://www.facebook.com/blacktrashproductions |
From: Sam V. <mr...@co...> - 2011-06-20 11:07:41
|
Christian Ebert writes: > * Sam Varshavchik on Saturday, June 18, 2011 at 10:11:45 -0400 > > Christian Ebert writes: > >> Is there a way to force a process inside backticks into > >> background? `&` seems to have no effect. > > > > Of course it does. There's no reason why it wouldn't work, as the > > entire command is passed to the shell for execution, without any > > interpretation by maildrop. > > > > However, maildrop creates a new process group, and the entire process > > group gets SIGHUPed when maildrop terminates, so that all processes > > that are involved in mail delivery get terminated, by default, when > > mail delivery is complete. > > > > So, your process most certainly goes into the background, but gets > > sighupped when maildrop terminates a little bit later. Try nohuping > > your entire command. > > Things I've tried: > > `printf '%s\n' $MATCH1 | uncompface -X | nohup xv -geometry 100x100 -wait 3 > - &` > > `nohup (printf '%s\n' $MATCH1 | uncompface -X | xv -geometry 100x100 -wait 3 > - &)` > > and other combinations: they just do nothing, i.e. no x-face is > displayed. The argument to nohup is a command to execute. "(…)" is not a command, it is a shell pipeline construct. This is parsed and interpreted by the shell. [mrsam@octopus maildrop]$ which nohup /usr/bin/nohup nohup is a standalone command. Its first argument is the command to execute with SIGHUP disabled. The remaining arguments are passed through as the arguments to the command. nohup does not incorporate an entire shell interpreter and parser inside of itself. As such, shell constructs, such as (pipeline) do not work when you pass it to nohup. Your backtick command should probably be: `nohup /bin/sh -c "command &"` This results in maildrop executing nohup, which turns of sighup, and executes /bin/sh, passing it the arguments "-c" and "command &". This, of course, loads the shell, the shell interprets the command, forks off a child process to run it, then the shell terminates. Also, I can't help but to note that, of course, you must've made sure that whatever you've matched via MATCH1 you have validated as to not contain any special shell metacharacters. Otherwise, a hostile party might be able to easily exploit your security vulnerability, if they have some knowledge of what you're doing with your mail, and have your mail script execute arbitrary commands, just by sending you a specially-crafted email. Using this situation as a sample example, if, say, you were matching against the subject line, then I could see how, depending on what you're doing with it, someone were to send you a mail with something like "Subject: ; rm -rf $HOME", then, depending on how you were using $MATCH1, it's possible that you will instruct maildrop to execute nohup /bin/sh -c "command arg arg…; rm -rf $HOME &" Then, after executing the initial command you intended to be executed -- whether the command succeeds or fails the shell proceeds to execute the next command, which deletes your entire home directory. Of course, if you intended to give anyone the ability to remove your home directory via email, this is fine, but otherwise, you have a problem. And this possibility exists whether you use maildrop, procmail, or anything else, to loosely grab parts of an arbitrary email message, that may contain anything, and shove it inside a canned shell command, and have the end result executes under your own userid, without implementing strict validity checking to make sure that the fragment of an arbitrary email message you pulled out does not contain anything that can be misinterpreted by the shell's parser. |
From: Christian E. <bla...@gm...> - 2011-06-20 11:59:32
|
* Sam Varshavchik on Monday, June 20, 2011 at 07:07:33 -0400 Thank you for your detailed response. I am aware that this is more or less off-topic, so feel free to tell me to bugger off ;-) > The argument to nohup is a command to execute. "(…)" is not a command, > it is a shell pipeline construct. This is parsed and interpreted by > the shell. > > [mrsam@octopus maildrop]$ which nohup > /usr/bin/nohup > > nohup is a standalone command. Its first argument is the command to > execute with SIGHUP disabled. The remaining arguments are passed > through as the arguments to the command. > > nohup does not incorporate an entire shell interpreter and parser > inside of itself. As such, shell constructs, such as (pipeline) do not > work when you pass it to nohup. > > Your backtick command should probably be: > > `nohup /bin/sh -c "command &"` Yes, I forgot to mention that I also tried: `nohup /bin/sh -c '$PRINTF "%s\n" $MATCH1 | $UNCOMPFACE -X | $VIEWER &'` which behaves exactly like without nohup: `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER &` > This results in maildrop executing nohup, which turns of sighup, and > executes /bin/sh, passing it the arguments "-c" and "command &". This, > of course, loads the shell, the shell interprets the command, forks > off a child process to run it, then the shell terminates. [...] > nohup /bin/sh -c "command arg arg…; rm -rf $HOME &" I am not running this on any incoming mail, only from within mutt as a display_filter, so I think I'm reasonably safe because "echo 'rm -rf $HOME' | uncompface" would just result in a strange or invalid x-face. Thank you for reminding me though. If course I also do not run this on incoming mail. thx c -- \black\trash movie _SAME TIME SAME PLACE_ --->> http://www.blacktrash.org/underdogma/stsp.php \black\trash audio _ANOTHER TIME ANOTHER PLACE_ --->> http://www.blacktrash.org/underdogma/atap.html |
From: Eric d'H. <eri...@gm...> - 2011-06-21 05:37:04
|
On 6/20/11, Sam Varshavchik <mr...@co...> wrote: > And this possibility exists whether you use maildrop, procmail, or anything > else, to loosely grab parts of an arbitrary email message, [...snip] Thanks for the security excursus Sam. Very much appreciated. -- No no no, my fish's name is Eric, Eric the fish. He's an halibut. I am not a looney! Why should I be tarred with the epithet looney merely because I have a pet halibut? |
From: Sam V. <mr...@co...> - 2011-06-20 12:44:50
|
Christian Ebert writes: > > > > Your backtick command should probably be: > > > > `nohup /bin/sh -c "command &"` > > Yes, I forgot to mention that I also tried: > > `nohup /bin/sh -c '$PRINTF "%s\n" $MATCH1 | $UNCOMPFACE -X | $VIEWER &'` > > which behaves exactly like without nohup: > > `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER &` Probably a syntax error somewhere. Start with something simple: `nohup /bin/sh -c 'xterm &'` If, now, you get an xterm window, you can keep making the command longer until you get what you want. It also occured to me that in order to get anything up on the display, the DISPLAY environment variable must be set. maildrop resets all environment variables by default, in spawned commands. I presumed that you've imported the DISPLAY environment variable in your maildrop recipe, before you attempt to run any X client. Depending on your environment you may also need to import other variables, like XAUTHORITY. |
From: Christian E. <bla...@gm...> - 2011-06-20 13:14:51
|
* Sam Varshavchik on Monday, June 20, 2011 at 08:44:43 -0400 > Christian Ebert writes: >> Yes, I forgot to mention that I also tried: >> >> `nohup /bin/sh -c '$PRINTF "%s\n" $MATCH1 | $UNCOMPFACE -X | $VIEWER &'` >> >> which behaves exactly like without nohup: >> >> `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER &` > > Probably a syntax error somewhere. > > Start with something simple: > > `nohup /bin/sh -c 'xterm &'` Same behaviour as: `xterm &` and as: `xterm` i.e. the process is not going into the background, maildrop waits for it to be terminated. > If, now, you get an xterm window, you can keep making the command > longer until you get what you want. I do get the desired result with my script, i.e. the xface is displayed, but these all behave the same way: `nohup /bin/sh -c '$PRINTF "%s\n" $MATCH1 | $UNCOMPFACE -X | $VIEWER &'` `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER &` `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER` > It also occured to me that in order to get anything up on the display, > the DISPLAY environment variable must be set. The detection of "$DISPLAY" in my recipe works fine. All works fine - except the background thingy. c -- theatre - books - texts - movies Black Trash Productions at home: http://www.blacktrash.org Black Trash Productions on Facebook: http://www.facebook.com/blacktrashproductions |
From: Sam V. <mr...@co...> - 2011-06-20 13:44:12
|
Christian Ebert writes: > `nohup /bin/sh -c '$PRINTF "%s\n" $MATCH1 | $UNCOMPFACE -X | $VIEWER &'` > `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER &` > `$PRINTF '%s\n' $MATCH1 | $UNCOMPFACE -X | $VIEWER` > > > It also occured to me that in order to get anything up on the display, > > the DISPLAY environment variable must be set. > > The detection of "$DISPLAY" in my recipe works fine. > > All works fine - except the background thingy. strace explains what's happening. maildrop's backtick operator captures the output of the command, and all processes spawned by the backtick operator write to the same pipe on standard output, so maildrop waits for the pipe to close, which won't happen until all processes terminate. Adding >/dev/null to the end fixes it. import DISPLAY `nohup /bin/sh -c "xterm &" >/dev/null` |
From: Christian E. <bla...@gm...> - 2011-06-20 15:18:03
|
* Sam Varshavchik on Monday, June 20, 2011 at 09:44:05 -0400 > strace explains what's happening. grrh, Mac OS X doesn't provide it. > maildrop's backtick operator captures the output of the > command, and all processes spawned by the backtick operator > write to the same pipe on standard output, so maildrop waits > for the pipe to close, which won't happen until all processes > terminate. > > Adding >/dev/null to the end fixes it. > > import DISPLAY > > `nohup /bin/sh -c "xterm &" >/dev/null` Yey! perfect, thank you. After some shell quote tweaking, my little recipe looks like this: # customize X11 viewer (has to accept stdin) VIEWER="/sw/bin/xv -geometry 100x100 -wait 3 -" # adjust paths of required programs for speed SHELL="/bin/sh" CAT="/bin/cat" AWK="/usr/bin/awk" OPENSSL="/usr/bin/openssl" NOHUP="/usr/bin/nohup" PRINTF="/usr/bin/printf" REFORMAIL="/usr/local/bin/reformail" UNCOMPFACE="/sw/bin/uncompface" # check if $DISPLAY is non-empty if ("$DISPLAY") { if (/^Face:\s+(.{16,})/) { # work around broken headers # (re)create lines of width 79, removing all spaces # delete potentially appended X-Face (Claws-Mail) `$NOHUP $SHELL -c '$PRINTF "%s\n" $MATCH1 | $AWK "{ if (\$1 == \"X-Face:\") exit; OFS = \"\"; \$1 = \$1; line = \$0; do { print substr(line, 0, 79); line = substr(line, 80); } while (length(line)); }" | $OPENSSL base64 -d | $VIEWER &' >/dev/null` } elsif (/^X-Face:\s+(.{16,})/) { # pipe it through uncompface `$NOHUP $SHELL -c '$PRINTF "%s\n" $MATCH1 | $UNCOMPFACE -X | $VIEWER &' >/dev/null` } # remove the header for mutt xfilter "$REFORMAIL -I'Face:' -I'X-Face:'" } to "| $CAT" Can be used in mutt as: set display_filter="maildrop ~/.mailfilters/x-face-filter" Thanks a lot for your help and patience. c -- \black\trash movie _SAME TIME SAME PLACE_ New York, in the summer of 2001 --->> http://www.blacktrash.org/underdogma/stsp.php |