|
From: David F. <Da...@me...> - 2004-12-01 00:10:24
|
Hi Ivan & everyone, Re: Chroot, piped logging & failure to restart child process I think I have found a fix for this problem, which can be applied to any piped logging process in the chroot, including rotatelogs. It makes use of the Monit server monitoring package (www.tildeslash.com/monit) which can take action from outside the chroot when the piped logging process dies. This solution isn't as nice as getting everything to 'just work', but I think it is the best I can do for piped logging from the chroot. Monit can easily restart apache, therefore restarting any failed piped logging process. However, Monit has to know that something is wrong, and it is nice if it can find this out *before* the web server begins to lock up. When apache tries to re-start the piped logging process it looks for the binary it was previously running, but on the inside of the chroot. Actually putting a copy of the binary there is a nuisance, and in any case the log files would start going inside the chroot rather than the normal place, which is messy. Much better is to put a simple statically linked binary as a 'fake' rotatelogs on the inside of the chroot. When apache runs this fake rotatelogs, this binary updates a status file (e.g. /tmp/log_fail.txt), which Monit notices by a timestamp change, and the server is automatically re-started. Code for the fake fakerotatelogs.c: ----------------------------------------------------------- #include <stdio.h> #include <stdlib.h> //Fake rotatelogs: compile with //gcc -static -o rotatelogs fakerotatelogs.c int main(int argc, char *argv[]) { FILE * out; time_t timer; timer=time(NULL); /*Open logging state file and write time of failure*/ out = fopen("/tmp/log_fail.txt", "w"); fprintf(out, "Logging failed on %s", asctime(localtime(&timer))); fclose(out); return 0; } ----------------------------------------------------------- Compile it and place the output inside the chroot as (in my case) /chroot/apache/usr/local/apache/bin/rotatelogs Add to the /etc/monitrc file an additional entry for watching the timestamp on the state file at /chroot/apache/tmp/log_fail.txt. /etc/monitrc entry ----------------------------------------------------------- check file logging_indicator with path /chroot/apache/tmp/log_fail.txt group www start program = "/usr/bin/touch /chroot/apache/tmp/log_fail.txt" if changed timestamp then exec"/usr/local/bin/monit restart apache" alert xx...@xx... ----------------------------------------------------------- This Monit entry also contains a "start" entry, which will create an initial empty state file if one doesn't already exist. Hopefully, with this in place the server will be restarted if a logging process dies, rather than locking up. Restarting is more drastic than simply creating a new logging process, but it should be a rare event. One worry is that the state file inside the chroot provides a way for someone to maliciously cause the server to re-start from inside the chroot. However, if the chroot is compromised things are fairly bad already!! An alternative approach might be to use Monit to spot the growing number of locked child httpd processes, and just assume a problem rather than getting into all these details. I hope this is of use to someone. Regards, David. -- ------------------------------------------------- Email: Da...@me... ------------------------------------------------- |
