[Sbcl-devel] [patch, suggestion] --daemon,
--restart and --lock runtime options for the SBCL program
From: Heka T. <zen...@gm...> - 2011-05-02 16:37:21
Attachments:
daemon.patch
|
SBCL is often used as an application server, so it would be nice to have an easy way to demonize, to lock (i.e. to block the other versions of the sbcl program) and to restart crashed sbcl program. There are several ways to implement this: 1) Just use third-party programs such as `screen' (or `detach') and `daemonutils'. This solution has several drawbacks, daemonizer knows nothing about restarter and SBCL, while restarer knows nothing about daemonizer and SBCL (for example, what about to capture arguments and environment in restarter, or to clean up files which created by daemonizer (pid file) and locker (lock file)). 2) Use pure-lisp solution, i.e. do demonization in lisp in late stage, when VM is already running. The problem is that we can't do daemonization simply (standard sequence of calls to fork, setsid, umask, chdir, open("/dev/null") and dup2 on 0/1/2 descriptors is just don't work) and need to do additional steps. Also, restarter can't live in VM, because if the VM falls, restarter fall too. It must be an external C program, or individual sbcl process (which is not optimal for memory usage). 3) Implement all features in early runtime in SBCL (until threads of the main process has't started). This solution has several advantages, namely, we can add daemonization, restarts and locking that may somehow depend on each other and be integrated into the runtime of SBCL itself. Besides, it does not require a lot of code and provides a simple interface in the form of options of the sbcl program. This approach is similar to that used in other languages with rich VMs which have their own solution to demonization and related tasks (like jsvc for Java, or Erlang's erl -detached option). This is a scratch patch. --lock option is independent. If SBCL run with --lock option, then it create sbcl.lock file in the TMP_DIR (TMP_DIR is taken from the environment, just like SBCL_HOME, by default TMP_DIR=/tmp) and other versions of the sbcl program will fail to start. QUIT function can delete this sbcl.lock file, so when SBCL closes normally, it is automatically unlocked (otherwise, unlock = rm $TMP_DIR/sbcl.lock) --daemon do standard daemonization. STDIN, STDOUT and STDERR is forwarded to the /dev/null and all interaction should be made with IPC (system logging, shared files, sockets, etc.). --daemon-print-errors is like --daemon options, but don't unbind STDERR, so we can see errors in terminal. I also has try to implement --restart option, but it don't work. When SBCL running with --restart option it start two processes, one is the main SBCL process (which takes all of those megabytes in memory) and other is the supervisor process (very small, about 50kb of memory), but when the main process falls it also leads to a crash in the supervisor (possibly due to randomization of memory?). Example. Let run locked sbcl as daemon-server: # In building directory: $ BASE=`pwd` $ SBCL=$BASE/src/runtime/sbcl $ CORE=$BASE/output/sbcl.core $ TEST=$BASE/test.lisp $ SBCL_HOME=$BASE/contrib $SBCL --core $CORE \ --daemon-print-errors \ --lock \ --restart \ --disable-ldb \ --lose-on-corruption \ --end-runtime-options \ --no-sysinit \ --no-userinit \ --disable-debugger \ --load $TEST where test.lisp is (load "/path/to/quicklisp/setup.lisp") (require :iolib.sockets) (defpackage #:tcp-repl (:use #:cl #:sb-ext #:iolib.sockets)) (in-package #:tcp-repl) (defun tcp-read (host port) (let ((host (lookup-hostname host))) (with-open-socket (socket :connect :active :address-family :internet :type :stream :external-format '(:utf-8 :eol-style :crlf) :ipv6 nil) (connect socket host :port port :wait t) (read socket)))) (defun tcp-print (port value) (with-open-socket (server :connect :passive :address-family :internet :type :stream :ipv6 nil :external-format '(:utf-8 :eol-style :crlf)) (bind-address server +ipv4-unspecified+ :port port :reuse-addr t) (listen-on server :backlog 5) (with-accept-connection (client server :wait t) (print value client) (finish-output client)))) (defun tcp-repl-server () (loop (tcp-print 8080 :wait) (sleep 0.1) (tcp-print 8080 (eval (tcp-read "127.0.0.1" 8081))))) (defun tcp-repl-client () (loop (let ((sexp (read))) (tcp-read "127.0.0.1" 8080) (tcp-print 8081 sexp) (format t "< ~A~%" (tcp-read "127.0.0.1" 8080)))))) #-client (progn (format *error-output* "Running server.~%") (tcp-repl-server)) #+client (tcp-repl-client) after while it must print: Running server. Try to run second version of the sbcl program: $ sh run-sbcl.sh locking on file: /tmp/sbcl.lock We can see a daemon with `ps aux | grep sbcl | grep daemon`, also there is a files in TMP_DIR: $ ls /tmp | grep sbcl sbcl.lock sbcl.pid Now run SLIME and do (pushnew :client *features*) (load "test.lisp") we can communicate with the daemon over a sockets: 5 < 5 (+ 1 2 3) < 6 If we put (quit) in SLIME: (quit) QUIT is unlock daemon: ps aux | grep sbcl | grep daemon nothing ls /tmp | grep sbcl nothing otherwise in emergency case: stop = kill `cat $TMP_DIR/sbcl.pid` |
From: David L. <da...@li...> - 2011-05-02 17:40:08
|
Quoting Heka Treep (zen...@gm...): > 3) Implement all features in early runtime in SBCL (until threads of > the main process has't started). This solution has several advantages, > namely, we can add daemonization, restarts and locking that may > somehow depend on each other and be integrated into the runtime of > SBCL itself. Besides, it does not require a lot of code and provides a > simple interface in the form of options of the sbcl program. This > approach is similar to that used in other languages ??????with rich VMs > which have their own solution to demonization and related tasks (like > jsvc for Java, or Erlang's erl -detached option). Most of these features sound really cool; I'm personally a big fan of reliable daemon processes (think djb daemontools, albeit perhaps a little more userfriendly) and I agree that screen/detachtty, while sometimes nice, are not the right solution for everyone. However, it seems to me that most of these features could be implemented in a C program that then exec()s SBCL. One exception is the deletion of the lock file, but an exit hook can accomplish that without changes to SBCL. d. |
From: Heka T. <zen...@gm...> - 2011-05-03 17:43:24
|
Initially, I used such an self written external program, say the "sbd", it is normal, this program first perform execve() then execute wait() and accompanies the main process to its completion, after which it can either restart sbcl or clean up temporary files (either '.pid' and '.lock' files). What I don't like is the fact that in such a program one have to do extra work, I mean it must parse its arguments, pass some arguments to SBCL, pass environment to SBCL, have a help message, and all that "I the wrapper of the SBCL program" kind. In addition, if we run "sbd --lock", then the new version of the SDB will be blocked, but we can run SBCL with "sbcl" command, so that the locking does not quite work because SBCL not changed itself. So the question is whether the SBCL program should include a mechanism of demonization or not ("all inclusive" or "use external tools"). If yes, then this can be done by adding a few lines in runtime.с as in that patch. As for the function QUIT, it is not necessary to modify, if the supervisor is always run, he can assume the role of cleaning '.pid' and '.lock' files. 2011/5/2, David Lichteblau <da...@li...>: > Quoting Heka Treep (zen...@gm...): >> 3) Implement all features in early runtime in SBCL (until threads of >> the main process has't started). This solution has several advantages, >> namely, we can add daemonization, restarts and locking that may >> somehow depend on each other and be integrated into the runtime of >> SBCL itself. Besides, it does not require a lot of code and provides a >> simple interface in the form of options of the sbcl program. This >> approach is similar to that used in other languages ??????with rich VMs >> which have their own solution to demonization and related tasks (like >> jsvc for Java, or Erlang's erl -detached option). > > Most of these features sound really cool; I'm personally a big fan of > reliable daemon processes (think djb daemontools, albeit perhaps a > little more userfriendly) and I agree that screen/detachtty, while > sometimes nice, are not the right solution for everyone. > > However, it seems to me that most of these features could be implemented > in a C program that then exec()s SBCL. One exception is the deletion > of the lock file, but an exit hook can accomplish that without changes > to SBCL. > > > d. > |
From: Thomas F. B. <tbu...@gm...> - 2011-05-04 09:02:03
|
2011/5/2 Heka Treep <zen...@gm...>: > 2) Use pure-lisp solution, i.e. do demonization in lisp in late stage, > when VM is already running. The problem is that we can't do > daemonization simply (standard sequence of calls to fork, setsid, > umask, chdir, open("/dev/null") and dup2 on 0/1/2 descriptors is just > don't work) and need to do additional steps. Also, restarter can't > live in VM, because if the VM falls, restarter fall too. It must be an > external C program, or individual sbcl process (which is not optimal > for memory usage). I have a few servers deployed using a pure-lisp demonization setup. It works just fine, and has the added benefit of only demonizing once the program has successfully opened its log files, bound the port, etc., so it can report startup errors to the user who started it (as well as the log file, if it can open it). The additional memory usage per active process is quite reasonable (something like 20 MB per child process). The rest of the overhead gets shared between all the running SBCLs. This is the solution I would recommend to anyone writing a server with SBCL -- it works just fine[*] for us, and the result behaves like a normal unix daemon. [*] Fine, except for an external-format issue: we had to hack the C external format code to force SBCL to use UTF8 as the default external format, even on machines which don't have a utf8 locale. |
From: Slobodan M. <slo...@gm...> - 2011-05-04 13:00:42
|
2011/5/4 Thomas F. Burdick <tbu...@gm...>: > I have a few servers deployed using a pure-lisp demonization setup. It > works just fine, and has the added benefit of only demonizing once the > program has successfully opened its log files, bound the port, etc., > so it can report startup errors to the user who started it (as well as > the log file, if it can open it). The additional memory usage per > active process is quite reasonable (something like 20 MB per child > process). The rest of the overhead gets shared between all the running > SBCLs. This is the solution I would recommend to anyone writing a > server with SBCL -- it works just fine[*] for us, and the result > behaves like a normal unix daemon. > > [*] Fine, except for an external-format issue: we had to hack the C > external format code to force SBCL to use UTF8 as the default external > format, even on machines which don't have a utf8 locale. Hi! Sorry for stealing the thread, but what you have is exactly what I need. :-D Would you be willing to share some code, or at least point to your sources (links) that you have been using to develop these pure-lisp demonization setups? I would like to try to make an setup like that, but I don't have enough knowledge yet to start it on my own. |
From: Nikodemus S. <nik...@ra...> - 2011-05-05 16:03:15
|
2011/5/4 Slobodan Milnović <slo...@gm...>: > 2011/5/4 Thomas F. Burdick <tbu...@gm...>: >> I have a few servers deployed using a pure-lisp demonization setup. It >> works just fine, and has the added benefit of only demonizing once the >> program has successfully opened its log files, bound the port, etc., >> so it can report startup errors to the user who started it (as well as > Would you be willing to share some code, or at least point to your > sources (links) that you have been using to develop these pure-lisp > demonization setups? I would like to try to make an setup like that, > but I don't have enough knowledge yet to start it on my own. I think SB-DAEMON would make a fine contrib... Cheers, -- Nikodemus |
From: Heka T. <zen...@gm...> - 2011-05-05 21:02:14
|
2011/5/4, Slobodan Milnović <slo...@gm...>: > > Hi! > > Sorry for stealing the thread, but what you have is exactly what I need. :-D > > Would you be willing to share some code, or at least point to your > sources (links) that you have been using to develop these pure-lisp > demonization setups? I would like to try to make an setup like that, > but I don't have enough knowledge yet to start it on my own. Look at cl-daemonize (https://github.com/mishoo/cl-daemonize), it works well with SBCL on Linux. Also restas-daemon.lisp (https://github.com/archimag/restas/blob/master/contrib/restas-daemon.lisp) from restas project can be used. But I not sure about automatic restarts, maybe such pure-lisp daemonization + daemonutils can be right solution. |
From: Slobodan M. <slo...@gm...> - 2011-05-06 07:01:36
|
2011/5/5 Heka Treep <zen...@gm...>: > 2011/5/4, Slobodan Milnović <slo...@gm...>: >> >> Hi! >> >> Sorry for stealing the thread, but what you have is exactly what I need. :-D >> >> Would you be willing to share some code, or at least point to your >> sources (links) that you have been using to develop these pure-lisp >> demonization setups? I would like to try to make an setup like that, >> but I don't have enough knowledge yet to start it on my own. > > Look at cl-daemonize (https://github.com/mishoo/cl-daemonize), it > works well with SBCL on Linux. Also restas-daemon.lisp > (https://github.com/archimag/restas/blob/master/contrib/restas-daemon.lisp) > from restas project can be used. Great links, thank you very much! > But I not sure about automatic restarts, maybe such pure-lisp > daemonization + daemonutils can be right solution. My experience as an sysadmin tells me that, even if the daemon has in itself an module to detect crash and to automatically restart itself (or do something else, like sending email), sometimes the situation can be so bad, that it is unable to do so. That's why nagios and similar external (in a sense of not beeing part of the daemon itself) monitoring tools are there for. But, of course, the pure-lisp automatic restarts setup would be great. :-) |
From: Thomas F. B. <tbu...@gm...> - 2011-05-06 11:47:15
|
2011/5/5 Nikodemus Siivola <nik...@ra...>: > 2011/5/4 Slobodan Milnović <slo...@gm...>: >> 2011/5/4 Thomas F. Burdick <tbu...@gm...>: >>> I have a few servers deployed using a pure-lisp demonization setup. It >>> works just fine, and has the added benefit of only demonizing once the >>> program has successfully opened its log files, bound the port, etc., >>> so it can report startup errors to the user who started it (as well as > >> Would you be willing to share some code, or at least point to your >> sources (links) that you have been using to develop these pure-lisp >> demonization setups? I would like to try to make an setup like that, >> but I don't have enough knowledge yet to start it on my own. > > I think SB-DAEMON would make a fine contrib... I'll look at pulling the daemon code apart from the Ltk code, and put something on repo.or.cz. |
From: Nikodemus S. <nik...@ra...> - 2011-05-26 19:09:30
|
2011/5/6 Thomas F. Burdick <tbu...@gm...>: >> I think SB-DAEMON would make a fine contrib... > > I'll look at pulling the daemon code apart from the Ltk code, and put > something on repo.or.cz. Turns out I needed this too, and the existing offerings didn't quite fit my bill so I rolled SB-DAEMON up. https://github.com/nikodemus/sb-daemon I'm not a POSIX process management guru by any stretch of the imagination, so "you should / shouldn't" comments are more than welcome. I eg. opted to do as many things as possible before forking in order to make the debugging the setup (permission issues with files, etc) a bit less painful. Not 100% sure this is kosher for general purpose daemonization. So are patches that add functionality other people need. Requests for such functionality are welcome too, but ... patches are better. Cheers, -- Nikodemus |
From: Nikodemus S. <nik...@ra...> - 2011-05-27 12:16:18
|
2011/5/26 Nikodemus Siivola <nik...@ra...>: > https://github.com/nikodemus/sb-daemon I incorporated some excellent advice from Andreas Fuchs, and then added some feature creep: * Current directory is always changed to / in the child. Those needing something else can chdir in the child afterwards. * Added process reaping for :EXIT-PARENT NIL case. Reaping can be explicitly disabled if the parents wants to waitpid() on its own. * Disabled debugger in child by default. * Made the daemonization interrupt safe-er. * Added support for :EXIT-HOOK for the :EXIT-PARENT NIL case. * Added options to install handlers in the child for :SIGTERM, :SIGABRT, and :SIGINT. (Adding new ones is trivial -- advice on which ones to add support for would be good. So would advice if there are any built-in handlers that would be good to provide so everyone doesn't have to write the same ones...) Cheers, -- nikodemus |
From: Jim W. <jw...@dr...> - 2011-05-27 13:31:31
|
Nikodemus Siivola <nik...@ra...> writes: > * Added options to install handlers in the child for :SIGTERM, > :SIGABRT, and :SIGINT. (Adding new ones is trivial -- advice on which > ones to add support for would be good. So would advice if there are > any built-in handlers that would be good to provide so everyone > doesn't have to write the same ones...) SIGHUP comes to mind -- idiomatically, the child should respond by reloading its configuration (if any), which will obviously be specific to the app, but a quick way to pass a proc to be called on HUP would be a win. -- Jim Wise jw...@dr... |
From: Nikodemus S. <nik...@ra...> - 2011-05-27 13:54:07
|
On 27 May 2011 16:31, Jim Wise <jw...@dr...> wrote: > SIGHUP comes to mind -- idiomatically, the child should respond by Done. :) Cheers, -- nikodemus |