From: Martin A. <ma...@at...> - 2001-01-10 12:12:44
Attachments:
load-foreign-patch.gz
|
Hi, with the attached patch, load-foreign seems to work. I've tested this using the sample C and alien code in the doc-string of load-1-foreign, so far. I also only loaded the modified foreign.lisp in a running lisp and did not build a new core, but this should not be a problem, since foreign.lisp is loaded in warm init. Cheers, Martin -- Homepage: http://www.atzmueller.net/ Email: ma...@at... |
From: William H. N. <wil...@ai...> - 2001-01-10 15:07:12
|
On Wed, Jan 10, 2001 at 02:11:43PM +0100, Martin Atzmueller wrote: > with the attached patch, load-foreign seems to work. > > I've tested this using the sample C and alien code in > the doc-string of load-1-foreign, so far. > I also only loaded the modified foreign.lisp in a running lisp > and did not build a new core, but this should not be a > problem, since foreign.lisp is loaded in warm init. OK, thank you. It may be a week or more before I do anything with the patch. I'd like to release 0.6.10 soon (maybe this weekend?), and I like to test a prerelease version for a little while before I release it, without tweaking it too much. There is an annoying bug in signal handling or the debugger in my current 0.6.9.23-in-progress, and I want to fix that, but that may well be the last change in the program before 0.6.10. -- William Harold Newman <wil...@ai...> software consultant PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C |
From: Martin A. <ma...@at...> - 2001-01-10 22:16:41
|
William Harold Newman wrote: [...] > It may be a week or more before I do anything with the patch. I'd like > to release 0.6.10 soon (maybe this weekend?), and I like to test a > prerelease version for a little while before I release it, without > tweaking it too much. There is an annoying bug in signal handling or > the debugger in my current 0.6.9.23-in-progress, and I want to fix > that, but that may well be the last change in the program before > 0.6.10. Well, ok. You'll see that it is quite a small patch, it kind of emulates sb-ext:*environment-list* behavior to get the current environment, and that is all. Currently the env parameter that can be passed to #'load-foreign is in the format of CMUCL's *environment-list*, so it is an a-list mapping keywords and simple strings. This is the same thing that can be given to #'run-program as its env-parameter. I think it makes sense to keep this, but nevertheless I have noted that the CMUCL manual, p. 120 talks about a list of strings (not an a-list) in the format of UN*X environment variables, like A=B (key A, value B) as the env-parameter of #'load-foreign It would be quite easy to convert this to the other format as above. If we keep the current implementation, probably the SBCL manual should mention this and so, in contrast to the CMUCL manual, describe the correct situation. Cheers, Martin -- Homepage: http://www.atzmueller.net/ Email: ma...@at... |
From: Daniel B. <da...@no...> - 2001-01-10 22:36:25
|
Martin Atzmueller <ma...@at...> writes: > Currently the env parameter that can be passed to #'load-foreign is in > the format of CMUCL's *environment-list*, so it is an a-list > mapping keywords and simple strings. > This is the same thing that can be given to #'run-program as its > env-parameter. It's worth noting that this fornmat is not information preserving: environment variable names can have lowercase characters in them, but CMUCL uppercases them into *environment-list*. 9 times of ten it won't matter, of course, but it's still clearly not The Right Thing -dan -- http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources |
From: William H. N. <wil...@ai...> - 2001-01-11 01:15:50
|
On Thu, Jan 11, 2001 at 12:13:21AM +0100, Martin Atzmueller wrote: > > It may be a week or more before I do anything with the patch. I'd like > > to release 0.6.10 soon (maybe this weekend?), and I like to test a > > prerelease version for a little while before I release it, without > > tweaking it too much. There is an annoying bug in signal handling or > > the debugger in my current 0.6.9.23-in-progress, and I want to fix > > that, but that may well be the last change in the program before > > 0.6.10. > Well, ok. You'll see that it is quite a small patch, it kind of > emulates sb-ext:*environment-list* behavior to get the current > environment, and that is all. If you or anyone has a strong interest in having it available in the next release, I can do it. I just didn't think it was urgent; and thinking that it wasn't urgent, I wanted to warn you so that you wouldn't think I'd forgotten it. > Currently the env parameter that can be passed to #'load-foreign is in > the format of CMUCL's *environment-list*, so it is an a-list > mapping keywords and simple strings. > This is the same thing that can be given to #'run-program as its > env-parameter. > I think it makes sense to keep this, but nevertheless I have noted that > the CMUCL manual, p. 120 talks about a list of strings (not an a-list) > in the format of UN*X environment variables, like A=B (key A, value B) > as the env-parameter of #'load-foreign > It would be quite easy to convert this to the other format as above. > If we keep the current implementation, probably the SBCL manual > should mention this and so, in contrast to the CMUCL manual, describe > the correct situation. Probably the Lispiest thing would be an alist, like this '(("USER" . "newman") ("PAGER" . "less") ("PWD" . "/usr/stuff") ("_" . "/usr/bin/env")) But that's only my second choice of format. My first choice would be a plain list of strings like this '("USER=newman" "PAGER=less" "PWD=/usr/stuff" "_=/usr/bin/env") This format is a little harder to work with, but since "man environ" says that the name=value form is only a convention, not a requirement, I don't want to make RUN-PROGRAM dependent on the name=value form. (And the list-of-strings format is only infinitesimally harder to work with than the alist format. One-liner functions to convert between the two formats are left as an exercise to the reader.) (And I agree with Daniel Barlow that the list-of-keywords approach that CMU CL uses is the wrong thing.) -- William Harold Newman <wil...@ai...> software consultant PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C |
From: Martin A. <ma...@at...> - 2001-01-11 13:40:22
|
William Harold Newman wrote: > > On Thu, Jan 11, 2001 at 12:13:21AM +0100, Martin Atzmueller wrote: > > > It may be a week or more before I do anything with the patch. I'd like > > > to release 0.6.10 soon (maybe this weekend?), and I like to test a > > > prerelease version for a little while before I release it, without > > > tweaking it too much. There is an annoying bug in signal handling or > > > the debugger in my current 0.6.9.23-in-progress, and I want to fix > > > that, but that may well be the last change in the program before > > > 0.6.10. > > Well, ok. You'll see that it is quite a small patch, it kind of > > emulates sb-ext:*environment-list* behavior to get the current > > environment, and that is all. > > If you or anyone has a strong interest in having it available in the > next release, I can do it. I just didn't think it was urgent; and > thinking that it wasn't urgent, I wanted to warn you so that you > wouldn't think I'd forgotten it. Thank you for the clarification. If you got the wrong impression, that I wanted to push this issue, well, this is my fault. ;) I think it is not that urgent to include the patch, because it can be loaded at runtime. I just thought it might be easy to include it, because it is just a very localized change. > > Currently the env parameter that can be passed to #'load-foreign is in > > the format of CMUCL's *environment-list*, so it is an a-list > > mapping keywords and simple strings. > > This is the same thing that can be given to #'run-program as its > > env-parameter. > > I think it makes sense to keep this, but nevertheless I have noted that > > the CMUCL manual, p. 120 talks about a list of strings (not an a-list) > > in the format of UN*X environment variables, like A=B (key A, value B) > > as the env-parameter of #'load-foreign [...] > Probably the Lispiest thing would be an alist, like this > '(("USER" . "newman") > ("PAGER" . "less") > ("PWD" . "/usr/stuff") > ("_" . "/usr/bin/env")) > But that's only my second choice of format. My first choice would > be a plain list of strings like this > '("USER=newman" "PAGER=less" "PWD=/usr/stuff" "_=/usr/bin/env") > This format is a little harder to work with, but since "man environ" > says that the name=value form is only a convention, not a requirement, > I don't want to make RUN-PROGRAM dependent on the name=value form. I agree. I just don't like the idea of having two different env formats in SBCL, one for run-program, and one for load-foreign. I have looked at the code of run-program: internally it converts the alist to the "list-of-strings" format. So, maybe both run-program and load-foreign should accept an alist like the one you described above, and this would also solve the information preserving problem pointed out by Dan Barlow Cheers, Martin -- Homepage: http://www.atzmueller.net/ Email: ma...@at... |
From: William H. N. <wil...@ai...> - 2001-01-11 14:45:43
|
On Thu, Jan 11, 2001 at 03:37:24PM +0100, Martin Atzmueller wrote: > > > Currently the env parameter that can be passed to #'load-foreign is in > > > the format of CMUCL's *environment-list*, so it is an a-list > > > mapping keywords and simple strings. > > > This is the same thing that can be given to #'run-program as its > > > env-parameter. > > > I think it makes sense to keep this, but nevertheless I have noted that > > > the CMUCL manual, p. 120 talks about a list of strings (not an a-list) > > > in the format of UN*X environment variables, like A=B (key A, value B) > > > as the env-parameter of #'load-foreign > > [...] > > Probably the Lispiest thing would be an alist, like this > > '(("USER" . "newman") > > ("PAGER" . "less") > > ("PWD" . "/usr/stuff") > > ("_" . "/usr/bin/env")) > > But that's only my second choice of format. My first choice would > > be a plain list of strings like this > > '("USER=newman" "PAGER=less" "PWD=/usr/stuff" "_=/usr/bin/env") > > This format is a little harder to work with, but since "man environ" > > says that the name=value form is only a convention, not a requirement, > > I don't want to make RUN-PROGRAM dependent on the name=value form. > I agree. I just don't like the idea of having two different env > formats in SBCL, one for run-program, and one for load-foreign. > I have looked at the code of run-program: > internally it converts the alist to the "list-of-strings" format. > So, maybe both run-program and load-foreign should accept an alist > like the one you described above, and this would also solve > the information preserving problem pointed out by Dan Barlow I had lost track of the fact that RUN-PROGRAM was already set up to use the CMU-CL-style alist; I thought somehow that when I got rid of the CMU CL *ENVIRONMENT* variable, I'd banished those lists from SBCL, so I thought now we were starting afresh. But even now that I realize that RUN-PROGRAM is already using the alist-of-keywords, I think the problems with the alist-of-keywords style are annoying enough that it's worth changing RUN-PROGRAM to list-of-strings style too. Perhaps, though, we should retain the RUN-PROGRAM ENV keyword argument, with the old alist-of-keywords interface, as a deprecated feature for a while. We could use another keyword argument, perhaps ENV-STRINGS, for the new interface. I could do this early in 0.6.10, probably next week. -- William Harold Newman <wil...@ai...> software consultant PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C |
From: Martin A. <ma...@at...> - 2001-01-15 12:26:36
|
William Harold Newman wrote: > > [...] > > > Probably the Lispiest thing would be an alist, like this > > > '(("USER" . "newman") > > > ("PAGER" . "less") > > > ("PWD" . "/usr/stuff") > > > ("_" . "/usr/bin/env")) > > > But that's only my second choice of format. My first choice would > > > be a plain list of strings like this > > > '("USER=newman" "PAGER=less" "PWD=/usr/stuff" "_=/usr/bin/env") > > > This format is a little harder to work with, but since "man environ" > > > says that the name=value form is only a convention, not a requirement, > > > I don't want to make RUN-PROGRAM dependent on the name=value form. > > I agree. I just don't like the idea of having two different env > > formats in SBCL, one for run-program, and one for load-foreign. > > I have looked at the code of run-program: > > internally it converts the alist to the "list-of-strings" format. > > So, maybe both run-program and load-foreign should accept an alist > > like the one you described above, and this would also solve > > the information preserving problem pointed out by Dan Barlow > > I had lost track of the fact that RUN-PROGRAM was already set up to > use the CMU-CL-style alist; I thought somehow that when I got rid of > the CMU CL *ENVIRONMENT* variable, I'd banished those lists from SBCL, > so I thought now we were starting afresh. But even now that I realize > that RUN-PROGRAM is already using the alist-of-keywords, I think the > problems with the alist-of-keywords style are annoying enough that > it's worth changing RUN-PROGRAM to list-of-strings style too. So, we are moving to the list-of-strings, like "PWD=/home/user/foo", for both run-program and load-foreign? I guess this would be reasonable. The "alist-of-strings" would be nice, but the "list-of-strings" is ok, IMO. > Perhaps, though, we should retain the RUN-PROGRAM ENV keyword > argument, with the old alist-of-keywords interface, as a deprecated > feature for a while. We could use another keyword argument, perhaps > ENV-STRINGS, for the new interface. I could do this early in 0.6.10, > probably next week. What does "deprecated" mean, in this context? Will the :ENV-STRINGS be replaced by the :ENV keyword, eventually? Cheers, Martin -- Homepage: http://www.atzmueller.net/ Email: ma...@at... |
From: William H. N. <wil...@ai...> - 2001-01-15 16:25:50
|
On Mon, Jan 15, 2001 at 02:19:26PM +0100, Martin Atzmueller wrote: > William Harold Newman wrote: [..] > > Perhaps, though, we should retain the RUN-PROGRAM ENV keyword > > argument, with the old alist-of-keywords interface, as a deprecated > > feature for a while. We could use another keyword argument, perhaps > > ENV-STRINGS, for the new interface. I could do this early in 0.6.10, > > probably next week. > What does "deprecated" mean, in this context? My ideas was to do it roughly the same way as in QUIT: code using the old interface still works for a while, but issues a warning. (defun quit (&key recklessly-p (unix-code 0 unix-code-p) (unix-status unix-code)) #!+sb-doc "Terminate the current Lisp. Things are cleaned up (with UNWIND-PROTECT and so forth) unless RECKLESSLY-P is non-NIL. On UNIX-like systems, UNIX-STATUS is used as the status code." (declare (type (signed-byte 32) unix-code)) ;; FIXME: UNIX-CODE was deprecated in sbcl-0.6.8, after having been ;; around for less than a year. It should be safe to remove it after ;; a year. (when unix-code-p (warn "The UNIX-CODE argument is deprecated. Use the UNIX-STATUS argument instead (which is another name for the same thing).")) (if recklessly-p (sb!unix:unix-exit unix-status) (throw '%end-of-the-world unix-status))) > Will the :ENV-STRINGS be replaced by the :ENV keyword, eventually? Ordinarily I'd say :ENV is a better name. However, since CMU CL is out there using ENV to mean something else in the same context, :ENV has the potential to cause confusion. Confusion seems especially likely since it's so reasonable to port software between CMU CL and SBCL. I'd be inclined to stick with :ENV-STRINGS; or at least some other other-than-:ENV name, if someone has a better suggestion than :ENV-STRINGS. -- William Harold Newman <wil...@ai...> software consultant PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C |
From: Daniel B. <da...@no...> - 2001-01-15 16:43:59
|
William Harold Newman <wil...@ai...> writes: > be inclined to stick with :ENV-STRINGS; or at least some other > other-than-:ENV name, if someone has a better suggestion than > :ENV-STRINGS. Following th style suggestion "there are n ways to abbreviate a name and only one to spell it out in full", what about :ENVIRONMENT ? OK, so it's more typing, but that's what symbol completion and dabbrev support are for. -dan -- http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources |
From: William H. N. <wil...@ai...> - 2001-01-15 23:09:56
|
On Mon, Jan 15, 2001 at 04:43:45PM +0000, Daniel Barlow wrote: > William Harold Newman <wil...@ai...> writes: > > > be inclined to stick with :ENV-STRINGS; or at least some other > > other-than-:ENV name, if someone has a better suggestion than > > :ENV-STRINGS. > > Following th style suggestion "there are n ways to abbreviate a name > and only one to spell it out in full", what about :ENVIRONMENT ? OK, > so it's more typing, but that's what symbol completion and dabbrev > support are for. That sounds completely reasonable to me in this particular case, where it's only used a few times. In the general case, IMHO it's good to impose a variant of the style suggestion for some common terms: "there's only one way to write this common term (in this software system) (and it's an abbreviation)." I agree that it's not hard to type out FUNCTION, VARIABLE, LEXICAL-ENVIRONMENT, and so forth, but they're long enough and common enough that they can start to impair the clarity of the program by causing too many lines to wrap; and they're common enough that it's not a great burden to remember their standard abbreviations, as long as the abbreviations are always used. One of the (low-priority, but also easy) items on my to-do list is to go through SBCL standardizing some commonly abbreviated things: FUN, VAR, LEXENV, ARG, INIT, INST(instruction), SEQ, PRED, etc. (Of course, I can't do anything about inconsistencies written into the ANSI standard itself, like CL:MAKE-SEQUENCE vs. CL:COPY-SEQ, but at least I can remove inconsistencies in SBCL's internal names.) -- William Harold Newman <wil...@ai...> software consultant PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C |
From: William H. N. <wil...@ai...> - 2001-01-17 23:16:45
|
On Mon, Jan 15, 2001 at 10:14:46AM -0600, I wrote: > On Mon, Jan 15, 2001 at 02:19:26PM +0100, Martin Atzmueller wrote: > > William Harold Newman wrote: > [..] > > > Perhaps, though, we should retain the RUN-PROGRAM ENV keyword > > > argument, with the old alist-of-keywords interface, as a deprecated > > > feature for a while. We could use another keyword argument, perhaps > > > ENV-STRINGS, for the new interface. I could do this early in 0.6.10, > > > probably next week. > > What does "deprecated" mean, in this context? > > My ideas was to do it roughly the same way as in QUIT: code using the > old interface still works for a while, but issues a warning. I'm now rewriting the handling of the Unix environment both in RUN-PROGRAM and in LOAD-FOREIGN. It seems to me that the current RUN-PROGRAM behavior of defaulting the environment to empty (instead of defaulting the environment to the current process's environment) is strange. Maybe it was normal when CMU CL was first written, but I'm used to the way that perl (and AFAIK most other programs) do it, defaulting to a copy of the current process's environment. Would it be reasonable to change this? It's an incompatible change, and I can't think of any gentle way to phase it in. However, I find the old behavior so surprising that it still seems reasonable to change it. (If it's a good idea, doing it as soon as possible might make sense. Alternatively, waiting until 0.7.x would also be reasonable.) -- perl -e 'system "env";' (run-program "/usr/bin/env" nil :output t) (run-program "/usr/bin/env" nil :output t :env '((:foo . "bar"))) -- William Harold Newman <wil...@ai...> software consultant PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C |
From: Martin A. <ma...@at...> - 2001-01-18 11:18:40
|
William Harold Newman wrote: [...] > I'm now rewriting the handling of the Unix environment both in > RUN-PROGRAM and in LOAD-FOREIGN. It seems to me that the current > RUN-PROGRAM behavior of defaulting the environment to empty (instead > of defaulting the environment to the current process's environment) is > strange. Maybe it was normal when CMU CL was first written, but I'm > used to the way that perl (and AFAIK most other programs) do it, > defaulting to a copy of the current process's environment. > > Would it be reasonable to change this? It's an incompatible change, > and I can't think of any gentle way to phase it in. However, I find > the old behavior so surprising that it still seems > reasonable to change it. Yes, I think it would be reasonable. Looking at your sample code ( SBCL vs. perl) quite effectively confirmed this. I don't know, if making this incompatible change has some bad effects. Currently I can't think of any ... Cheers, Martin -- Martin Atzmueller <ma...@at...> |
From: Daniel B. <da...@no...> - 2001-01-18 12:52:57
|
Martin Atzmueller <ma...@at...> writes: > Yes, I think it would be reasonable. Looking at your sample code > ( SBCL vs. perl) quite effectively confirmed this. > I don't know, if making this incompatible change has some bad effects. > Currently I can't think of any ... From a security standpoint there's an argument for starting new programs with known environment contents rather than "whatever the parent had". If (a) the parent allows its environment to be changed by a user - and in a large Lisp system it's going to be moderately difficult to prove it doesn't (turned *read-eval* off? everywhere?), and (b) the parent program operates with different privileges from whatever the user would usually have (not necessarily 'root'; it could be some kind of persistent process running as 'httpd' or 'nobody', for example), and invokes the child with those same "interesting" privileges, there are all kinds of fun environment variables that can be set to subvert the child's behaviour I'm thinking about things like the telnetd exploit of yesteryear. From memory (I could be wrong) telnetd stripped everything from the enviromnent that it thought might be dangerous (reset PATH, IFS to sane values, etc) but having been written in the days before dynamic loading it didn't know about LD_PRELOAD. And LD_PRELOAD when set by a malicious user can be _very_ dangerous. Emptying the environment won't make it impossible for people to code up systems with these kinds of holes, but will help it not to be the default action. -dan -- http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources |
From: William H. N. <wil...@ai...> - 2001-01-18 19:15:55
|
On Thu, Jan 18, 2001 at 12:52:36PM +0000, Daniel Barlow wrote: > >From a security standpoint there's an argument for starting new > programs with known environment contents rather than "whatever the > parent had". If > > (a) the parent allows its environment to be changed by a user - and > in a large Lisp system it's going to be moderately difficult to prove > it doesn't (turned *read-eval* off? everywhere?), and (I agree with the general thrust of your argument, but I'd quibble here: If you have *READ-EVAL* on for data which is under the control of your adversary, your security is gone immediately. The adversary doesn't need to mess around with the Unix environment.) > (b) the parent program operates with different privileges from > whatever the user would usually have (not necessarily 'root'; it > could be some kind of persistent process running as 'httpd' or > 'nobody', for example), and invokes the child with those same > "interesting" privileges, > > there are all kinds of fun environment variables that can be set to > subvert the child's behaviour > > I'm thinking about things like the telnetd exploit of yesteryear. > >From memory (I could be wrong) telnetd stripped everything from the > enviromnent that it thought might be dangerous (reset PATH, IFS to sane > values, etc) but having been written in the days before dynamic > loading it didn't know about LD_PRELOAD. And LD_PRELOAD when set by a > malicious user can be _very_ dangerous. > > Emptying the environment won't make it impossible for people to code > up systems with these kinds of holes, but will help it not to be the > default action. That's a valid point, but I think that instead of trying to defend innocents who wander into the wild world of setuid security, I'm going to surrender without a fight. Even in Perl, where setuid security is a big deal and they've devoted a lot of engineering effort to the problem (tracking tainted data, etc.) you can still inadvertently make an insecure script. And there's no way we're going to devote nearly that much engineering effort to the problem in SBCL. Besides that, SBCL itself is already a security problem if the adversary controls the Unix environment, because of the SBCL_HOME variable. So I think I will copy the Unix environment by default. However, in recognition of the long and distinguished history of Unix setuid security screwups, I'll also put a note in the RUN-PROGRAM doc string, something like this: Running Unix programs from a setuid process, or in any other situation where the Unix environment is under the control of someone else, is a mother lode of security problems. If you are contemplating doing this, read about it first. (The Perl community has a lot of good documentation about this and other security issues in script-like programs.) (This is the second time I can remember you pointing out security issues in SBCL. I'll have to consider your software the next time I set up a server. It should go well with OpenBSD.:-) -- William Harold Newman <wil...@ai...> software consultant PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C |