I don't know if my previous message made it, due to the recent moderation settings change on the list.
I'm trying to publish a new version of cl-launch, and I would like to add support for standalone executables on clisp.
The issue is that (at least up to the clisp 1:2.44-1 included in debian), there is no mechanism to prevent a saved image from processing such options as -M -x -q -norc, etc. The :script t option to saveinitmem doesn't do that at all (as of the above-mentioned clisp at least).
I want all downstream users to be able to gain access to the lisp prompt.
I am still unclear how to do that while surrendering all command line argument processing downstream.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
2- In case no binary is available (not provided, lost, etc.), you could or make it trivial to find where the flag is in the binary, so that anyone can edit it and flip the flag... for instance store the flag as an ASCII character at the end of an easy-to-find ASCII string e.g. "PARSE-OPTIONS+" vs "PARSE-OPTIONS-".
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
2- The important thing here is that there could be a trivial algorithm to flip the bit, and that anyone with emacs could do it. A string with PARSE-OPTIONS=Y or PARSE-OPTIONS=N prominently at the beginning or end of the binary file, would do wonders (defaulting to Y if nothing is found).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
editing binary files is much harder than you seem to think.
even emacs may fail if you have
(add-hook 'write-file-functions 'delete-trailing-whitespace)
in your .emacs.
and chances are that a windows (or aix or ...) user does NOT have emacs installed.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You can provide a simple lisp script that will unlock a binary (after proper sanity checks), split it back into executable and image, juggle with 9 balls, etc.
But when I'm implementing my lisp version of ls, I want ls -x * to mean "list entries by line", not "evaluate the first file name as a lisp expression".
If for whatever reason you want to get a REPL from my binary so as to hot patch it, I can use that simple tool to "unlock" the binary. And if I forgot that tool and don't want to download it, I can just use my text editor.
(Or then again, I can rework from the sources from which I built the application, if I have them - like anyone does in any other language.)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
the model situation is a person who has your binary for his obscure platform and nothing else.
he wants clisp prompt.
he cannot build (he does not have gcc installed).
he cannot edit (he does not have emacs installed).
using an unlocking script on a different machine is an option...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> the model situation is a person who
> has your binary for his obscure
> platform and nothing else.
> he wants clisp prompt.
> he cannot build (he does not have
> gcc installed).
> he cannot edit (he does not have
> emacs installed).
This is a good point.
I think that if a person has a stand-alone
executable named myprog which is stored
using (saveinitmem ... :parse-options nil)
then the user should be able to counter
the :parse-option using the commandline.
As an example, one could decide that e.g.
giving --clisp-m 1000MB to myprog was
handled like giving -m 1000MB to clisp
And one could decide that e.g.
> myprog --clisp arg1 ... argn
was handled like
> clisp arg1 ... argn
Or one could decide that
> myprog ++ arg1 ... argn
was handled like
> clisp arg1 ... argn
so that ++ was the opposite of --.
My original patch was naive.
At that time I wrote:
> If :parse-options is NIL then all argv
> parameters are passed untouched
> to the program as if they were
> preceeded by an implicit "--".
I think "untouched" should be
"almost untouched" since there
should be particular options which
are still recognized such as
--clisp-m, --clisp, or ++.
One could even insist that --help,
--version, and --license were *always*
processed by clisp, leaving image
specific help to -help-image. Maybe
there should be a -version-image
and -license-image also.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
# create image:
$ clisp -x '(saveinitmem "foo" :executable 0 :init-function (lambda () (print *args*) (quit 0)))'
# it delegates all arguments to the init-function
$ ./foo -M a bc d -a -x
# except for arguments prefixed with --clisp:
$ ./foo --clisp--help
# which allows you to recover clisp prompt
$ ./foo --clisp-x '(saveinitmem "myclisp" :executable t :init-function (function sys::main-loop))'
# here is your clisp prompt
$ ./myclisp
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
On the one hand, I'm not 100% satisfied because who it's still not 100% transparent,
but on the other hand I'm 99.9% satisfied because someone really has to be looking for trouble to detect the discrepancy.
Thanks a whole lot. And .1% grrr!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK, so I've added support for standalone executables with CLISP 2.48, with the below caveat in the documentation explaining the security hazard of a setuid such standalone executable:
Note that with CLISP, standalone executables will react magically if
invoked with options such as --clisp-help or --clisp-x '(sys::main-loop)'.
That's a pretty far-fetched thing to hit by mistake, and the CLISP
maintainers consider it a feature, but don't let untrusted users control
arguments given to such executables that are run with extra privileges.
Maybe any --clisp-foo option should cause CLISP to drop any setuid privileges?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
are you seriously considering installing a clisp-based application suid root?!
wow.
OK, not a problem: check (ext:argv) from a function in *init-hooks* and remove setuid,
see http://clisp.podval.org/impnotes/image.html#image-exec
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well, I would like to eventually turn CL into a systems programming language, and make it reasonably portable with cl-launch as the front-end for casual deployment.
As for your init-hook, needn't I only care about the first argument, instead of doing a find? And shouldn't I be using POSIX:UID (i.e. getuid()) instead of POSIX:USER-INFO (since getlogin() that might fail when there's no terminal, or maybe even lie on some unices if it trusts LOGNAME)?
And shouldn't that init-hook be on by default?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>needn't I only care about the first argument, instead of doing a find?
I think you do need a find.
>And shouldn't I be using POSIX:UID (i.e. getuid()) instead of
do you think (setf (uid) (uid)) will do you much good? :-)
oh, it should be (setf (euid) (uid) (egid) (gid)), right?
> POSIX:USER-INFO (since getlogin() that might fail when there's no terminal,
> or maybe even lie on some unices if it trusts LOGNAME)?
(user-info :default) is quite robust. if getlogin fails, it calls getuid.
I was not aware that getlogin might ever rely on LOGNAME.
> And shouldn't that init-hook be on by default?
this is an interesting question.
maybe indeed it should be done in C...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> are you seriously considering installing a clisp-based application suid root?!
> wow.
I have run a clisp executable (the "Logiweb demon")
as root for years. Of course it drops its privileges as soon
as it can. There can be good safety reasons for running
suid root. As an example, the application might want to
fence itself into a chroot jail.
But I agree one should not install the clisp stand alone
executable *itself* suid root. One should invoke it from
a script or something where one has control over the
arguments to the clisp executable.
> OK, not a problem: check (ext:argv) from a function in *init-hooks* and remove setuid,
Is that enough? If an init hook can drop privileges
regardless of what the user writes on the command
line, then I suppose it can also (quit) or loop indefinitely
regardless of what the user writes on the command line,
in which case one would be unable to recover the clisp
prompt.
Nevertheless, I think the current solution where one can
always recover the clisp prompt is fine, and those of us
who run clisp as root or suid root must take the necessary
precautions.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>If an init hook can drop privileges
>regardless of what the user writes on the command
>line, then I suppose it can also (quit) or loop indefinitely
>regardless of what the user writes on the command line,
>in which case one would be unable to recover the clisp
>prompt.
I thought about it.
yes, this is a trade off.
However, I do not expect explicit malice from application developers.
If they want to be nasty, they can modify the clisp sources to disable
the "--clisp-" options anyway.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> you can now do:
> # create image:
> $ clisp -x '(saveinitmem "foo" :executable 0
> :init-function (lambda () (print *args*) (quit 0)))'
> # it delegates all arguments to the init-function
> $ ./foo -M a bc d -a -x
> # except for arguments prefixed with --clisp:
> $ ./foo --clisp--help
Great! Thanks. -Klaus
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Other security issue with the mandatory escape: if using such a binary as part of utilities available from a restricted shell setup, you might find that your shell is not as restricted as you thought. Suddenly, whether you compiled your binaries with SBCL or CLISP becomes a security issue.
Once again, I'd really like a bit that's flippable through modifying the binary rather than as a command-line argument. If you care for doing it even without other clisp utilities, the binary could even include the instructions on how to flip the bit when you read it with /usr/bin/strings.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>Why is it bad to be both readable and setuid? Security through obscurity?
yes. http://www.google.com/search?q=security+setuid+script http://www.google.com/search?q=security+setuid+readable
STO is bad when it's the only security.
adding a bit of STO on top of other (valid!) security measures cannot hurt.
IOW, why help the intruder by making the executable readable?
you might not like this logic, but it is valid at least to some people,
so your setuid clisp-based application will likely be installed non-readable
and the "bit flip" method will not work.
>Interestingly, I recently found that setuid perl-scripts have to be readable,
>too, or perl will fail to execute them.
yep
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I don't know if my previous message made it, due to the recent moderation settings change on the list.
I'm trying to publish a new version of cl-launch, and I would like to add support for standalone executables on clisp.
The issue is that (at least up to the clisp 1:2.44-1 included in debian), there is no mechanism to prevent a saved image from processing such options as -M -x -q -norc, etc. The :script t option to saveinitmem doesn't do that at all (as of the above-mentioned clisp at least).
Back in the days, Klaus Grue had a patch to add an option :parse-options to that effect, but as far as I can tell it was rejected for what seems to me was the wrong reason.
https://sourceforge.net/tracker/?func=detail&atid=301355&aid=1446245&group_id=1355
Could said patch be applied, or something equivalent implemented?
(If something like that was done already since that time, please just refer me to the proper documentation.)
Thanks!
I want all downstream users to be able to gain access to the lisp prompt.
I am still unclear how to do that while surrendering all command line argument processing downstream.
1- You could have an optional, unset by default flag surrender the processing downstream when the executable is invoked
2- You could invoke different binary an the image (say with clisp -M) to get a REPL and debug.
1 - what flag? where?
2 - what if the customer does not have that different binary?
1- Klaus Grue's parse-options would be fine.
2- In case no binary is available (not provided, lost, etc.), you could or make it trivial to find where the flag is in the binary, so that anyone can edit it and flip the flag... for instance store the flag as an ASCII character at the end of an easy-to-find ASCII string e.g. "PARSE-OPTIONS+" vs "PARSE-OPTIONS-".
user cannot be expected to be able to edit the binary.
or can he?...
2- The important thing here is that there could be a trivial algorithm to flip the bit, and that anyone with emacs could do it. A string with PARSE-OPTIONS=Y or PARSE-OPTIONS=N prominently at the beginning or end of the binary file, would do wonders (defaulting to Y if nothing is found).
editing binary files is much harder than you seem to think.
even emacs may fail if you have
(add-hook 'write-file-functions 'delete-trailing-whitespace)
in your .emacs.
and chances are that a windows (or aix or ...) user does NOT have emacs installed.
That editing binaries by hand may be difficult is beside the point, since there will be a simple script to edit it for you.
Once again, normal end-user will just use the program as ls, or whatever the programmer wants to deliver as a standalone binary.
Programmers can trivially turn their standalone binary into something an interactive REPL.
You can provide a simple lisp script that will unlock a binary (after proper sanity checks), split it back into executable and image, juggle with 9 balls, etc.
But when I'm implementing my lisp version of ls, I want ls -x * to mean "list entries by line", not "evaluate the first file name as a lisp expression".
If for whatever reason you want to get a REPL from my binary so as to hot patch it, I can use that simple tool to "unlock" the binary. And if I forgot that tool and don't want to download it, I can just use my text editor.
(Or then again, I can rework from the sources from which I built the application, if I have them - like anyone does in any other language.)
the model situation is a person who has your binary for his obscure platform and nothing else.
he wants clisp prompt.
he cannot build (he does not have gcc installed).
he cannot edit (he does not have emacs installed).
using an unlocking script on a different machine is an option...
> the model situation is a person who
> has your binary for his obscure
> platform and nothing else.
> he wants clisp prompt.
> he cannot build (he does not have
> gcc installed).
> he cannot edit (he does not have
> emacs installed).
This is a good point.
I think that if a person has a stand-alone
executable named myprog which is stored
using (saveinitmem ... :parse-options nil)
then the user should be able to counter
the :parse-option using the commandline.
As an example, one could decide that e.g.
giving --clisp-m 1000MB to myprog was
handled like giving -m 1000MB to clisp
And one could decide that e.g.
> myprog --clisp arg1 ... argn
was handled like
> clisp arg1 ... argn
Or one could decide that
> myprog ++ arg1 ... argn
was handled like
> clisp arg1 ... argn
so that ++ was the opposite of --.
My original patch was naive.
At that time I wrote:
> If :parse-options is NIL then all argv
> parameters are passed untouched
> to the program as if they were
> preceeded by an implicit "--".
I think "untouched" should be
"almost untouched" since there
should be particular options which
are still recognized such as
--clisp-m, --clisp, or ++.
One could even insist that --help,
--version, and --license were *always*
processed by clisp, leaving image
specific help to -help-image. Maybe
there should be a -version-image
and -license-image also.
OK, please try cvs head.
you can now do:
# create image:
$ clisp -x '(saveinitmem "foo" :executable 0 :init-function (lambda () (print *args*) (quit 0)))'
# it delegates all arguments to the init-function
$ ./foo -M a bc d -a -x
# except for arguments prefixed with --clisp:
$ ./foo --clisp--help
# which allows you to recover clisp prompt
$ ./foo --clisp-x '(saveinitmem "myclisp" :executable t :init-function (function sys::main-loop))'
# here is your clisp prompt
$ ./myclisp
Looks nice.
On the one hand, I'm not 100% satisfied because who it's still not 100% transparent,
but on the other hand I'm 99.9% satisfied because someone really has to be looking for trouble to detect the discrepancy.
Thanks a whole lot. And .1% grrr!
OK, so I've added support for standalone executables with CLISP 2.48, with the below caveat in the documentation explaining the security hazard of a setuid such standalone executable:
Note that with CLISP, standalone executables will react magically if
invoked with options such as --clisp-help or --clisp-x '(sys::main-loop)'.
That's a pretty far-fetched thing to hit by mistake, and the CLISP
maintainers consider it a feature, but don't let untrusted users control
arguments given to such executables that are run with extra privileges.
Maybe any --clisp-foo option should cause CLISP to drop any setuid privileges?
are you seriously considering installing a clisp-based application suid root?!
wow.
OK, not a problem: check (ext:argv) from a function in *init-hooks* and remove setuid,
see http://clisp.podval.org/impnotes/image.html#image-exec
Well, I would like to eventually turn CL into a systems programming language, and make it reasonably portable with cl-launch as the front-end for casual deployment.
As for your init-hook, needn't I only care about the first argument, instead of doing a find? And shouldn't I be using POSIX:UID (i.e. getuid()) instead of POSIX:USER-INFO (since getlogin() that might fail when there's no terminal, or maybe even lie on some unices if it trusts LOGNAME)?
And shouldn't that init-hook be on by default?
>needn't I only care about the first argument, instead of doing a find?
I think you do need a find.
>And shouldn't I be using POSIX:UID (i.e. getuid()) instead of
do you think (setf (uid) (uid)) will do you much good? :-)
oh, it should be (setf (euid) (uid) (egid) (gid)), right?
> POSIX:USER-INFO (since getlogin() that might fail when there's no terminal,
> or maybe even lie on some unices if it trusts LOGNAME)?
(user-info :default) is quite robust. if getlogin fails, it calls getuid.
I was not aware that getlogin might ever rely on LOGNAME.
> And shouldn't that init-hook be on by default?
this is an interesting question.
maybe indeed it should be done in C...
> are you seriously considering installing a clisp-based application suid root?!
> wow.
I have run a clisp executable (the "Logiweb demon")
as root for years. Of course it drops its privileges as soon
as it can. There can be good safety reasons for running
suid root. As an example, the application might want to
fence itself into a chroot jail.
But I agree one should not install the clisp stand alone
executable *itself* suid root. One should invoke it from
a script or something where one has control over the
arguments to the clisp executable.
> OK, not a problem: check (ext:argv) from a function in *init-hooks* and remove setuid,
Is that enough? If an init hook can drop privileges
regardless of what the user writes on the command
line, then I suppose it can also (quit) or loop indefinitely
regardless of what the user writes on the command line,
in which case one would be unable to recover the clisp
prompt.
Nevertheless, I think the current solution where one can
always recover the clisp prompt is fine, and those of us
who run clisp as root or suid root must take the necessary
precautions.
>If an init hook can drop privileges
>regardless of what the user writes on the command
>line, then I suppose it can also (quit) or loop indefinitely
>regardless of what the user writes on the command line,
>in which case one would be unable to recover the clisp
>prompt.
I thought about it.
yes, this is a trade off.
However, I do not expect explicit malice from application developers.
If they want to be nasty, they can modify the clisp sources to disable
the "--clisp-" options anyway.
> you can now do:
> # create image:
> $ clisp -x '(saveinitmem "foo" :executable 0
> :init-function (lambda () (print *args*) (quit 0)))'
> # it delegates all arguments to the init-function
> $ ./foo -M a bc d -a -x
> # except for arguments prefixed with --clisp:
> $ ./foo --clisp--help
Great! Thanks. -Klaus
Other security issue with the mandatory escape: if using such a binary as part of utilities available from a restricted shell setup, you might find that your shell is not as restricted as you thought. Suddenly, whether you compiled your binaries with SBCL or CLISP becomes a security issue.
Once again, I'd really like a bit that's flippable through modifying the binary rather than as a command-line argument. If you care for doing it even without other clisp utilities, the binary could even include the instructions on how to flip the bit when you read it with /usr/bin/strings.
>I'd really like a bit that's flippable through modifying the binary
it is a VERY bad idea to have an executable both readable and setuid.
Why is it bad to be both readable and setuid? Security through obscurity?
Interestingly, I recently found that setuid perl-scripts have to be readable, too, or perl will fail to execute them.
>Why is it bad to be both readable and setuid? Security through obscurity?
yes.
http://www.google.com/search?q=security+setuid+script
http://www.google.com/search?q=security+setuid+readable
STO is bad when it's the only security.
adding a bit of STO on top of other (valid!) security measures cannot hurt.
IOW, why help the intruder by making the executable readable?
you might not like this logic, but it is valid at least to some people,
so your setuid clisp-based application will likely be installed non-readable
and the "bit flip" method will not work.
>Interestingly, I recently found that setuid perl-scripts have to be readable,
>too, or perl will fail to execute them.
yep