clisp-devel

Re: configure scripts in mercurial

[Daniel J. <dan...@gm...>]

> Here and now is the right time and place for people to voice their
> opinions - do you like the current system (configure scripts in
> mercurial) or would you prefer the Emacs style (configure scripts in the
> distribution tar ball but regenerated by each developer).

I prefer the Emacs style, too.

This is probably obvious since I somehow started this discussion :)

Re: configure scripts in mercurial

[Sam S. <sd...@gn...>]

> * Sam Steingold <fq...@ta...> [2016-09-01 10:56:21 -0400]:
>
>> *  <Wbr...@g-...> [2016-09-01 11:34:06 +0000]:
>>
>> Somehow, it would be a pain for a user to have to get clisp source
>> from hg but a working configure from inside a downloaded distribution,
>> instead of having everything in the tree.
>
> This is not right. You should regenerate the configure files yourself.
> E.g., when I clone the Emacs git tree, I run "./autogen.sh all" which
> does all the magic.
>
> At any rate, you vote is noted: keep configure in mercurial.

... and, given your level of seniority with clisp, your vote wins. :-)

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://think-israel.org http://dhimmi.org
http://www.dhimmitude.org http://americancensorship.org
Bus error -- driver executed.

Re: rawsock on windows

[Daniel J. <dan...@gm...>]

> all you need to do is "#include <netdb.h>" unconditionally.
> http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00338.html

Hm, I think I was a bit unclear about the issue that I have in
mind. Sure we can unconditionally include netdb.h. But we also have no
way to check whether the functions it declares are actually
available. gnulib doesn't (AFAIK) define (e.g.) HAVE_SETNETENT either,
even if it provides a replacement.

Thus we'd get completely unfunctional RAWSOCK:NETWORK and
RAWSOCK:PROTOCOL functions, which is IMO worse than not binding
functions to these symbols at all. (The later can and is tested with
BOUNDP already)

Or am I missing something here?

Re: rawsock on windows

[Sam S. <sd...@gn...>]

> * Daniel Jour <qna...@tz...> [2016-09-02 00:33:01 +0200]:
>
>> all you need to do is "#include <netdb.h>" unconditionally.
>> http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00338.html
>
> Hm, I think I was a bit unclear about the issue that I have in
> mind. Sure we can unconditionally include netdb.h. But we also have no
> way to check whether the functions it declares are actually
> available. gnulib doesn't (AFAIK) define (e.g.) HAVE_SETNETENT either,
> even if it provides a replacement.
>
> Thus we'd get completely unfunctional RAWSOCK:NETWORK and
> RAWSOCK:PROTOCOL functions, which is IMO worse than not binding
> functions to these symbols at all. (The later can and is tested with
> BOUNDP already)
>
> Or am I missing something here?

If gnulib provides the functions, I expect them to do something useful.
If they are just stubs returning NIL or raising exception on all inputs,
I think you should report that as a bug to gnulib.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://truepeace.org http://americancensorship.org
http://www.memritv.org http://openvotingconsortium.org
If it has syntax, it isn't user friendly.

Re: alloca and stack overflow

[Daniel J. <dan...@gm...>]

Would this be ok? Fixing only the "critical" issue for now. (created
using hg export)

--8<------------------cut here------------------------------>8---
# HG changeset patch
# User Daniel Jour <dan...@gm...>
# Date 1472772999 -7200
#      Fri Sep 02 01:36:39 2016 +0200
# Node ID f663fe7ede9f27807a97f8c44de87d96edc7f9e6
# Parent  bb23fd0915fa5961a6c28f6b9f750ce98c31a560
Avoid stack overflow for large number of sub expressions in regex pattern

diff -r bb23fd0915fa -r f663fe7ede9f modules/regexp/regexi.c
--- a/modules/regexp/regexi.c Tue Aug 30 22:45:08 2016 -0400
+++ b/modules/regexp/regexi.c Fri Sep 02 01:36:39 2016 +0200
@@ -87,6 +87,7 @@
   rettype_t rettype = CHECK_RETTYPE(STACK_2);
   regex_t *re;
   regmatch_t *ret;
+  size_t ret_buffer_size;
   skipSTACK(3);                 /* drop all options */
   for (;;) {
     STACK_1 = check_fpointer(STACK_1,true);
@@ -106,10 +107,16 @@
     funcall(L(make_array),7);
     string = value1;
   }
-  begin_system_call();
-  ret = (regmatch_t*)alloca((re->re_nsub+1)*sizeof(regmatch_t));
-  end_system_call();
-  if (ret == NULL) OS_error();
+  ret_buffer_size = (re->re_nsub+1)*sizeof(regmatch_t);
+  if (ret_buffer_size <= BUFSIZ) {
+    begin_system_call();
+    ret = (regmatch_t*)alloca(ret_buffer_size);
+    end_system_call();
+    if (ret == NULL) OS_error();
+  } else {
+    /* Don't use alloca for sizes > BUFSIZ, it's not safe! */
+    ret = (regmatch_t*)clisp_malloc(ret_buffer_size);
+  }
   with_string_0(string,GLO(misc_encoding),stringz, {
     begin_system_call();
     status = regexec(re,stringz,re->re_nsub+1,ret,eflags);
@@ -142,5 +149,11 @@
       case ret_bool:   VALUES1(T); break;
     }
   }
+  if (ret_buffer_size > BUFSIZ) {
+    /* buffer allocated using malloc, needs to be free'd */
+    begin_system_call();
+    free(ret);
+    end_system_call();
+  }
   skipSTACK(2);                 /* drop pattern & string */
 }
diff -r bb23fd0915fa -r f663fe7ede9f src/ChangeLog
--- a/src/ChangeLog Tue Aug 30 22:45:08 2016 -0400
+++ b/src/ChangeLog Fri Sep 02 01:36:39 2016 +0200
@@ -1,3 +1,8 @@
+2016-09-02  Daniel Jour  <dan...@gm...>
+
+ * modules/regexp/regexi.c (REGEXP-EXEC): Avoid stack overflow for
+ large number of sub expressions
+
 2016-08-29  Sam Steingold  <sd...@gn...>

  * lispbibl.d, built.d, spvw.d, spvw_garcol.d, spvw_garcol_old.d:

Re: alloca and stack overflow

[Sam S. <sd...@gn...>]

> * Daniel Jour <qna...@tz...> [2016-09-02 01:44:46 +0200]:
>
> Would this be ok? Fixing only the "critical" issue for now. (created
> using hg export)

Do you think it would make sense to make a similar change in all
modules?

PS. Please use tabs for indentation in ChangeLog.


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://openvotingconsortium.org
http://ffii.org http://americancensorship.org http://www.memritv.org
Yellow wine is called "white" because it is made out of green grapes.

Re: /bin/pwd

[Daniel J. <dan...@gm...>]

I'd like to come back to this (keeping changes only locally works, but
until now I've found no better solution than to have a patch, apply it
and always revert before making modifications, which is not nice.):

Jan Stary wrote:
> You are suggesting to discard the standard, portable way of doing things
> in order to accommodate one Frankenstein system.
> Not that I have any say in this, but no way.

What's more portable: /bin/pwd (which does not work on my
"Frankenstein system") or pwd (which works on my system, too)?

Or put differently: Is there any system that we know of where my
change actually breaks something?


As a side note:
> How about you locally make a /bin/pwd symlink
> and leave things like they should be?

This might work on my laptop, where I'm root and so on. But I might
also want to push the build (nix allows for that in a very convenient
way) to one random (low load) computer in my universities computer
pool.

> Hm, yet another Frankenstein. If you have a cp(1) that cannot cp
> or a mv(1) that cannot mv, fix _that_ before writing any scripts.

Yes, I fix it by providing my own ... in ~/bin.
One is not always root.


Another note: /bin/pwd will (does) break on a system with GNU Guix,
too. Guix is the (only?) package manager under the GNU umbrella.

Re: gnulib & modules

[Daniel J. <dan...@gm...>]

Oh my, I think I made some false assumptions. You're right, we don't
need separate gnulib checkouts. Neither do we need a shared libgnu.

If we have a single gnulib checkout, and we add in (all) the gnulib
modules needed by "boot CLISP", the base CLISP modules and all other
CLISP modules (that we keep in our repository), and then link ...

1) "base CLISP" ("boot CLISP" + the base modules) against libgnu.a (by
   including libgnu.a in the linking set)

2) every "non base" module against that same libgnu.a (and resolving all
   dependencies, so probably a "relocatable object" produced by ld -r
   will be necessary here)

... then, because we're linking against a static library, the linker
will only pull in exactly the symbols (and code) it needs.

Thus "base CLISP" will not get "fatter" when adding gnulib modules due
to some (non base) CLISP module, and neither will we be including "too
much" in (non base) CLISP modules.

We still have duplicate code for overlapping functionality in "base"
and "non base", though (but I'd guess this won't be avoidable). Oh,
and the base linking set (the directory, not the executable) of course
thus contains the possibly fat libgnu.a (but disk space is cheap,
isn't it?)

And as far as I can see, this is - without the "relocatable object" -
exactly what we've been doing already, right? Or am I on the wrong
track with this?

(If only that insight came earlier, that would've saved me quite some
time and thinking in the last months)


> Even on linux - on LINUX, Carl! - it pulls in ioctl!

Hehe :)

Re: gnulib & modules

[Sam S. <sd...@gn...>]

> * Daniel Jour <qna...@tz...> [2016-09-02 02:30:25 +0200]:
>
> ... then, because we're linking against a static library, the linker
> will only pull in exactly the symbols (and code) it needs.

I think you are forgetting that the default is now dynamic_modules.


-- 
Sam Steingold (http://sds.podval.org/) on Ubuntu 16.04 (xenial) X 11.0.11803000
http://www.childpsy.net/ http://www.dhimmitude.org http://memri.org
http://www.memritv.org http://truepeace.org http://dhimmi.org
A PC without Windows is like ice cream without ketchup.

Re: /bin/pwd

[Jan S. <ha...@st...>]

On Sep 02 02:04:26, dan...@gm... wrote:
> I'd like to come back to this (keeping changes only locally works, but
> until now I've found no better solution than to have a patch, apply it
> and always revert before making modifications, which is not nice.):
> 
> Jan Stary wrote:
> > You are suggesting to discard the standard, portable way of doing things
> > in order to accommodate one Frankenstein system.
> > Not that I have any say in this, but no way.
> 
> What's more portable: /bin/pwd (which does not work on my
> "Frankenstein system") or pwd (which works on my system, too)?

/bin/pwd, because it's the standrad, POSIX-defined tool.
Also, it's an absolute path, not dependent of PATH setting.

Portable means it runs on every standards-compliant system;
it seems that portability for you is "it works on my system".


> Or put differently: Is there any system that we know of where my
> change actually breaks something?

What kind of argument is that? The burden of proof that
your proposed deviation from the standard is good is on you.

It "breaks" on every system where "pwd" is not the POSIX-defined pwd(1)
- for example a shell builtin, implemented by a shell we don't know,
in a way we don't know. Or anywhere the user Prints White Dots (or whatever)
with an unrelated "pwd" binary in his PATH.

I'll repeat: if you want to know the name of the current directory
in a script, why would you call anything else than /bin/pwd [-L-P]?
Please give a better reason than "my broken non-UNIX system doesn't
even have /bin/pwd".

> As a side note:
> > How about you locally make a /bin/pwd symlink
> > and leave things like they should be?
> 
> This might work on my laptop, where I'm root and so on. But I might
> also want to push the build (nix allows for that in a very convenient
> way) to one random (low load) computer in my universities computer
> pool.

Again, you propose accommodating a broken system
as a reason to deviate from the standard. No way.
(Disclaimer: I don't have any say in this.)

The problem you have here is that your system doesn't have /bin/pwd,
not that clisp scripts call /bin/pwd.

> Another note: /bin/pwd will (does) break on a system with GNU Guix,
> too. Guix is the (only?) package manager under the GNU umbrella.

What does "packaging" or GNU have to do with this?

Re: /bin/pwd

[Tomas H. <to...@lo...>]

Jan Stary <ha...@st...> writes:
> /bin/pwd, because it's the standrad, POSIX-defined tool.

Could you please point me to the paragraph in the POSIX standard, which
mandates this?  I am unable to find it.

So far, I have found this
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap10.html
which doesn't even mention the /bin directory.

> Please give a better reason than "my broken non-UNIX system doesn't
> even have /bin/pwd".

This project is called GNU CLISP, where GNU stands for GNU's Not Unix!
Are we talking about UNIX or POSIX?

Some newer OS distributions explore different ways of doing things for
warious reasons and some conventions or rules (often arbitrary to some
extend) written in stone in the 1980s are standing in the way.

> What does "packaging" or GNU have to do with this?

A lot.

I would love to see clisp maintained again.  I hope Daniel succeeds.

Good luck Daniel!

Tomas

Re: gnulib & modules

[Daniel J. <dan...@gm...>]

> I think you are forgetting that the default is now dynamic_modules.

Even then, linking (as in producing) lib-module.so (why the non-standard
naming scheme btw?) could be done by resolving symbols with libgnu.a first.

E.g.:

When foo.o and bar.o are the CLISP module's object files, then

ld -r -o module.o foo.o bar.o libgnu.a

resolves gnulib dependencies from these two object files (and the
dependencies between them) and gives us a module.o that we can easily
convert into a shared library (without having to worry about gnulib
anymore).

Re: gnulib & modules

[Sam S. <sd...@gn...>]

> * Daniel Jour <qna...@tz...> [2016-09-03 01:14:21 +0200]:
>
>> I think you are forgetting that the default is now dynamic_modules.
>
> Even then, linking (as in producing) lib-module.so (why the non-standard
> naming scheme btw?) could be done by resolving symbols with libgnu.a first.

what would the standard naming scheme be?


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://ffii.org http://iris.org.il http://dhimmi.org
http://americancensorship.org http://think-israel.org http://memri.org
If abortion is murder, then oral sex is cannibalism.