clisp-devel — closed list for developers and testers

Re: BEOS/HAIKU support

[Daniel J. <dan...@gm...>]

> Should we remove BEOS/HAIKU support?

I think we should. In case someday someone comes around with patches
for HAIKU we can reconsider, but currently that's just ballast.

2016-09-04 20:48 GMT+02:00 Jan Stary <ha...@st...>:
> On Sep 04 13:33:59, sd...@gn... wrote:
>> Should we remove BEOS/HAIKU support?
>
> Please do.
>
>         Jan
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> clisp-devel mailing list
> cli...@li...
> https://lists.sourceforge.net/lists/listinfo/clisp-devel

Re: alloca and stack overflow

[Sam S. <sd...@gn...>]

> * Daniel Jour <qnavry.bregjvt@tznvy.pbz> [2016-09-02 01:44:46 +0200]:
>
> Would this be ok? Fixing only the "critical" issue for now. (created
> using hg export)

Do you think it would make sense to make a similar change in all
modules?

PS. Please use tabs for indentation in ChangeLog.


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://openvotingconsortium.org
http://ffii.org http://americancensorship.org http://www.memritv.org
Yellow wine is called "white" because it is made out of green grapes.

Re: BEOS/HAIKU support

[Jan S. <ha...@st...>]

On Sep 04 13:33:59, sd...@gn... wrote:
> Should we remove BEOS/HAIKU support?

Please do.

	Jan

BEOS/HAIKU support

[Sam S. <sd...@gn...>]

Should we remove BEOS/HAIKU support?

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://dhimmi.org http://memri.org http://iris.org.il
http://think-israel.org http://mideasttruth.com http://truepeace.org
I just forgot my whole philosophy of life!!!

Re: gnulib & modules

[Sam S. <sd...@gn...>]

> * Daniel Jour <qnavry.bregjvt@tznvy.pbz> [2016-09-03 01:14:21 +0200]:
>
>> I think you are forgetting that the default is now dynamic_modules.
>
> Even then, linking (as in producing) lib-module.so (why the non-standard
> naming scheme btw?) could be done by resolving symbols with libgnu.a first.

what would the standard naming scheme be?


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://ffii.org http://iris.org.il http://dhimmi.org
http://americancensorship.org http://think-israel.org http://memri.org
If abortion is murder, then oral sex is cannibalism.

Re: gnulib & modules

[Daniel J. <dan...@gm...>]

> I think you are forgetting that the default is now dynamic_modules.

Even then, linking (as in producing) lib-module.so (why the non-standard
naming scheme btw?) could be done by resolving symbols with libgnu.a first.

E.g.:

When foo.o and bar.o are the CLISP module's object files, then

ld -r -o module.o foo.o bar.o libgnu.a

resolves gnulib dependencies from these two object files (and the
dependencies between them) and gives us a module.o that we can easily
convert into a shared library (without having to worry about gnulib
anymore).

Re: BUFSIZ

[Daniel J. <dan...@gm...>]

> BUFSIZ is usually page size (4kB).

Ok, so since AFAIK we're not expecting to have deeply nested stack
frames (> 1000) this shouldn't be an issue then.

The downside of such large buffers is that this might screw up a CPU
cache line, but I'd say this would be a very speculative reason for
premature optimization :)


> Unless there is a commonly used clearly correct alternative approach,
> the "char buffer[BUFSIZ];" is going to stay with us,
> If it will ever be replaced, it will be a pervasive change throughout
> the CLISP sources, not just the regexp module.

The commonly used clearly correct alternative is to use buffers with
the (situation dependend) maximum size of the expected strings.
Though, as you pointed out, in case it's changed it should be changed
throughout the sources, and that's a lot of effort for a questionable gain.

Re: /bin/pwd

[Tomas H. <to...@lo...>]

Jan Stary <ha...@st...> writes:
> /bin/pwd, because it's the standrad, POSIX-defined tool.

Could you please point me to the paragraph in the POSIX standard, which
mandates this?  I am unable to find it.

So far, I have found this
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap10.html
which doesn't even mention the /bin directory.

> Please give a better reason than "my broken non-UNIX system doesn't
> even have /bin/pwd".

This project is called GNU CLISP, where GNU stands for GNU's Not Unix!
Are we talking about UNIX or POSIX?

Some newer OS distributions explore different ways of doing things for
warious reasons and some conventions or rules (often arbitrary to some
extend) written in stone in the 1980s are standing in the way.

> What does "packaging" or GNU have to do with this?

A lot.

I would love to see clisp maintained again.  I hope Daniel succeeds.

Good luck Daniel!

Tomas

Re: /bin/pwd

[Jan S. <ha...@st...>]

On Sep 02 02:04:26, dan...@gm... wrote:
> I'd like to come back to this (keeping changes only locally works, but
> until now I've found no better solution than to have a patch, apply it
> and always revert before making modifications, which is not nice.):
> 
> Jan Stary wrote:
> > You are suggesting to discard the standard, portable way of doing things
> > in order to accommodate one Frankenstein system.
> > Not that I have any say in this, but no way.
> 
> What's more portable: /bin/pwd (which does not work on my
> "Frankenstein system") or pwd (which works on my system, too)?

/bin/pwd, because it's the standrad, POSIX-defined tool.
Also, it's an absolute path, not dependent of PATH setting.

Portable means it runs on every standards-compliant system;
it seems that portability for you is "it works on my system".


> Or put differently: Is there any system that we know of where my
> change actually breaks something?

What kind of argument is that? The burden of proof that
your proposed deviation from the standard is good is on you.

It "breaks" on every system where "pwd" is not the POSIX-defined pwd(1)
- for example a shell builtin, implemented by a shell we don't know,
in a way we don't know. Or anywhere the user Prints White Dots (or whatever)
with an unrelated "pwd" binary in his PATH.

I'll repeat: if you want to know the name of the current directory
in a script, why would you call anything else than /bin/pwd [-L-P]?
Please give a better reason than "my broken non-UNIX system doesn't
even have /bin/pwd".

> As a side note:
> > How about you locally make a /bin/pwd symlink
> > and leave things like they should be?
> 
> This might work on my laptop, where I'm root and so on. But I might
> also want to push the build (nix allows for that in a very convenient
> way) to one random (low load) computer in my universities computer
> pool.

Again, you propose accommodating a broken system
as a reason to deviate from the standard. No way.
(Disclaimer: I don't have any say in this.)

The problem you have here is that your system doesn't have /bin/pwd,
not that clisp scripts call /bin/pwd.

> Another note: /bin/pwd will (does) break on a system with GNU Guix,
> too. Guix is the (only?) package manager under the GNU umbrella.

What does "packaging" or GNU have to do with this?

Re: gnulib & modules

[Sam S. <sd...@gn...>]

> * Daniel Jour <qnavry.bregjvt@tznvy.pbz> [2016-09-02 02:30:25 +0200]:
>
> ... then, because we're linking against a static library, the linker
> will only pull in exactly the symbols (and code) it needs.

I think you are forgetting that the default is now dynamic_modules.


-- 
Sam Steingold (http://sds.podval.org/) on Ubuntu 16.04 (xenial) X 11.0.11803000
http://www.childpsy.net/ http://www.dhimmitude.org http://memri.org
http://www.memritv.org http://truepeace.org http://dhimmi.org
A PC without Windows is like ice cream without ketchup.

Re: gnulib & modules

[Daniel J. <dan...@gm...>]

Oh my, I think I made some false assumptions. You're right, we don't
need separate gnulib checkouts. Neither do we need a shared libgnu.

If we have a single gnulib checkout, and we add in (all) the gnulib
modules needed by "boot CLISP", the base CLISP modules and all other
CLISP modules (that we keep in our repository), and then link ...

1) "base CLISP" ("boot CLISP" + the base modules) against libgnu.a (by
   including libgnu.a in the linking set)

2) every "non base" module against that same libgnu.a (and resolving all
   dependencies, so probably a "relocatable object" produced by ld -r
   will be necessary here)

... then, because we're linking against a static library, the linker
will only pull in exactly the symbols (and code) it needs.

Thus "base CLISP" will not get "fatter" when adding gnulib modules due
to some (non base) CLISP module, and neither will we be including "too
much" in (non base) CLISP modules.

We still have duplicate code for overlapping functionality in "base"
and "non base", though (but I'd guess this won't be avoidable). Oh,
and the base linking set (the directory, not the executable) of course
thus contains the possibly fat libgnu.a (but disk space is cheap,
isn't it?)

And as far as I can see, this is - without the "relocatable object" -
exactly what we've been doing already, right? Or am I on the wrong
track with this?

(If only that insight came earlier, that would've saved me quite some
time and thinking in the last months)


> Even on linux - on LINUX, Carl! - it pulls in ioctl!

Hehe :)

Re: /bin/pwd

[Daniel J. <dan...@gm...>]

I'd like to come back to this (keeping changes only locally works, but
until now I've found no better solution than to have a patch, apply it
and always revert before making modifications, which is not nice.):

Jan Stary wrote:
> You are suggesting to discard the standard, portable way of doing things
> in order to accommodate one Frankenstein system.
> Not that I have any say in this, but no way.

What's more portable: /bin/pwd (which does not work on my
"Frankenstein system") or pwd (which works on my system, too)?

Or put differently: Is there any system that we know of where my
change actually breaks something?


As a side note:
> How about you locally make a /bin/pwd symlink
> and leave things like they should be?

This might work on my laptop, where I'm root and so on. But I might
also want to push the build (nix allows for that in a very convenient
way) to one random (low load) computer in my universities computer
pool.

> Hm, yet another Frankenstein. If you have a cp(1) that cannot cp
> or a mv(1) that cannot mv, fix _that_ before writing any scripts.

Yes, I fix it by providing my own ... in ~/bin.
One is not always root.


Another note: /bin/pwd will (does) break on a system with GNU Guix,
too. Guix is the (only?) package manager under the GNU umbrella.

Re: alloca and stack overflow

[Daniel J. <dan...@gm...>]

Would this be ok? Fixing only the "critical" issue for now. (created
using hg export)

--8<------------------cut here------------------------------>8---
# HG changeset patch
# User Daniel Jour <dan...@gm...>
# Date 1472772999 -7200
#      Fri Sep 02 01:36:39 2016 +0200
# Node ID f663fe7ede9f27807a97f8c44de87d96edc7f9e6
# Parent  bb23fd0915fa5961a6c28f6b9f750ce98c31a560
Avoid stack overflow for large number of sub expressions in regex pattern

diff -r bb23fd0915fa -r f663fe7ede9f modules/regexp/regexi.c
--- a/modules/regexp/regexi.c Tue Aug 30 22:45:08 2016 -0400
+++ b/modules/regexp/regexi.c Fri Sep 02 01:36:39 2016 +0200
@@ -87,6 +87,7 @@
   rettype_t rettype = CHECK_RETTYPE(STACK_2);
   regex_t *re;
   regmatch_t *ret;
+  size_t ret_buffer_size;
   skipSTACK(3);                 /* drop all options */
   for (;;) {
     STACK_1 = check_fpointer(STACK_1,true);
@@ -106,10 +107,16 @@
     funcall(L(make_array),7);
     string = value1;
   }
-  begin_system_call();
-  ret = (regmatch_t*)alloca((re->re_nsub+1)*sizeof(regmatch_t));
-  end_system_call();
-  if (ret == NULL) OS_error();
+  ret_buffer_size = (re->re_nsub+1)*sizeof(regmatch_t);
+  if (ret_buffer_size <= BUFSIZ) {
+    begin_system_call();
+    ret = (regmatch_t*)alloca(ret_buffer_size);
+    end_system_call();
+    if (ret == NULL) OS_error();
+  } else {
+    /* Don't use alloca for sizes > BUFSIZ, it's not safe! */
+    ret = (regmatch_t*)clisp_malloc(ret_buffer_size);
+  }
   with_string_0(string,GLO(misc_encoding),stringz, {
     begin_system_call();
     status = regexec(re,stringz,re->re_nsub+1,ret,eflags);
@@ -142,5 +149,11 @@
       case ret_bool:   VALUES1(T); break;
     }
   }
+  if (ret_buffer_size > BUFSIZ) {
+    /* buffer allocated using malloc, needs to be free'd */
+    begin_system_call();
+    free(ret);
+    end_system_call();
+  }
   skipSTACK(2);                 /* drop pattern & string */
 }
diff -r bb23fd0915fa -r f663fe7ede9f src/ChangeLog
--- a/src/ChangeLog Tue Aug 30 22:45:08 2016 -0400
+++ b/src/ChangeLog Fri Sep 02 01:36:39 2016 +0200
@@ -1,3 +1,8 @@
+2016-09-02  Daniel Jour  <dan...@gm...>
+
+ * modules/regexp/regexi.c (REGEXP-EXEC): Avoid stack overflow for
+ large number of sub expressions
+
 2016-08-29  Sam Steingold  <sd...@gn...>

  * lispbibl.d, built.d, spvw.d, spvw_garcol.d, spvw_garcol_old.d:

Re: configure scripts in mercurial

[Sam S. <sd...@gn...>]

> * Sam Steingold <fq...@ta...t> [2016-09-01 10:56:21 -0400]:
>
>> *  <Wbret-Plevy.Ubruyr@g-flfgrzf.pbz> [2016-09-01 11:34:06 +0000]:
>>
>> Somehow, it would be a pain for a user to have to get clisp source
>> from hg but a working configure from inside a downloaded distribution,
>> instead of having everything in the tree.
>
> This is not right. You should regenerate the configure files yourself.
> E.g., when I clone the Emacs git tree, I run "./autogen.sh all" which
> does all the magic.
>
> At any rate, you vote is noted: keep configure in mercurial.

... and, given your level of seniority with clisp, your vote wins. :-)

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://think-israel.org http://dhimmi.org
http://www.dhimmitude.org http://americancensorship.org
Bus error -- driver executed.

Re: rawsock on windows

[Sam S. <sd...@gn...>]

> * Daniel Jour <qnavry.bregjvt@tznvy.pbz> [2016-09-02 00:33:01 +0200]:
>
>> all you need to do is "#include <netdb.h>" unconditionally.
>> http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00338.html
>
> Hm, I think I was a bit unclear about the issue that I have in
> mind. Sure we can unconditionally include netdb.h. But we also have no
> way to check whether the functions it declares are actually
> available. gnulib doesn't (AFAIK) define (e.g.) HAVE_SETNETENT either,
> even if it provides a replacement.
>
> Thus we'd get completely unfunctional RAWSOCK:NETWORK and
> RAWSOCK:PROTOCOL functions, which is IMO worse than not binding
> functions to these symbols at all. (The later can and is tested with
> BOUNDP already)
>
> Or am I missing something here?

If gnulib provides the functions, I expect them to do something useful.
If they are just stubs returning NIL or raising exception on all inputs,
I think you should report that as a bug to gnulib.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://truepeace.org http://americancensorship.org
http://www.memritv.org http://openvotingconsortium.org
If it has syntax, it isn't user friendly.

Re: rawsock on windows

[Daniel J. <dan...@gm...>]

> all you need to do is "#include <netdb.h>" unconditionally.
> http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00338.html

Hm, I think I was a bit unclear about the issue that I have in
mind. Sure we can unconditionally include netdb.h. But we also have no
way to check whether the functions it declares are actually
available. gnulib doesn't (AFAIK) define (e.g.) HAVE_SETNETENT either,
even if it provides a replacement.

Thus we'd get completely unfunctional RAWSOCK:NETWORK and
RAWSOCK:PROTOCOL functions, which is IMO worse than not binding
functions to these symbols at all. (The later can and is tested with
BOUNDP already)

Or am I missing something here?

Re: configure scripts in mercurial

[Daniel J. <dan...@gm...>]

> Here and now is the right time and place for people to voice their
> opinions - do you like the current system (configure scripts in
> mercurial) or would you prefer the Emacs style (configure scripts in the
> distribution tar ball but regenerated by each developer).

I prefer the Emacs style, too.

This is probably obvious since I somehow started this discussion :)

Re: configure scripts in mercurial

[Jan S. <ha...@st...>]

> Here and now is the right time and place for people to voice their
> opinions - do you like the current system (configure scripts in
> mercurial) or would you prefer the Emacs style (configure scripts in the
> distribution tar ball but regenerated by each developer).

I believe that the best way is to have a simple ./configure script
which is _not_ generated, but written by hand. Such a script is
then in the repository of course. For example http://mdocml.bsd.lv/
does this precisely to avoid dealing with the various versions of auto*.
(It also avoids this question.)
http://mdocml.bsd.lv/cgi-bin/cvsweb/configure

If the ./configure is to be regenerated from configure.in,
then a generated ./configure should be present in every
distribution tarball of course, but not in the repo,
being a generated file.

> > Somehow, it would be a pain for a user to have to get clisp source
> > from hg but a working configure from inside a downloaded distribution,
> > instead of having everything in the tree.
> 
> This is not right. You should regenerate the configure files yourself.

Yes.

	Jan

Re: configure scripts in mercurial

[Ken B. <kb...@co...>]

On 9/1/2016 10:56 AM, Sam Steingold wrote:
> Here and now is the right time and place for people to voice their
> opinions - do you like the current system (configure scripts in
> mercurial) or would you prefer the Emacs style (configure scripts in the
> distribution tar ball but regenerated by each developer).

I prefer the Emacs style.

Ken

Re: configure scripts in mercurial

[Sam S. <sd...@gn...>]

Hi,

> *  <Wbret-Plevy.Ubruyr@g-flfgrzf.pbz> [2016-09-01 11:34:06 +0000]:
>
> Daniel Jour replied to Sam Steingold:
>>> Also, you removed the configure script, so now people have to install 
>>> autoconf to build clisp.
>>> Are you sure this is right?
>>I removed it from the repository because it is a generated file now. It
>>would be part of a source distribution (a release) though, thus only
>>CLISP developers need to have autoconf/automake/etc.
>
> 1. It's good that you plan to keep configure as part of a release
>    distribution. People expect to write ./configure ... && make.

Absolutely!

> I believe some people could be upset if they had to use autotools.
> Probably that is precisely the reason why some open source project
> nevertheless *include* configure within their RCS/SVN/hg/git tree,
> beside configure.in.  The former file is then typically generated by
> one or a team of maintainers with a known (working) configuration, not
> by individual users.  Often enough, the file is only updated at
> release time in hg/git.  For instance, the wine project works exactly
> like this.

And Emacs no longer does.

Getting autotools is trivial on most widely used platforms, e.g., linux,
*bsd (including macosx), windows (cygwin/mingw).

However, we don't have to switch away from our current modus operandi.

Here and now is the right time and place for people to voice their
opinions - do you like the current system (configure scripts in
mercurial) or would you prefer the Emacs style (configure scripts in the
distribution tar ball but regenerated by each developer).

> Somehow, it would be a pain for a user to have to get clisp source
> from hg but a working configure from inside a downloaded distribution,
> instead of having everything in the tree.

This is not right. You should regenerate the configure files yourself.
E.g., when I clone the Emacs git tree, I run "./autogen.sh all" which
does all the magic.

At any rate, you vote is noted: keep configure in mercurial.

Thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://palestinefacts.org http://iris.org.il
http://islamexposedonline.com http://mideasttruth.com http://www.memritv.org
Microsoft wants to monopolize the right to be a monopoly.

Re: status update

[<Joe...@t-...>]

Hi,

Daniel Jour replied to Sam Steingold:
>> Also, you removed the configure script, so now people have to install 
>> autoconf to build clisp.
>> Are you sure this is right?
>I removed it from the repository because it is a generated file now. It
>would be part of a source distribution (a release) though, thus only
>CLISP developers need to have autoconf/automake/etc.

1. It's good that you plan to keep configure as part of a release
   distribution. People expect to write ./configure ... && make.

I believe some people could be upset if they had to use autotools.  I remember
lots of reports (rumours? problems long solved?) in misc. projects about
version mismatch issues coming from autotools, when the developer and the user
have different versions of autotools.  Having the same configure for everybody
nicely solves this issue.

2. Probably that is precisely the reason why some open source project nevertheless
  *include* configure within their RCS/SVN/hg/git tree, beside configure.in.  The
  former file is then typically generated by one or a team of maintainers with a
  known (working) configuration, not by individual users.  Often enough,
  the file is only updated at release time in hg/git.  For instance, the wine
  project works exactly like this.

Somehow, it would be a pain for a user to have to get clisp source from hg but a
working configure from inside a downloaded distribution, instead of having everything
in the tree.

Regards,
	Jörg höhle

Re: rawsock on windows

[Sam S. <sd...@gn...>]

> * Daniel Jour <qnavry.bregjvt@tznvy.pbz> [2016-08-31 13:57:12 +0200]:
>
>> Fine. This means that the change necessary for a release is actually
>> quite small:
>> [snip]
>> Right?
>
> Unfortunately it's not that easy: rawsock fails to build on Windows
> (MinGW) because the gnulib code (from core CLISP) is too old and makes
> a (now) false assumption about the internals of MinGW header files.
>
> Thus, the necessary changes are:
>
> 1) Either update gnulib for core CLISP, or give rawsock its own gnulib
>    (that's what I did and - due to the reasoning above - I'd argue
>    for)
>
> 2) Remove "windows" specific code from rawsock.c: The typedef,
>    including windows headers, the parts with #if defined(WIN32_NATIVE)
>
> At this point it will compile, but has reduced functionality, because
> gnulib has a (IMHO) design flaw: If there's no (for example) netdb.h,
> then the corresponding gnulib module provides one. But it does not
> #define HAVE_NETDB_H, thus our code would not use it (because of the
> #if defined(HAVE_NETDB_H) conditional source parts).

all you need to do is "#include <netdb.h>" unconditionally.
http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00338.html

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://camera.org http://jihadwatch.org
http://memri.org http://americancensorship.org http://www.dhimmitude.org
Why use Windows, when there are Doors?

Re: gnulib & modules

[Sam S. <sd...@gn...>]

> * Daniel Jour <qnavry.bregjvt@tznvy.pbz> [2016-08-31 13:57:12 +0200]:
>
>> This is why we don't run gnulib-tool in that directory!
>> We only ever run it in src.
>
> Hm, I'm afraid that this not a good idea, at least not a scalable
> one. Let me explain: We have modules because we don't want to have
> their code in core CLISP, and we want to be able to (or let the user)
> provide modules to extend CLISP at will (and even at runtime, with
> dynamic loading).

The rule above is only for _base_ modules.
http://clisp.org/impnotes/modules.html#base-modules
These are the modules that are always available to the user.

> And adding a new module should not require changes to core CLISP,
> right? A user should be able to write some module, and let clisp-link
> from the installed CLISP do it's magic.

Yes, other modules (e.g., rawsock & pcre) should have their own gnulib
shared libraries - but not the code.

> Now assume a CLISP module needs (for example) access to some_function,
> but core CLISP does not. The gnulib module the_module provides that
> some_function on systems where it's not available. Should we add
> the_module to core CLISP? I think no, because that would bloat core
> CLISP (and we'd need special linker flags to actually have it in the
> resulting binary). Thus we add it to the CLISP module, and
> everything's fine.

Right.
Note that "core CLISP" is actually the base linkset, not the boot
linkset.

Also, the pernicious nature of gnulib is the dependency creep.
IOW, if you want to use gnulib_module_1, chances are that it will also
pull gnulib_module_2, gnulib_module_3, gnulib_module_4 &c.
And if some of these modules are already present in the base clisp, we
do not want the module to pull it in.

> My point is: gnulib is not designed to be something "shareable" across
> projects (and core CLISP and a module is basically that: two separate
> projects).

This sucks.

> Thus IMO the correct approach at using gnulib is to have a gnulib
> checkout for core CLISP, using only the gnulib modules needed by core
> CLISP. And letting each CLISP module (which wants/needs to use gnulib)
> maintain an own gnulib checkout with only the gnulib modules needed by
> that particular CLISP module.

Indeed this is an easy way.

> This adds some bloat when functionality overlaps (rawsock and
> socket.d, and basic stuff like file IO), but probably only on "gnulib
> intensive" platforms (windows?):
> https://sourceforge.net/p/clisp/bugs/634/

The dependency creep of gnulib means that on any non-glibc platform it
is often a copy of a large part of glibc.
Even on linux - on LINUX, Carl! - it pulls in ioctl!

> Also I think complaining about a missing libgnu.so won't help. That's
> just not the way gnulib is supposed to be used.

This sucks double.

However, I tried to fight this battle 5 years ago and I am not
interested in re-fighting it now.
If every non-base module gets a 1MB libgnu.a, I guess our users will
have to live with it.

Bruno, is this really the way to go?

>> what was the problem [updating gnulib for core CLISP]?
>
> I'm not entierly sure. Makefile.devel tried to update (?) something
> (configure?) for all modules first, but this failed for all
> modules. IMO updating gnulib should - in the end - be no more effort
> than running gnulib-tool --update (in the correct directories, that is
> the top level directory and every module directory that has an own
> gnulib checkout, we could put that into a script or Makefile.devel
> then).

That's what the gnulib-imported target does, more or less.


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://jihadwatch.org http://islamexposedonline.com
http://truepeace.org http://mideasttruth.com http://americancensorship.org
To iterate is human; to recurse, divine.

Re: alloca and stack overflow

[Sam S. <sd...@gn...>]

> * Daniel Jour <qnavry.bregjvt@tznvy.pbz> [2016-08-31 13:57:12 +0200]:
>
>> Let us revisit this issue at a later date.
>> I think the with_string_0 mechanism is good enough.
>> If disagree, you will have to argue for it to be changed pervasively
>> throughout CLISP.
>
> with_string_0 is not involved here. I'm concerned by this (from regexi.c):
>
> begin_system_call();
> ret = (regmatch_t*)alloca((re->re_nsub+1)*sizeof(regmatch_t));
> end_system_call();
>
> re->re_nsub is the number of subexpressions, and if the regex is in
> anyway "modifyable" by a malicious actor (e.g. a POST parameter for a
> search field), then that actor could pass a regex with lots of
> subexpressions, thus causing above alloca to produce a stack overflow
> (in the best case).

I see.
We should handle it the same way we do in
clisp/modules/syscalls/calls.c:CONFSTR:

--8<---------------cut here---------------start------------->8---
#define CS_S(cmd) \
  begin_system_call(); res = confstr(cmd,buf,BUFSIZ); end_system_call(); \
  if (res == 0) value1 = T;                                             \
  else if (res <= BUFSIZ) value1 = asciz_to_string(buf,GLO(misc_encoding)); \
  else {                                                                \
    /* Here we cannot use alloca(), because alloca() is generally unsafe \
       for sizes > BUFSIZ. */                                           \
    char *tmp = (char*)clisp_malloc(res);                               \
    begin_system_call();                                                \
    confstr(cmd,tmp,res);                                               \
    end_system_call();                                                  \
    /* FIXME: asciz_to_string may signal an error in which case tmp leaks */ \
    value1 = asciz_to_string(tmp,GLO(misc_encoding));                   \
    begin_system_call();                                                \
    free(tmp);                                                          \
    end_system_call();                                                  \
  }
--8<---------------cut here---------------end--------------->8---


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://islamexposedonline.com
http://iris.org.il http://thereligionofpeace.com http://camera.org
The dark past once was the bright future.

Re: BUFSIZ

[Sam S. <sd...@gn...>]

> * Daniel Jour <qnavry.bregjvt@tznvy.pbz> [2016-08-31 13:57:12 +0200]:
>
>> BUFSIZ is a pretty standard constant for all string buffers.
>
> Eh, can you support that? It's a standard constant for STREAM
> buffers. This is even defined by the C (C99) standard. Using it for
> anything that's not directly related to a stream thus seems wrong to
> me.
>
> The only thing that might be interpreted as "standard for [..] string
> buffers" is this quote from the glibc documentation:
>
>> Sometimes people also use BUFSIZ as the allocation size of buffers
>> used for related purposes, such as strings used to receive a line of
>> input with fgets (see Character Input). There is no particular
>> reason to use BUFSIZ for this instead of any other integer, except
>> that it might lead to doing I/O in chunks of an efficient size.

yes, this is precisely what I was talking about.

> Though this too does not specify whether BUFSIZ will be small enough
> to be put onto the stack. Moreover it's just in the documentation of a
> single libc, there might be systems that have a huge BUFSIZ but only
> provide limited stack space.

BUFSIZ is usually page size (4kB).

Given the amount of legacy code which does what glibc documentation
talks about, we are in a good company.

This is the same issue as /bin/pwd - if it ain't broke, don't fix it.

Unless there is a commonly used clearly correct alternative approach,
the "char buffer[BUFSIZ];" is going to stay with us,
If it will ever be replaced, it will be a pervasive change throughout
the CLISP sources, not just the regexp module.


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1404
http://www.childpsy.net/ http://memri.org http://truepeace.org
http://honestreporting.com http://openvotingconsortium.org
People with a good taste are especially appreciated by cannibals.