Re: status update

[Daniel J. <dan...@gm...>]

> pwd is also a shell built-in, so full path is necessary.

Hm, I understand why pwd may give (in case of symbolic links) a
different result as a non-builtin pwd. Why do we need the physical
(not logical, i.e. without the symlinks) path, though?

As for /bin/pwd ... this is a bad idea. /bin (the directory) is about
to die (ok, that's a bit drastic, I'm reffering to the "usr merge"
here), and there may be more than one pwd on a single system (this is
the main reason for this change, because it directly affects me. This
could be fixed downstream by the currently few affected distributions,
too. But in the future this might affect every linux distribution that
uses systemd.)

Same goes for /bin/cat: There might be more than one cat, and all of
them might be in a different location than /bin. Thus it's better to
let the user (or the user's configured environment) choose which cat
to use.

Perhaps we could use (if it's still needed then) an autoconf macro to
determine the "correct" cat and pwd? (Using $CAT and $PWD then, so the
user could change these.)



> 15624::::Daniel Jour 2016-06-22 Remove comment5 utility, and any remaining non-C comments
>
> Nope.
> Each line with separate /* .. */ is ugly.
> We need to use either blocks or //.

Hm, indeed ...


> When did // comments go into the C standard?

C99, "ANSI C" (C89/90) didn't have these yet (but I'd bet the major
compilers wouldn't complain when not in -pedantic mode, didn't check
though.)


> Also, I think this should be done together with removing all
> pre-processing and renaming all *.d files to *.c.


This would also extremely simplify the Makefile.am file for
automake. What's the best base (revision) to make such a change on?



> 15620::::Daniel Jour 2016-06-19 regexp: use autotools, gnulib, improve buffer handling
>
> 0! This is 3 patches in one:

Yes, this is too much for a single change. I was working on all three
changes at the same time, which is how this huge commit came into
being (sorry).

> 1. Why are you replacing stack allocation with arbitrary limits?

I don't? I'm switching the buffer size for the error message from
BUFSIZ (which is an unrelated constant that could be easily too much
for the stack) to another constant SAFE_ERROR_MESSAGE_LENGTH which we
can define to a known "good" value (128 is a first start .. if there's
an error message that's longer we can expand this, and consistently
test whether that's too much on the stack or not).

The other change is more important: With a carefully crafted regex
expression one can currently crash (or potentially worse) CLISP,
because the buffer for the matches can overflow the stack. I think
this might be an issue for e.g. web applications that pass a regex
expression such that a - potentially malicious - user can modify
it. As application developer I wouldn't expect an regex of some (arbitrary,
stack frame dependent) length to be able to crush my whole application.

Therefore I changed it so that a small buffer is still allocated on
the stack (because of the speed benefit), but larger buffers get
allocated using dynamic memory allocation (which, in case of a
failure, raises a condition).

It's not waterproof though: A large number of matches (from a
carefully crafted expression) could still cause the LISP stack to
overflow. I don't know how to prevent that, though (except for
enforcing some maximum).


> 2. Is it really necessary to create a separate m4 directory for the
>   single file gnulib-cache.m4? It's really ugly! (same for the
>   rawsock change below).

Yes, that's the directory where all the gnulib (m4 macros) should
reside in, too. Running gnulib-tool --update in the regexp module
directory populates that directory (as well as lib/).

I think the current state (in which I commited this) is unfortunate,
though: The gnulib files (in m4/ and lib/) should be put under (our)
revision control, too. What do you think would be the best approach
here?
(gnulib has some discussion about this very issue:
https://www.gnu.org/software/gnulib/manual/gnulib.html#VCS-Issues)



> 15625:::tip:Daniel Jour 2016-08-23 rawsock: autotools build system, own gnulib checkout
>
> Copying code from socked.d into rawsock.c is no good.

Yes, this is a (dirty) work around: I was not able to update the
gnulib code for core CLISP (thus socket.d).  I was planing to revert
that as soon as core CLISP also uses updated gnulib code.

I'm a bit worried about these dependencies (between the modules -
rawsock needing OS - and the rawsock module and the socket code from
core CLISP) though. Though this is IMO not relevant for now.


> Also, you removed the configure script, so now people have to install
> autoconf to build clisp.
> Are you sure this is right?

I removed it from the repository because it is a generated file
now. It would be part of a source distribution (a release) though,
thus only CLISP developers need to have autoconf/automake/etc.


> DIUC that the main change required to make rawsock work on windows
> was the switch from rawsock_t to int?

Basically, yes. gnulib is (or in part, will be) handling the windows
specific code. rawsock is interfacing to POSIX/BSD sockets, gnulib is
implementing them on windows.

But I noticed that this commit still contains (printf) debugging and
other oddities. I'll fix that ASAP.



> what does "wip" in "wip-autotools-export" stand for?

work in progress. This is the state of switching to autotools at the
end of the coding period. It's not finished or useable yet. Which is
why ...

> the change to built.d seems incomplete.

... a lot of the changes in that commit are indeed still incomplete.


> I need to understand what you are doing before importing anything.
> The best way would be if you made an isolated change
> (e.g., I don't see why you had to touch built.d).

I'll try to isolate the changes needed to get an autotools based build
system from integrating the configuration into autoconf macros. This
should reduce the size of each change. I don't know how fast I'm able
to provide this atm, though.



> please take a look at clisp/modules/berkeley-db/bdb.c:bdb_handle()
> for getting the regex_t pointer out of the struct.

Hm .. is this faster than the approach that I took? It saves a level
of indirection ("one pointer"), right? Or is there something else
wrong with the approach I took in regexp?