clisp-devel — closed list for developers and testers

Re: workarounds around GCC miscompilation

[<Joe...@t-...>]

Hi,

>> There's no configure check for it, it's simply hard-coded.
>Yes. Do you have another suggestion?

>> (I'm not suggesting that configure should check for it, I'm just 
>> saying that it would be useful for packagers to be able to identify work-arounds easily.
>What would you suggest to achieve this?

Just names. Perhaps "HACK:" as a prefix in the one-liner log message would be enough to raise attention. Or TRICK or WA (work-around) or whatever convention.
-> HACK: Fix build failure on Linux/sparc64: Disable optimizations in eval.d
The keyword HACK could be repeated in makemake.in


Likewise, I felt uncomfortable about this log message:
 < Make intparam.c compile with g++ >= 6.
-> Make intparam.c compile with g++ >= 6. The "auto" specifier changed with C++11.
Intparam and g++ 6 are symptoms -- or noise. "auto" is the root cause.

It appears C++ now interprets "auto" as a type specifier, whereas it used to (and still does in C) indicate a storage class (like static or register). Therefore, "auto int i = 0;" is old fashioned code in C, whereas it has become incorrect in C++14 because of the redundancy: either you write e.g. int i =..., or you let the compiler derive the type, and write auto i = ...;

OTOH, my experience is that few people read history. Therefore good descriptions in log messages
are often in vain and arguably lost effort.

Furthermore, somebody encountering this issue is more likely to google the symptoms, e.g.
"intparam.c won't compile with g++" rather than guess the cure, e.g. "auto changed in C++".

Regards,
	Jörg

Re: workarounds around GCC miscompilation

[Bruno H. <br...@cl...>]

Hi Jörg,

> Some summary/stats would be nice at the end of your work: How much was caused
> by individual bugs in GCC, how much by subtly broken definitions in CLISP in
> some configurations.

So far, at least 2/3 of the effort is spent on, like you say, "subtly broken
definitions in clisp", and less than 1/3 on miscompilation by GCC.

> There's no configure check for it, it's simply hard-coded.

Yes. Do you have another suggestion?

> (I'm not suggesting that configure should check for it, I'm just saying that it would be
> useful for packagers to be able to identify work-arounds easily.

What would you suggest to achieve this?

Bruno

Re: support for pre-C99 compilers

[<Joe...@t-...>]

Hi,

>It's more time-consuming to fix the various GC crashes that I'm observing
>on the various porting targets...
Some summary/stats would be nice at the end of your work: How much was caused by individual bugs in GCC, how much by subtly broken definitions in CLISP in some configurations.

BTW, the other day I had a feeling that some of the commit logs should mention in their one liner
summary that they are specific work-around for bugs in (current versions of) GCC.
The summary was overly general, whereas the patch actually and really was a work around for
some particular version of GCC on some particular machine or OS. This annoyed me, since perhaps
next year, GCC may be fixed, however the work-around in CLISP will likely remain for 10 years.

E.g. "Fix build failure on Linux/s390x."
https://sourceforge.net/p/clisp/clisp/ci/c845e414641a5c4805285f38239606c9b1397964/
but what it actually does is compile lisparit.c with -O1 instead of the normal -O2.

There's no configure check for it, it's simply hard-coded.

Same for eval.c with -O0 in some configurations. makemake.in reads:
  if [ $f = eval -a "$cpu" = ia64 -a $XCC_GCC = true ] ; then
    # gcc-2.96 on Linux/ia64 miscompiles eval.d when -O is used.
    flags2=$flags2' -O0'
  fi
gcc-2.xx? Even on my Amiga, I had something newer ;-)

-O0 for eval.d which contains the bytecode interpreter -- that must hurt performance!

(I'm not suggesting that configure should check for it, I'm just saying that it would be
useful for packagers to be able to identify work-arounds easily. Makemake.in counts 4300
lines, and the above is lost within them.)

Regards,
	Jörg

Re: support for pre-C99 compilers

[Sam S. <sd...@gn...>]

> * Bruno Haible <oe...@py...t> [2017-10-20 01:28:20 +0200]:
>
> Sam wrote:
>> > removing 'varbrace' does not have a significant benefit.
>>
>> The huge benefit is that C files will look like normal C.
>
> This would not be a benefit. It would be a drawback.

People are put off by the weird *.d extension (which falsely hints in
the direction of the D language).
People are put off by the gratuitous syntax modifications.
People keep telling you that and you are ignoring them.

> During development, it is important to see all local variable
> declarations, in an easy way.

Yep - just like all the other syntax highlighting.

> With plain C syntax, neither Emacs nor kate can do this.

I have never used kate/pico/nano/axe/...
Emacs & VIM handle C syntax highlighting just fine
(and MultiEdit did 25 years ago).
Even the extra CLISP types are handled with the clisp/emacs/d-mode.el
and clisp/emacs/d.vim extensions (and, before you ask, if we move to
*.c from *.d, it would be _easier_ to handle them).

> Most probably, because the C syntax is a bit ambiguous.

C syntax is trash, but Emacs & VIM handle it just fine.

> Whereas with 'var' syntax, editors can do this (see attached screenshot).

I removed those "var" markers in Emacs and created a screenshot.
Enjoy.

Re: support for pre-C99 compilers

[Bruno H. <br...@cl...>]

Sam wrote:
> > removing 'varbrace' does not have a significant benefit.
> 
> The huge benefit is that C files will look like normal C.

This would not be a benefit. It would be a drawback.

During development, it is important to see all local variable declarations,
in an easy way.

With plain C syntax, neither Emacs nor kate can do this. Most probably, because
the C syntax is a bit ambiguous.

Whereas with 'var' syntax, editors can do this (see attached screenshot).

For this reason, plain C syntax without 'var' is inferior. Even if many people
use it.

> Removing varbrace and comment5 (what does "5" stand for, btw?) is just
> as important for CLISP as making cua-mode the default for Emacs. ;-)

You can remove comment5, if you or someone else converts the comments in the
remeining files (mostly lisparit.d and its include files) in a sensible way -
not in a cheap automated way.

I now prefer to write
  #if 0
   #define foo bar
  #endif
over
  # define foo bar
And comments are well highlighted in today's editors (because the C commment
syntax is unambiguous).

Bruno

Re: support for pre-C99 compilers

[Sam S. <sd...@gn...>]

> * Bruno Haible <oe...@py...t> [2017-10-19 22:08:58 +0200]:
>
> Sam wrote:
>> > Anyway, this pre-C99 support is not a lot of trouble.
>> 
>> This prevents us from dumping varbrace, so it's kinda big deal.
>
> No. Until we can also remove 'gctrigger',

How can that ever happen?!

> removing 'varbrace' does not have a significant benefit.

The huge benefit is that C files will look like normal C.
Removing varbrace and comment5 (what does "5" stand for, btw?) is just
as important for CLISP as making cua-mode the default for Emacs. ;-)

> (We had this discussion already.)

yes, and I disagree with you.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://jij.org
http://www.memritv.org http://americancensorship.org http://think-israel.org
Never argue with an idiot: he has more experience with idiotic arguments.

Re: support for pre-C99 compilers

[Bruno H. <br...@cl...>]

Sam wrote:
> > Anyway, this pre-C99 support is not a lot of trouble.
> 
> This prevents us from dumping varbrace, so it's kinda big deal.

No. Until we can also remove 'gctrigger', removing 'varbrace' does not
have a significant benefit. (We had this discussion already.)

Bruno

Re: support for pre-C99 compilers

[Sam S. <sd...@gn...>]

> * Bruno Haible <oe...@py...t> [2017-10-19 01:12:48 +0200]:
>
>> > Fix build failure with pre-C99 compilers (regression from 2008-10-29).
>> 
>> I thought we decided to require C99?
>
> Yes, a couple of months ago I too thought we would lose no porting targets
> by assuming C99 or better. But I had forgotten about IRIX 6.5, where the
> only available compiler for the 'o32' ABI is cc (at least on the machine
> I have access to), and this cc does not grok statements after declarations.

https://gcc.gnu.org/wiki/CompileFarm does not have any IRIX.
If you give them IRIX, you might have gcc on it...

> Anyway, this pre-C99 support is not a lot of trouble.

This prevents us from dumping varbrace, so it's kinda big deal.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net https://ffii.org
http://www.memritv.org http://islamexposedonline.com
Whether pronounced "leenooks" or "line-uks", it's better than Windows.

Re: support for pre-C99 compilers

[Bruno H. <br...@cl...>]

> > Fix build failure with pre-C99 compilers (regression from 2008-10-29).
> 
> I thought we decided to require C99?

Yes, a couple of months ago I too thought we would lose no porting targets
by assuming C99 or better. But I had forgotten about IRIX 6.5, where the
only available compiler for the 'o32' ABI is cc (at least on the machine
I have access to), and this cc does not grok statements after declarations.

Anyway, this pre-C99 support is not a lot of trouble.

It's more time-consuming to fix the various GC crashes that I'm observing
on the various porting targets...

Bruno

Re: support for pre-C99 compilers

[Sam S. <sd...@gn...>]

> * CLISP - an ANSI Common Lisp Mercurial repository <ab...@py...g> [2017-10-18 21:41:25 +0000]:
>
> Fix build failure with pre-C99 compilers (regression from 2008-10-29).
>
> By Bruno Haible on 10/18/2017 18:21
> [**View Changes**](https://sourceforge.net/p/clisp/clisp/ci/77f89d0b1b04cd53d2f1d9bed6bd264809e5a456/)

I thought we decided to require C99?

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://iris.org.il
http://islamexposedonline.com http://thereligionofpeace.com http://jij.org
Sex is a chemical reaction; alcohol is its catalyst.

Re: Warning: Return value of malloc()...

[Bruno H. <br...@cl...>]

Hi Don,

Thanks for the data.

> $ grep 'define MAPPABLE_ADDRESS_RANGE' lispbibl.h
> #define MAPPABLE_ADDRESS_RANGE_END 0x7DFFFFFFFFFFUL
> #define MAPPABLE_ADDRESS_RANGE_START 0x010000000000UL

> Memory dump:
>   0x400000 - 0x6c1fff
>   0x8c1000 - 0x8defff
>   0x8df000 - 0x908fff
>   0x909000 - 0x90efff
>   0x9d1000 - 0x9f2fff
>   0x7f2ffb2e2000 - 0x7f2ffb49afff
>   0x7f2ffb49b000 - 0x7f2ffb699fff

This is as expected. The MAPPABLE_ADDRESS_RANGE is correct.

> #define SINGLEMAP_MEMORY 
> #define SINGLEMAP_MEMORY_STACK 

In this configuration, the memory block fetched through malloc() is actually
not used for anything. I'll fix this warning in a couple of days (by eliminating
the useless call to malloc).

Bruno

Re: Warning: Return value of malloc()...

[<don...@is...>]

Bruno Haible writes:
 > Hello Don,
 > 
 > >   Warning: Return value of malloc() = 7f924a51b010 is not compatible with type code distribution.
 > > The image seems to work, but I suspect that someone will want to change
 > > some code to fix whatever it is that causes that message.
 > 
 > Thanks for the report. Indeed, I'm currently working on this area. Can you
 > please report back:
 > 
 > 1) After
 >    $ make lispbibl.h
 >    what is the output of
 >    $ grep 'define MAPPABLE_ADDRESS_RANGE' lispbibl.h
 >    $ grep MAP_MEMORY lispbibl.h
 >    $ grep CODE_ADDRESSS_RANGE config.status
 > 
 > 2) What is the result of 3 consecutive runs of
 >    $ ./lisp.run -mm

$ cd build-dir/
[2017-10-16 10:47:54 don@number15 ~/hg/clisp/build-dir]
$ make lispbibl.h
( (gcc -m64 -E -I/home/don/hg/clisp/src -I/home/don/hg/clisp/build-dir/gllib -I/home/don/hg/clisp/src/gllib -g -O2 -W -Wswitch -Wcomment -Wpointer-arith -Wreturn-type -Wmissing-declarations -Wimplicit -Wno-sign-compare -Wno-format-nonliteral -Wno-shift-negative-value -fwrapv -fno-strict-aliasing -ggdb -O0 -DDEBUG_OS_ERROR -DDEBUG_SPVW -DDEBUG_BYTECODE -DSAFETY=3 -DENABLE_UNICODE -DDYNAMIC_FFI -DDYNAMIC_MODULES -P lispbibl.c | grep -v "^ *$") ; (gcc -m64 -E -I/home/don/hg/clisp/src -I/home/don/hg/clisp/build-dir/gllib -I/home/don/hg/clisp/src/gllib -g -O2 -W -Wswitch -Wcomment -Wpointer-arith -Wreturn-type -Wmissing-declarations -Wimplicit -Wno-sign-compare -Wno-format-nonliteral -Wno-shift-negative-value -fwrapv -fno-strict-aliasing -ggdb -O0 -DDEBUG_OS_ERROR -DDEBUG_SPVW -DDEBUG_BYTECODE -DSAFETY=3 -DENABLE_UNICODE -DDYNAMIC_FFI -DDYNAMIC_MODULES -P -dM lispbibl.c | sort) ) > lispbibl.h
[2017-10-16 10:48:06 don@number15 ~/hg/clisp/build-dir]
$ grep 'define MAPPABLE_ADDRESS_RANGE' lispbibl.h
#define MAPPABLE_ADDRESS_RANGE_END 0x7DFFFFFFFFFFUL
#define MAPPABLE_ADDRESS_RANGE_START 0x010000000000UL
[2017-10-16 10:49:05 don@number15 ~/hg/clisp/build-dir]
$ grep MAP_MEMORY lispbibl.h
#define MAP_MEMORY_TABLES 
#define SINGLEMAP_MEMORY 
#define SINGLEMAP_MEMORY_STACK 
[2017-10-16 10:49:15 don@number15 ~/hg/clisp/build-dir]
$ grep CODE_ADDRESSS_RANGE config.status
[2017-10-16 10:49:25 don@number15 ~/hg/clisp/build-dir]
$ ./lisp.run -mm
Memory dump:
  0x400000 - 0x6c1fff
  0x8c1000 - 0x8defff
  0x8df000 - 0x908fff
  0x909000 - 0x90efff
  0x9d1000 - 0x9f2fff
  0x7f2ffb2e2000 - 0x7f2ffb49afff
  0x7f2ffb49b000 - 0x7f2ffb699fff
  0x7f2ffb69a000 - 0x7f2ffb69dfff
  0x7f2ffb69e000 - 0x7f2ffb69ffff
  0x7f2ffb6a0000 - 0x7f2ffb6a3fff
  0x7f2ffb6a4000 - 0x7f2ffb6a6fff
  0x7f2ffb6a7000 - 0x7f2ffb8a5fff
  0x7f2ffb8a6000 - 0x7f2ffb8a6fff
  0x7f2ffb8a7000 - 0x7f2ffb8a7fff
  0x7f2ffb8a8000 - 0x7f2ffb8cefff
  0x7f2ffb8cf000 - 0x7f2ffbacefff
  0x7f2ffbacf000 - 0x7f2ffbad2fff
  0x7f2ffbad3000 - 0x7f2ffbad3fff
  0x7f2ffbad4000 - 0x7f2ffbafafff
  0x7f2ffbafb000 - 0x7f2ffbcfafff
  0x7f2ffbcfb000 - 0x7f2ffbcfbfff
  0x7f2ffbcfc000 - 0x7f2ffbcfcfff
  0x7f2ffbcfd000 - 0x7f2ffbd3bfff
  0x7f2ffbd3c000 - 0x7f2ffbf3bfff
  0x7f2ffbf3c000 - 0x7f2ffbf3efff
  0x7f2ffbf3f000 - 0x7f2ffbf44fff
  0x7f2ffbf45000 - 0x7f2ffbf45fff
  0x7f2ffbf46000 - 0x7f2ffbf69fff
  0x7f2ffc13b000 - 0x7f2ffc13efff
  0x7f2ffc168000 - 0x7f2ffc168fff
  0x7f2ffc169000 - 0x7f2ffc169fff
  0x7f2ffc16a000 - 0x7f2ffc16afff
  0x7f2ffc16b000 - 0x7f2ffc16bfff
  0x7ffe8a99b000 - 0x7ffe8a9bbfff
  0x7ffe8a9dd000 - 0x7ffe8a9defff
  0x7ffe8a9df000 - 0x7ffe8a9e0fff
  0xffffffffff600000 - 0xffffffffff600fff
[2017-10-16 10:49:34 don@number15 ~/hg/clisp/build-dir]
$ ./lisp.run -mm
Memory dump:
  0x400000 - 0x6c1fff
  0x8c1000 - 0x8defff
  0x8df000 - 0x908fff
  0x909000 - 0x90efff
  0x21c4000 - 0x21e5fff
  0x7f962a72c000 - 0x7f962a8e4fff
  0x7f962a8e5000 - 0x7f962aae3fff
  0x7f962aae4000 - 0x7f962aae7fff
  0x7f962aae8000 - 0x7f962aae9fff
  0x7f962aaea000 - 0x7f962aaedfff
  0x7f962aaee000 - 0x7f962aaf0fff
  0x7f962aaf1000 - 0x7f962aceffff
  0x7f962acf0000 - 0x7f962acf0fff
  0x7f962acf1000 - 0x7f962acf1fff
  0x7f962acf2000 - 0x7f962ad18fff
  0x7f962ad19000 - 0x7f962af18fff
  0x7f962af19000 - 0x7f962af1cfff
  0x7f962af1d000 - 0x7f962af1dfff
  0x7f962af1e000 - 0x7f962af44fff
  0x7f962af45000 - 0x7f962b144fff
  0x7f962b145000 - 0x7f962b145fff
  0x7f962b146000 - 0x7f962b146fff
  0x7f962b147000 - 0x7f962b185fff
  0x7f962b186000 - 0x7f962b385fff
  0x7f962b386000 - 0x7f962b388fff
  0x7f962b389000 - 0x7f962b38efff
  0x7f962b38f000 - 0x7f962b38ffff
  0x7f962b390000 - 0x7f962b3b3fff
  0x7f962b585000 - 0x7f962b588fff
  0x7f962b5b2000 - 0x7f962b5b2fff
  0x7f962b5b3000 - 0x7f962b5b3fff
  0x7f962b5b4000 - 0x7f962b5b4fff
  0x7f962b5b5000 - 0x7f962b5b5fff
  0x7fffbd3f2000 - 0x7fffbd412fff
  0x7fffbd4ae000 - 0x7fffbd4affff
  0x7fffbd4b0000 - 0x7fffbd4b1fff
  0xffffffffff600000 - 0xffffffffff600fff
[2017-10-16 10:49:37 don@number15 ~/hg/clisp/build-dir]
$ ./lisp.run -mm
Memory dump:
  0x400000 - 0x6c1fff
  0x8c1000 - 0x8defff
  0x8df000 - 0x908fff
  0x909000 - 0x90efff
  0x1213000 - 0x1234fff
  0x7fdc0ac79000 - 0x7fdc0ae31fff
  0x7fdc0ae32000 - 0x7fdc0b030fff
  0x7fdc0b031000 - 0x7fdc0b034fff
  0x7fdc0b035000 - 0x7fdc0b036fff
  0x7fdc0b037000 - 0x7fdc0b03afff
  0x7fdc0b03b000 - 0x7fdc0b03dfff
  0x7fdc0b03e000 - 0x7fdc0b23cfff
  0x7fdc0b23d000 - 0x7fdc0b23dfff
  0x7fdc0b23e000 - 0x7fdc0b23efff
  0x7fdc0b23f000 - 0x7fdc0b265fff
  0x7fdc0b266000 - 0x7fdc0b465fff
  0x7fdc0b466000 - 0x7fdc0b469fff
  0x7fdc0b46a000 - 0x7fdc0b46afff
  0x7fdc0b46b000 - 0x7fdc0b491fff
  0x7fdc0b492000 - 0x7fdc0b691fff
  0x7fdc0b692000 - 0x7fdc0b692fff
  0x7fdc0b693000 - 0x7fdc0b693fff
  0x7fdc0b694000 - 0x7fdc0b6d2fff
  0x7fdc0b6d3000 - 0x7fdc0b8d2fff
  0x7fdc0b8d3000 - 0x7fdc0b8d5fff
  0x7fdc0b8d6000 - 0x7fdc0b8dbfff
  0x7fdc0b8dc000 - 0x7fdc0b8dcfff
  0x7fdc0b8dd000 - 0x7fdc0b900fff
  0x7fdc0bad2000 - 0x7fdc0bad5fff
  0x7fdc0baff000 - 0x7fdc0bafffff
  0x7fdc0bb00000 - 0x7fdc0bb00fff
  0x7fdc0bb01000 - 0x7fdc0bb01fff
  0x7fdc0bb02000 - 0x7fdc0bb02fff
  0x7ffed71e7000 - 0x7ffed7207fff
  0x7ffed72b5000 - 0x7ffed72b6fff
  0x7ffed72b7000 - 0x7ffed72b8fff
  0xffffffffff600000 - 0xffffffffff600fff
[2017-10-16 10:49:40 don@number15 ~/hg/clisp/build-dir]
$ 

Re: Warning: Return value of malloc()...

[Bruno H. <br...@cl...>]

Hello Don,

>   Warning: Return value of malloc() = 7f924a51b010 is not compatible with type code distribution.
> The image seems to work, but I suspect that someone will want to change
> some code to fix whatever it is that causes that message.

Thanks for the report. Indeed, I'm currently working on this area. Can you
please report back:

1) After
   $ make lispbibl.h
   what is the output of
   $ grep 'define MAPPABLE_ADDRESS_RANGE' lispbibl.h
   $ grep MAP_MEMORY lispbibl.h
   $ grep CODE_ADDRESSS_RANGE config.status

2) What is the result of 3 consecutive runs of
   $ ./lisp.run -mm

Generational GC relies on being able to map memory with MAP_FIXED.
In order to do this, one needs knowledge about where to find a free
address range. I've done this for many platforms (lispbibl.d around lines
2100-2500). But with a different Linux kernel version, things may well be
different.

Thanks!

Bruno

Warning: Return value of malloc()...

[<don...@is...>]

I think that this is the consequence of a recent change to
 http://hg.code.sf.net/p/clisp/clisp :
I notice in last night's build the following message whenever
the newly built image starts:
  Warning: Return value of malloc() = 7f924a51b010 is not compatible with type code distribution.
The image seems to work, but I suspect that someone will want to change
some code to fix whatever it is that causes that message.

Interestingly, this does not occur in the MT version.

Here are the two configure lines.
 ./configure CC='gcc -m64' --disable-maintainer-mode --with-debug --with-module=rawsock build-dir

./configure CC='gcc -m64' --with-threads=POSIX_THREADS --disable-maintainer-mode --with-debug --with-module=rawsock build-mt

The system that is doing the build:
$ uname -a
Linux number15.don-eve 4.11.12-100.fc24.x86_64 #1 SMP Fri Jul 21 17:35:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Re: merge conflicts?

[<don...@is...>]

 > As a last resort, you can of course archive your current clisp checkout
 > and create a new one from scratch (hg clone ...).

This is not what I would have guessed.
Since I don't think I'm trying to write at all, I imagined that
this meant there was a conflict in what others were trying to write
and they would have to resolve the conflict before anyone could
proceed.

One of the posts,
https://stackoverflow.com/questions/12690557/mercurial-hg-abort-outstanding-uncommitted-merges
suggested:
 hg update --clean -r tip
and this does seem to solve the problem.
But I have no idea why.  Or why I should have to do that.
Should that be part of my nightly script?
Why shouldn't my previous
 hg pull -u
be enough?

Re: merge conflicts?

[Bruno H. <br...@cl...>]

Don Cohen wrote:
>  > Did you try a Google search for
>  >   "outstanding merge conflicts" site:stackoverflow.com
> 
> I hadn't before, and now that I have I don't see how it helps.

Well, usually stackoverflow.com is an excellent resource for programming
problems other people may already have encountered.

As a last resort, you can of course archive your current clisp checkout
and create a new one from scratch (hg clone ...).

I am not very familiar with 'hg', therefore you cannot get a more intelligent
advice from me than from stackoverflow.

Bruno

Re: merge conflicts?

[<don...@is...>]

Bruno Haible writes:
...
 > >  abort: outstanding merge conflicts
 > 
 > Did you try a Google search for
 >   "outstanding merge conflicts" site:stackoverflow.com

I hadn't before, and now that I have I don't see how it helps.

Re: merge conflicts?

[Bruno H. <br...@cl...>]

Hi Don,

> Yesterday's nightly build transcript showed:
>  Sat Oct 14 03:40:06 PDT 2017
>  pulling from http://hg.code.sf.net/p/clisp/clisp
>  searching for changes
>  adding changesets
>  adding manifests
>  adding file changes
>  added 1 changesets with 2 changes to 2 files
>  abort: outstanding merge conflicts
> 
> I imagined this was a temporary problem and decided to wait
> and see whether it resolved itself on the next try.
> 
> Today's shows:
>  Sun Oct 15 04:09:04 PDT 2017
>  pulling from http://hg.code.sf.net/p/clisp/clisp
>  searching for changes
>  adding changesets
>  adding manifests
>  adding file changes
>  added 5 changesets with 11 changes to 3 files
>  abort: outstanding merge conflicts

Did you try a Google search for
  "outstanding merge conflicts" site:stackoverflow.com

Bruno

merge conflicts?

[<don...@is...>]

Yesterday's nightly build transcript showed:
 Sat Oct 14 03:40:06 PDT 2017
 pulling from http://hg.code.sf.net/p/clisp/clisp
 searching for changes
 adding changesets
 adding manifests
 adding file changes
 added 1 changesets with 2 changes to 2 files
 abort: outstanding merge conflicts

I imagined this was a temporary problem and decided to wait
and see whether it resolved itself on the next try.

Today's shows:
 Sun Oct 15 04:09:04 PDT 2017
 pulling from http://hg.code.sf.net/p/clisp/clisp
 searching for changes
 adding changesets
 adding manifests
 adding file changes
 added 5 changesets with 11 changes to 3 files
 abort: outstanding merge conflicts

So the problem seems if anything to be getting worse.
I mention it to clisp-devel in case it's something that someone
reading this has to do something about and doesn't realize it.
Let me know if I should have reported this earlier or waited
longer (how much?) to report it.

Re: DESCRIBE, impnotes, and https client

[Sam S. <sd...@gn...>]

> * Bruno Haible <oe...@py...t> [2017-09-21 01:15:32 +0200]:
>
> 1) The DESCRIBE function makes accesses to https://clisp.org/.
> 3) The DESCRIBE function makes accesses to the CLHS on the web.

Both are easily overridable by user variables, either at ./configure
time or in ~/.clisp.lisp or on the fly.
They are supposed to be set in config.lisp by the distributor,
who is supposed to be packaging the docs and clisp.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://jij.org
http://camera.org http://iris.org.il https://ffii.org http://no2bds.org
Let us remember that ours is a nation of lawyers and order.

Re: CLHS vs. free CL documentation

[Jean L. <bu...@gn...>]

I understand that users are looking into documentation. I am too. 

There is some free reference but I could not yet find it. It shall be starting point for future even if it cannot be done right now.

It is GNU software, it shall not download the non free stuff. Reading and browsing fine, but no need to ask them to accept the non free license.

On September 21, 2017 5:05:16 PM GMT+03:00, Bruno Haible <br...@cl...> wrote:
>Jean Louis wrote:
>> I think there is free version being made and other
>> sources for documentation.
>> 
>> Maybe something like
>> http://clqr.boundp.org/source.html
>
>This text covers between 5% and 10% of CL. Development stopped 2 years
>ago.
>
>> Something like this maybe:
>>
>http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/doc/standard/ansi/0.html
>
>This text is not a good basis for free documentation. It was taken
>(stolen,
>you might say) from the ANSI standardization without permission by the
>ANSI bodies. Maybe it is not legal to distribute it at all. Later, Kent
>Pitman negotiated the license of what would become the CLHS with ANSI.
>
>> Or like this
>> http://jtra.cz/stuff/lisp/sclr/index.html
>
>This text covers between 1% and 3% of CL.
>
>> It is better putting effort to make a free
>> documentation for Common Lisp and provide such to
>> users.
>> ...
>> GNU software or free software shall not drive the
>> users to non-free documentation in my opinion
>
>I disagree. Of course, it would be better if ANSI CL would have been
>distributed as free documentation; this would have made it possible
>for the clisp documentation to be distributed as a single coherent
>body - rather than as an "implementation notes" part that refers to
>a standard in hundreds of places.
>
>But the fact is:
>  * The unity of Common Lisp (i.e. the similarity of behaviour of
>    Common Lisp implementations, which translates to portability of
>    Lisp code) is caused by the fact that people (especially the
>    users) *can* and *do* look at the CLHS frequently.
>    Therefore you do NOT serve the users if you tell them not to
>    look at the CLHS.
>  * Through the pointers that you have given above, you have shown
>    that creating a free Common Lisp documentation is a multi-
>    person*year effort, that none of the existing projects are
>    going to achieve in the next 10 years.
>
>Bruno

Re: CLHS vs. free CL documentation

[Bruno H. <br...@cl...>]

Jean Louis wrote:
> I think there is free version being made and other
> sources for documentation.
> 
> Maybe something like
> http://clqr.boundp.org/source.html

This text covers between 5% and 10% of CL. Development stopped 2 years ago.

> Something like this maybe:
> http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/doc/standard/ansi/0.html

This text is not a good basis for free documentation. It was taken (stolen,
you might say) from the ANSI standardization without permission by the
ANSI bodies. Maybe it is not legal to distribute it at all. Later, Kent
Pitman negotiated the license of what would become the CLHS with ANSI.

> Or like this
> http://jtra.cz/stuff/lisp/sclr/index.html

This text covers between 1% and 3% of CL.

> It is better putting effort to make a free
> documentation for Common Lisp and provide such to
> users.
> ...
> GNU software or free software shall not drive the
> users to non-free documentation in my opinion

I disagree. Of course, it would be better if ANSI CL would have been
distributed as free documentation; this would have made it possible
for the clisp documentation to be distributed as a single coherent
body - rather than as an "implementation notes" part that refers to
a standard in hundreds of places.

But the fact is:
  * The unity of Common Lisp (i.e. the similarity of behaviour of
    Common Lisp implementations, which translates to portability of
    Lisp code) is caused by the fact that people (especially the
    users) *can* and *do* look at the CLHS frequently.
    Therefore you do NOT serve the users if you tell them not to
    look at the CLHS.
  * Through the pointers that you have given above, you have shown
    that creating a free Common Lisp documentation is a multi-
    person*year effort, that none of the existing projects are
    going to achieve in the next 10 years.

Bruno

Re: DESCRIBE, impnotes, and https client

[Jean L. <bu...@gn...>]

On Thu, Sep 21, 2017 at 01:15:32AM +0200, Bruno Haible wrote:
> We have three problems. Here's what I plan to do about them.
> 
> 1) The DESCRIBE function makes accesses to https://clisp.org/. This
>    has drawbacks:
>    - The user cannot do DESCRIBE when an internet connection is not
>      available.
>    - The documentation that the user accesses does not match the version
>      of clisp she is using.
>    - https access is currently not implemented in clisp.
>    - It violates the user's privacy if other parties can infer by sniffing
>      what documentation she is accessing.
> 
>    Proposed solution:
>    Let clisp use the *installed* documentation (packaged with clisp).
>    => No network accesses in this case.
>    For "make check" to work before clisp is installed, add a command-line
>    option that specifies the location of the
>    doc. (Like option '-N'.)

That is good idea. It also does not need to access
HTML files with http:// as one can simply use the
file:// facility instead.

The local documentation gives references to the
version that user is using.

I also think that using a browser is overburden,
maybe documentation could be made in similar
fashion various Emacs documentation, maybe
something as Texinfo so that specific sections may
be accessed much easier, either through Emacs or
info program.

> 2) https access is currently not implemented in clisp. But the web consists
>    of more and more https.
>    For https, one needs SSL support with all its intricacies (TLS protocol
>    versions, certificate checking etc.).
> 
>    I'm no longer in favour of including something like Drakma in clisp.
>    Instead, I believe we need to distinguish the bottom layer (SSL) and the
>    layer sitting on top of it (https). Given SSL, https can well be done in
>    Lisp. However, the SSL support ought to use a system-wide standard facility
>    and the system-wide certificates (/usr/share/ca-certificates/mozilla/ on
>    some systems) - otherwise revocations won't be handled right. This leaves us
>    with three options:
>      - GNUTLS,
>      - openssl,
>      - CL+SSL, which is based on openssl.
> 
>    I'll go with GNUTLS in the first place, and openssl as fallback (for
>    portability).
> 
>    Since SSL cannot be applied to random streams, pipe streams, etc., but
>    only to socket streams, the result will be a variant of SOCKET-STREAMs
>    in clisp.

With local documentation there is no need for any
HTTP connection, as file:// facility would simply
be enough. It also works with HTML anchors.

> 3) The DESCRIBE function makes accesses to the CLHS on the web. This has 
>    drawbacks:
>    - The user cannot do DESCRIBE when an internet connection is not
>      available.
>    - It violates the user's privacy if other parties can infer by sniffing
>      what documentation she is accessing.
> 
>    Proposed solution:
>    Add a facility through which the user can download and install the
>    complete CLHS on their system once; this requires acknowledging a license.
>    Cf. [1]
> 
> Comments?

GNU software or free software shall not drive the
users to non-free documentation in my opinion, it
shall not drive users to accept non-free license
and download the HTML that is non-free.

I think there is free version being made and other
sources for documentation.

Maybe something like
http://clqr.boundp.org/source.html

Something like this maybe:
http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/doc/standard/ansi/0.html

Or like this
http://jtra.cz/stuff/lisp/sclr/index.html

It is better putting effort to make a free
documentation for Common Lisp and provide such to
users.

Jean 

Re: https client

[<don...@is...>]

Bruno Haible writes:
 > Don Cohen wrote:
 > >  > 2) https access is currently not implemented in clisp. But the web consists
 > >  >    of more and more https.
 > >  >    For https, one needs SSL support with all its intricacies (TLS protocol
 > >  >    versions, certificate checking etc.).
 > > 
 > > I've used stunnel to solve this problem for a long time.
 > > It seems perfectly adequate and also reasonable.
 > 
 > stunnel is a daemon, right? What is the point of having a separate process,
 > for something that most programs do through library code?

Most programs do this?
By using calls to the openssl library?
I have no idea what's involved.  If it's easy, go ahead.

 > Daemons are a hassle: if they are owned by the system, it requires privileges
 > to start them; if not, the functionality breaks when the daemon happens to
 > have crashed.

I'd guess that a user can run stunnel if it only has to allocate user ports.
If you want to listen on port 443 you're going to need the same privileges
to do that in clisp as you would need to start stunnel listening there.

I don't remember off hand ever seeing stunnel crash.

Re: https client

[Bruno H. <br...@cl...>]

Don Cohen wrote:
>  > 2) https access is currently not implemented in clisp. But the web consists
>  >    of more and more https.
>  >    For https, one needs SSL support with all its intricacies (TLS protocol
>  >    versions, certificate checking etc.).
> 
> I've used stunnel to solve this problem for a long time.
> It seems perfectly adequate and also reasonable.

stunnel is a daemon, right? What is the point of having a separate process,
for something that most programs do through library code?

Daemons are a hassle: if they are owned by the system, it requires privileges
to start them; if not, the functionality breaks when the daemon happens to
have crashed.

Bruno