clisp-devel

new command-line option -nextgc-factor

[Bruno H. <br...@cl...>]

Hi all,

For debugging GC problems, I have added a new option -nextgc-factor.
It is a factor that gets applied to the amount of space that can be
consumed before the next GC is triggered.

Setting it to large values can inhibit GC for a long time:

$ ./lisp.run -q -M lispinit.mem -nextgc-factor 1e9
[1]> (time (setq a (make-array 1000000) b nil))
Real time: 0.038371 sec.
Run time: 0.04 sec.
Space: 8000016 Bytes
NIL
[2]> (room)

Number of garbage collections:                0
Bytes freed by GC:                            0
Time spent in GC:                           0.0 sec
Bytes permanently allocated:            158,480
Bytes currently in use:              11,107,792
Bytes available until next GC: 1,536,935,991,966,080
11107792 ;
1536935991966080 ;
158480 ;
0 ;
0 ;
0

Setting it to small values can make GCs much more frequent:

$ ./lisp.run -q -M lispinit.mem -nextgc-factor 0.001
[1]> (room)

Number of garbage collections:               16
Bytes freed by GC:                       44,648
Time spent in GC:                         1.112 sec
Bytes permanently allocated:            158,480
Bytes currently in use:               3,056,200
Bytes available until next GC:               71
3056200 ;
71 ;
158480 ;
16 ;
44648 ;
1112000
[2]> (time (setq a (make-list 1000) b nil))
Real time: 0.687479 sec.
Run time: 0.688 sec.
Space: 16000 Bytes
GC: 10, GC time: 0.688 sec.
NIL


I don't want to document this option, because the default GC frequency
is completely sufficient for the average user.

Bruno

new command-line option -nextgc-factor

[<don...@is...>]

Bruno Haible writes:

 > For debugging GC problems, I have added a new option -nextgc-factor.
 > It is a factor that gets applied to the amount of space that can be
 > consumed before the next GC is triggered.
...
 > I don't want to document this option, because the default GC frequency
 > is completely sufficient for the average user.

You do have a section on debugging (also gc) in impnotes.
Why not include what you wrote in this message there?
And in the doc for command line arguments refer to
the appropriate section of impnotes.

Re: new command-line option -nextgc-factor

[Bruno H. <br...@cl...>]

Hi Don,

>  > I don't want to document this option, because the default GC frequency
>  > is completely sufficient for the average user.
> 
> You do have a section on debugging (also gc) in impnotes.
> Why not include what you wrote in this message there?

Thanks for the suggestion. Done.

> And in the doc for command line arguments refer to
> the appropriate section of impnotes.

No. I don't want to make this command-line option too widely known.

Bruno

new command-line option -nextgc-factor

[<don...@is...>]

I just did hg pull and I don't see this new option.
Am I using the right repository?  Is there a delay?

 [don@localhost clisp]$ hg pull
 pulling from http://hg.code.sf.net/p/clisp/clisp
 searching for changes
 no changes found
 [don@localhost clisp]$ find . -type f -exec fgrep -il nextgc-factor {} \;
 [don@localhost clisp]$  

Re: new command-line option -nextgc-factor

[Bruno H. <br...@cl...>]

Don Cohen wrote:
> I just did hg pull and I don't see this new option.
> Am I using the right repository?  Is there a delay?
> 
>  [don@localhost clisp]$ hg pull

Always use "hg pull --update".

Bruno

[1] https://stackoverflow.com/questions/19270303/why-do-i-need-hg-update-after-hg-pull-while-in-git-im-doing-only-git-pull

Re: new command-line option -nextgc-factor

[<don...@is...>]

Bruno Haible writes:
 > Don Cohen wrote:
 > > I just did hg pull and I don't see this new option.
 > > Am I using the right repository?  Is there a delay?
 > > 
 > >  [don@localhost clisp]$ hg pull
 > 
 > Always use "hg pull --update".

That also did not work for me.
I finally gave up and re cloned in order to get latest updates,
inc. GC fix.

Now the build seems to work:
 unlike last time I have no problem with the new po files
Also make check only shows the two permission denied
errors from streams (the segmentation fault is fixed).

Also no AVCs, this is still with
 selinuxuser_execheap --> off
 selinuxuser_execmod --> off
 selinuxuser_execstack --> off
(the last two changed from default settings)
Also this is with the listener on port 9090 turned off:
 # systemctl status cockpit.socket 
 cockpit.socket - Cockpit Web Service Socket
   Loaded: loaded (/usr/lib/systemd/system/cockpit.socket; enabled; vendor preset: enabled)
   Active: inactive (dead) since Thu 2017-05-25 12:09:22 PDT; 4 days ago
     Docs: man:cockpit-ws(8)
   Listen: [::]:9090 (Stream)
 ...

I'm now assuming you're supposed to build clisp as a non-root user
(I recall getting complaints when building as root), so it seems to 
me that the make check tests ought to assume you're running without
root permissions.
Or possibly accept as a result to certain tests either what you 
should get if you don't have permission or what you should get if
you do.

And perhaps the test that uses port 9090 could try a few other ports
if that one turns out to be in use.

Can you think of any other tests you'd like to try on
this Fedora-Server-dvd-x86_64-25-1.3 VM ?

Re: hg pull

[Bruno H. <br...@cl...>]

>  > Always use "hg pull --update".
> 
> That also did not work for me.
> I finally gave up and re cloned in order to get latest updates,
> inc. GC fix.

I was in a similar situation, after "hgview" was showing that the
'default' branch had two heads. To get rid of the unused unnamed
branch, I also ended up re-cloning.

Bruno

Re: errors on streams

[Bruno H. <br...@cl...>]

Don Cohen wrote:
> Also make check only shows the two permission denied
> errors from streams (the segmentation fault is fixed).
> 
> Also no AVCs, this is still with
>  selinuxuser_execheap --> off
>  selinuxuser_execmod --> off
>  selinuxuser_execstack --> off
> (the last two changed from default settings)
> ...
> I'm now assuming you're supposed to build clisp as a non-root user

Of course you should never build any package as 'root'.
But what are the "permission denied errors from streams" that you mention?

Bruno

Re: errors on streams

[<don...@is...>]

Bruno Haible writes:
 > Of course you should never build any package as 'root'.

An yet there are features that can only be used as root, so if you
want to test them you have to test as root.

 > But what are the "permission denied errors from streams" that you
 > mention?

(WITH-OPEN-FILE (COPY S) (STREAMP COPY))
[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied

(STREAMP (SETQ S (MAKE-STREAM :INPUT)))
EQL-OK: T
(OR (NOT (SEARCH "#P" (PRIN1-TO-STRING S))) (PATHNAMEP (TRUENAME S)))
EQL-OK: T
(HANDLER-CASE (WRITE-LINE "foo" S) (STREAM-ERROR (C) (PRINC-ERROR C) T))
[SIMPLE-STREAM-ERROR]: WRITE-CHAR on #1=#<INPUT UNBUFFERED FILE-STREAM CHARACTER #P"/dev/fd/0" @1> is illegal

EQL-OK: T
(WITH-OPEN-FILE (COPY S) (STREAMP COPY))
[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied

whereas above returns T as root

in this case:
[7]> s
#<INPUT UNBUFFERED FILE-STREAM CHARACTER #P"/dev/fd/0" @1>


(STREAMP (SETQ S (MAKE-STREAM :ERROR)))
EQL-OK: T
(OR (NOT (SEARCH "#P" (PRIN1-TO-STRING S))) (PATHNAMEP (TRUENAME S)))
EQL-OK: T
(WRITE-LINE "foo" S)
foo
EQUAL-OK: "foo"
(LET ((*REOPEN-OPEN-FILE* NIL)) (WITH-OPEN-FILE (COPY S :DIRECTION :OUTPUT) (STREAMP COPY)))
[OS-FILE-ERROR]: OS-FILE-ERROR(13): Permission denied

returns T as root

Re: errors on streams

[Sam S. <sd...@gn...>]

> * Don Cohen <qba...@vf...3-vap.pbz> [2017-05-30 19:35:56 +0000]:
>
> Bruno Haible writes:
>  > Of course you should never build any package as 'root'.
>
> An yet there are features that can only be used as root, so if you
> want to test them you have to test as root.

Indeed.
Still, you never build the package as root.
When you built it, you can _test_ specific features as root though.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://memri.org
http://americancensorship.org https://jihadwatch.org http://no2bds.org
The biggest childhood mistake is thinking that being an adult is fun.

Re: new command-line option -nextgc-factor

[Reini U. <rei...@gm...>]

With "Fix GC bug that affects some stream types (regression from 2004-04-28)"
the first time my linux smokers pass:

https://travis-ci.org/rurban/clisp/builds

windows still has the TRUENAME: Filename for #<OUTPUT UNBUFFERED FILE-STREAM CHARACTER> is unknown
problem:

https://ci.appveyor.com/project/rurban/clisp/history

Reini Urban
ru...@cp...



> On May 28, 2017, at 2:19 PM, Bruno Haible <br...@cl...> wrote:
> 
> Hi all,
> 
> For debugging GC problems, I have added a new option -nextgc-factor.
> It is a factor that gets applied to the amount of space that can be
> consumed before the next GC is triggered.
> 
> Setting it to large values can inhibit GC for a long time:
> 
> $ ./lisp.run -q -M lispinit.mem -nextgc-factor 1e9
> [1]> (time (setq a (make-array 1000000) b nil))
> Real time: 0.038371 sec.
> Run time: 0.04 sec.
> Space: 8000016 Bytes
> NIL
> [2]> (room)
> 
> Number of garbage collections:                0
> Bytes freed by GC:                            0
> Time spent in GC:                           0.0 sec
> Bytes permanently allocated:            158,480
> Bytes currently in use:              11,107,792
> Bytes available until next GC: 1,536,935,991,966,080
> 11107792 ;
> 1536935991966080 ;
> 158480 ;
> 0 ;
> 0 ;
> 0
> 
> Setting it to small values can make GCs much more frequent:
> 
> $ ./lisp.run -q -M lispinit.mem -nextgc-factor 0.001
> [1]> (room)
> 
> Number of garbage collections:               16
> Bytes freed by GC:                       44,648
> Time spent in GC:                         1.112 sec
> Bytes permanently allocated:            158,480
> Bytes currently in use:               3,056,200
> Bytes available until next GC:               71
> 3056200 ;
> 71 ;
> 158480 ;
> 16 ;
> 44648 ;
> 1112000
> [2]> (time (setq a (make-list 1000) b nil))
> Real time: 0.687479 sec.
> Run time: 0.688 sec.
> Space: 16000 Bytes
> GC: 10, GC time: 0.688 sec.
> NIL
> 
> 
> I don't want to document this option, because the default GC frequency
> is completely sufficient for the average user.
> 
> Bruno
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> clisp-devel mailing list
> cli...@li...
> https://lists.sourceforge.net/lists/listinfo/clisp-devel

Re: ci results

[Bruno H. <br...@cl...>]

Hi Reini,

Please routinely set the mail subject. This makes it easier to manage the mails
and to peruse the archives.

> With "Fix GC bug that affects some stream types (regression from 2004-04-28)"
> the first time my linux smokers pass:
> 
> https://travis-ci.org/rurban/clisp/builds

Great! Kudos to Vladimir, again.

Bruno

Re: TRUENAME on Windows

[Bruno H. <br...@cl...>]

Hi Reini,

Please routinely set the mail subject. This makes it easier to manage the mails
and to peruse the archives.

> windows still has the TRUENAME: Filename for #<OUTPUT UNBUFFERED FILE-STREAM CHARACTER> is unknown
> problem:
> 
> https://ci.appveyor.com/project/rurban/clisp/history

Can you describe this issue for Sam (who was the last one to work on TRUENAME
for streams), please?

I personally won't look at the Windows build soon; the various Unix builds
have higher priority for me.

Bruno

Re: TRUENAME on Windows

[Sam S. <sd...@gn...>]

> * Reini Urban <ervav.heona@tznvy.pbz> [2017-05-30 11:35:47 +0200]:
>
> windows still has the TRUENAME: Filename for #<OUTPUT UNBUFFERED
> FILE-STREAM CHARACTER> is unknown
> problem:

I have no access to a windows box (and I cannot figure out how to use
the links you sent, sorry).
Could you please tell me which specific test(s?) fail?

Thanks!

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net https://ffii.org
http://no2bds.org http://iris.org.il http://islamexposedonline.com
Life sucks, but the alternative is even worse.