From: Juho S. <js...@ik...> - 2011-11-28 04:29:36
|
Hi, If nothing unexpected happens, I'll be releasing a new SBCL next weekend. Until then, testing and fixes for regressions will be appreciated more than radical code changes :-) -- Juho Snellman |
From: Nikodemus S. <nik...@ra...> - 2011-11-28 09:10:10
|
On 28 November 2011 06:29, Juho Snellman <js...@ik...> wrote: > If nothing unexpected happens, I'll be releasing a new SBCL next weekend. > Until then, testing and fixes for regressions will be appreciated more than > radical code changes :-) What's the sentiment re. updating to ASDF 2.019 during first days of the freeze? Cheers, -- nikodemus |
From: Juho S. <js...@ik...> - 2011-11-28 12:10:31
|
On Mon, Nov 28, 2011 at 10:09 AM, Nikodemus Siivola < nik...@ra...> wrote: > On 28 November 2011 06:29, Juho Snellman <js...@ik...> wrote: > > > If nothing unexpected happens, I'll be releasing a new SBCL next weekend. > > Until then, testing and fixes for regressions will be appreciated more > than > > radical code changes :-) > > What's the sentiment re. updating to ASDF 2.019 during first days of the > freeze? > Fine, but the sooner the better :-) -- Juho Snellman |
From: Nikodemus S. <nik...@ra...> - 2011-11-28 12:32:14
|
On 28 November 2011 14:10, Juho Snellman <js...@ik...> wrote: > Fine, but the sooner the better :-) Done. -- Nikodemus |
From: Eric M. <eri...@fr...> - 2011-11-28 13:19:26
|
>>>>> "js" == Juho Snellman <js...@ik...> writes: js> If nothing unexpected happens, I'll be releasing a new SBCL next weekend. js> Until then, testing and fixes for regressions will be appreciated more than js> radical code changes :-) Hi, I am seeing some gencgc instability on Linux/AMD64: GC invariants lost at lines 1518 and 1520 in gencgc.c while doing random testing this morning. The LDB backtrace (first 20 frames similar in both cases) is below. (SBCL 1.0.53.100-7da051b-dirty, dirtiness is very small and not in the runtime) fatal error encountered in SBCL pid 23926(tid 140737353946880): GC invariant lost, file "gencgc.c", line 1518 Welcome to LDB, a low-level debugger for the Lisp runtime environment. ldb> backtrace Backtrace: 0: Foreign function (null), fp = 0x7ffff705af60, ra = 0x4119da 1: Foreign function (null), fp = 0x7ffff705b050, ra = 0x411b9a 2: Foreign function (null), fp = 0x7ffff705b0a0, ra = 0x42016e 3: Foreign function (null), fp = 0x7ffff705b0d0, ra = 0x40de40 4: Foreign function scavenge, fp = 0x7ffff705b120, ra = 0x40eeee 5: Foreign function (null), fp = 0x7ffff705b140, ra = 0x40f00e 6: Foreign function scavenge, fp = 0x7ffff705b190, ra = 0x40eeee 7: Foreign function (null), fp = 0x7ffff705b1b0, ra = 0x41cf2f 8: Foreign function collect_garbage, fp = 0x7ffff705b270, ra = 0x420a8a 9: SB-KERNEL::COLLECT-GARBAGE 10: (COMMON-LISP::FLET SB-THREAD::WITH-MUTEX-THUNK KEYWORD::IN SB-KERNEL::SUB-GC) 11: (COMMON-LISP::FLET WITHOUT-INTERRUPTS-BODY-88523 KEYWORD::IN SB-THREAD::CALL-WITH-MUTEX) 12: (COMMON-LISP::FLET SB-THREAD::EXEC KEYWORD::IN SB-KERNEL::SUB-GC) 13: (COMMON-LISP::FLET WITHOUT-INTERRUPTS-BODY-243352 KEYWORD::IN SB-KERNEL::SUB-GC) 14: (SB-C::VARARGS-ENTRY SB-KERNEL::SUB-GC) 15: Foreign function call_into_lisp, fp = 0x7ffff705b610, ra = 0x4230df 16: Foreign function maybe_gc, fp = 0x7ffff705b640, ra = 0x411884 17: Foreign function interrupt_handle_pending, fp = 0x7ffff705b670, ra = 0x414639 18: Foreign function (null), fp = 0x7ffff705b6b0, ra = 0x41232b 19: Foreign function (null), fp = 0x7ffff705bc80, ra = 0x7ffff79cc020 20: SB-IMPL::LIST-REMOVE-DUPLICATES* 21: (SB-C::VARARGS-ENTRY COMMON-LISP::REMOVE-DUPLICATES) 22: (COMMON-LISP::LAMBDA (G659) KEYWORD::IN !LATE-TYPE-COLD-INIT) 23: SB-KERNEL::VALUES-SPECIFIER-TYPE 24: SB-KERNEL::SPECIFIER-TYPE 25: SB-C::CAREFUL-SPECIFIER-TYPE 26: SB-C::SOURCE-TRANSFORM-TYPEP 27: SB-C::IR1-CONVERT-SRCTRAN 28: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 29: SB-C::IR1-CONVERT-THE 30: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 31: SB-C::IR1-CONVERT-GLOBAL-FUNCTOID 32: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 33: SB-C::IR1-CONVERT-IF 34: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 35: SB-C::IR1-CONVERT-PROGN-BODY 36: SB-C::IR1-CONVERT-AUX-BINDINGS 37: SB-C::IR1-CONVERT-SPECIAL-BINDINGS 38: (SB-C::VARARGS-ENTRY SB-C::IR1-CONVERT-LAMBDA-BODY) 39: (COMMON-LISP::LAMBDA (SB-C::NEXT SB-C::RESULT SB-C::POST-BINDING-LEXENV) KEYWORD::IN SB-C::IR1-CONVERT-LET) 40: SB-C::%PROCESSING-DECLS 41: SB-C::IR1-CONVERT-LET 42: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 43: SB-C::IR1-CONVERT-GLOBAL-FUNCTOID 44: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 45: SB-C::IR1-CONVERT-PROGN-BODY 46: SB-C::IR1-CONVERT-AUX-BINDINGS 47: SB-C::IR1-CONVERT-SPECIAL-BINDINGS 48: (SB-C::VARARGS-ENTRY SB-C::IR1-CONVERT-LAMBDA-BODY) 49: (COMMON-LISP::LAMBDA (SB-C::NEXT SB-C::RESULT SB-C::POST-BINDING-LEXENV) KEYWORD::IN SB-C::IR1-CONVERT-LET) 50: SB-C::%PROCESSING-DECLS 51: SB-C::IR1-CONVERT-LET 52: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 53: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 54: SB-C::IR1-CONVERT-IF 55: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 56: SB-C::IR1-CONVERT-GLOBAL-FUNCTOID 57: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 58: SB-C::IR1-CONVERT-PROGN-BODY 59: SB-C::IR1-CONVERT-AUX-BINDINGS 60: SB-C::IR1-CONVERT-SPECIAL-BINDINGS 61: (SB-C::VARARGS-ENTRY SB-C::IR1-CONVERT-LAMBDA-BODY) 62: (COMMON-LISP::LAMBDA (SB-C::NEXT SB-C::RESULT SB-C::POST-BINDING-LEXENV) KEYWORD::IN SB-C::IR1-CONVERT-LET) 63: SB-C::%PROCESSING-DECLS 64: SB-C::IR1-CONVERT-LET 65: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 66: SB-C::IR1-CONVERT-GLOBAL-FUNCTOID 67: (SB-C::HAIRY-ARG-PROCESSOR SB-C::IR1-CONVERT) 68: (COMMON-LISP::FLET SB-C::CLOSURE-NEEDING-IR1-ENVIRONMENT-FROM-NODE KEYWORD::IN SB-C::FILTER-LVAR) 69: SB-C::%WITH-IR1-ENVIRONMENT-FROM-NODE 70: SB-C::FILTER-LVAR 71: SB-C::CONVERT-TYPE-CHECK 72: SB-C::GENERATE-TYPE-CHECKS 73: SB-C::IR1-PHASES 74: SB-C::COMPILE-COMPONENT 75: (SB-C::VARARGS-ENTRY SB-C::%COMPILE) 76: (COMMON-LISP::LAMBDA () KEYWORD::IN SB-C::ACTUALLY-COMPILE) 77: (COMMON-LISP::FLET SB-THREAD::WITH-RECURSIVE-LOCK-THUNK KEYWORD::IN SB-C::%WITH-COMPILATION-UNIT) 78: (COMMON-LISP::FLET WITHOUT-INTERRUPTS-BODY-88552 KEYWORD::IN SB-THREAD::CALL-WITH-RECURSIVE-LOCK) 79: SB-THREAD::CALL-WITH-RECURSIVE-LOCK 80: (COMMON-LISP::FLET SB-C::WITH-IT KEYWORD::IN SB-C::%WITH-COMPILATION-UNIT) 81: SB-C::ACTUALLY-COMPILE 82: (SB-C::HAIRY-ARG-PROCESSOR SB-C::COMPILE-IN-LEXENV) 83: (SB-C::HAIRY-ARG-PROCESSOR COMMON-LISP::COMPILE) 84: (SB-C::VARARGS-ENTRY CL-TEST::DO-RANDOM-TYPE-PROP-TESTS) 85: SB-INT::SIMPLE-EVAL-IN-LEXENV 86: COMMON-LISP::EVAL -- Eric Marsden |
From: Nikodemus S. <nik...@ra...> - 2011-11-28 14:12:57
|
On 28 November 2011 15:18, Eric Marsden <eri...@fr...> wrote: > I am seeing some gencgc instability on Linux/AMD64: GC invariants lost > at lines 1518 and 1520 in gencgc.c while doing random testing this > morning. The LDB backtrace (first 20 frames similar in both cases) is below. > (SBCL 1.0.53.100-7da051b-dirty, dirtiness is very small and not in the runtime) Thanks for the report. I'll dig in to this. Happily (?) the likely culprit in a hunk of code I refactored recently, so it's easy to back out if I can't find the bug soonish. How easily can you reproduce this? Cheers, -- Nikodemus |
From: Nikodemus S. <nik...@ra...> - 2011-11-28 14:36:24
|
On 28 November 2011 15:49, Nikodemus Siivola <nik...@ra...> wrote: > Happily (?) the likely culprit in a hunk of code I refactored > recently, so it's easy to back out if I can't find the bug soonish. > > How easily can you reproduce this? Found the bug, I think, though I haven't actually managed to reproduce the failure yet. Cheers, -- nikodemus |
From: Eric M. <eri...@fr...> - 2011-11-28 14:56:44
|
>>>>> "ns" == Nikodemus Siivola <nik...@ra...> writes: ns> Happily (?) the likely culprit in a hunk of code I refactored ns> recently, so it's easy to back out if I can't find the bug soonish. ns> ns> How easily can you reproduce this? It happens after around half an hour of intense CPU activity, not always at the same point. -- Eric Marsden |
From: Nikodemus S. <nik...@ra...> - 2011-11-28 15:16:40
|
On 28 November 2011 16:56, Eric Marsden <eri...@fr...> wrote: > ns> How easily can you reproduce this? > > It happens after around half an hour of intense CPU activity, not > always at the same point. Ok, this sounds appropriately hard for my reading of the code. :) I still haven't managed to trigger this, even though now I think I know what happens. I think there are two issues in there, actually. The proximate cause of the bug is that I moved an loop condition inside the loop and turned it into an assert. I can't honestly say anymore if I did that by accident, or because I believed it should always hold. Re-reading the code, however, it seems to me that it /should/ always hold. However, there also seems to be an old bug that causes us to sort of lose track of a page when large object shrinks so that it ends right on a page boundary -- which I /think/ can cause just the sort of confusion in the page table that that assert will fail... if we later happen to allocate another large object in pages before that leftover one, matching exactly the same starting point as the original object, and the new object must then subsequently shrink so that at least one page at its end is freed. *boggle* I'll make two patches: Patch A will fix the old bug. Patch B will re-instate the asserts as loop conditions. It would be great if you could try out both ones. Cheers, -- Nikodemus |
From: Nikodemus S. <nik...@ra...> - 2011-11-28 16:10:35
|
On 28 November 2011 17:16, Nikodemus Siivola <nik...@ra...> wrote: > I think there are two issues in there, actually. I misread the code, and the "old" bug isn't there after all. So only one patch, reinstating the loop conditions as conditions instead of asserts. Cheers, -- Nikodemus |
From: Martin C. <cra...@co...> - 2011-11-28 16:35:35
|
Juho Snellman wrote on Mon, Nov 28, 2011 at 05:29:29AM +0100: > Hi, > > If nothing unexpected happens, I'll be releasing a new SBCL next weekend. > Until then, testing and fixes for regressions will be appreciated more than > radical code changes :-) The new heap management doesn't quite work for me. I can't make sense of this relation between bytes allocated and max heap size: Saving image, compiler settings ((COMPILATION-SPEED . 1) (DEBUG . 0) (INHIBIT-WARNINGS . 1) (INSERT-ARRAY-BOUNDS-CHECKS . 3) (SAFETY . 0) (SPACE . 1) (SPEED . 3)) Heap exhausted during garbage collection: 48 bytes available, 64 requested. Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB LUB !move Alloc Waste Trig WP GCs Mem-age 0: 0 0 0 0 0 0 0 0 0 0 0 214748364 0 0 0.0000 1: 0 0 0 0 0 0 0 0 0 0 0 214748364 0 0 0.0000 2: 0 0 0 0 0 0 0 0 0 0 0 214748364 0 0 0.0000 3: 0 0 0 0 0 0 0 0 0 0 0 214748364 0 0 0.0000 4: 0 0 0 0 0 0 0 0 0 0 0 214748364 0 0 0.0000 5: 99225 99283 0 0 37916 15016 0 0 0 1714268928 20206848 1884745404 0 1 0.9996 6: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000 Total bytes allocated = 2750723280 Dynamic-space-size bytes = 4294967296 GC control variables: *GC-INHIBIT* = false *GC-PENDING* = false *STOP-FOR-GC-PENDING* = false fatal error encountered in SBCL pid 32226(tid 46912508853632): Heap exhausted, game over. real 0m35.052s user 0m31.073s sys 0m3.699s -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer <cra...@co...> http://www.cons.org/cracauer/ |
From: Jim W. <jw...@dr...> - 2011-11-28 16:36:12
|
Juho Snellman <js...@ik...> writes: > Hi, > > If nothing unexpected happens, I'll be releasing a new SBCL next > weekend. Until then, testing and fixes for regressions will be > appreciated more than radical code changes :-) Current status on Solaris: Solaris x86 still cons'es madly in room.test.sh -- reverting this commit e7b2c507 'more robust backtraces for syscalls on x86' addresses this completely, but I'm not sure of the correct fix for this. Otherwise, tests look good. Solaris/x86_64 has major problems in debug.impure.lisp, however: * test BUG-795245 hangs indefinitely. * If I mark that test to be skipped, I see a heap corruption immediately thereafter -- I'm investigating. * If I mark the next test to be skipped, I see an unhandled SIGILL on the next test The remaining tests, if enabled, all fail, each with a different error -- overall, I suspect something earlier in this set of tests is corrupting the running SBCL. I'm digging further. Thanks, -- Jim Wise jw...@dr... |
From: Harald Hanche-O. <ha...@ma...> - 2011-11-28 18:09:56
|
The test fails on my system (OSX 10.7, threads enabled): ::: Running :PARALLEL-DEFCLASS sleeping! 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111iiiiiiiiiii ... The above line is 27130 characters long. After more than an hour of nothing happening, I interrupted the test by typing ^C, with the following result: unhandled SB-SYS:INTERACTIVE-INTERRUPT in thread #<SB-THREAD:THREAD "initial thread" RUNNING {1002999EC3}>: Interactive interrupt at #x7FFF82357DF2. 0: (SB-DEBUG::MAP-BACKTRACE #<CLOSURE (LAMBDA # :IN BACKTRACE) {10070A7EDB}> :START 0 :COUNT 128) 1: (BACKTRACE 128 #<SYNONYM-STREAM :SYMBOL SB-SYS:*STDERR* {1000171B33}>) 2: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<SB-SYS:INTERACTIVE-INTERRUPT {10070A6403}> #<unavailable argument>) 3: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<SB-SYS:INTERACTIVE-INTERRUPT {10070A6403}>) 4: (INVOKE-DEBUGGER #<SB-SYS:INTERACTIVE-INTERRUPT {10070A6403}>) 5: (SB-INT:%BREAK SB-UNIX:SIGINT #<SB-SYS:INTERACTIVE-INTERRUPT {10070A6403}>) 6: ((FLET SB-UNIX::INTERRUPT-IT :IN SB-UNIX::SIGINT-HANDLER)) 7: ((FLET #:WITHOUT-INTERRUPTS-BODY-243431 :IN SB-THREAD:INTERRUPT-THREAD)) 8: ((FLET #:WITHOUT-INTERRUPTS-BODY-26593 :IN SB-SYS:INVOKE-INTERRUPTION)) 9: ((FLET SB-THREAD::EXEC :IN SB-SYS:INVOKE-INTERRUPTION)) 10: ((FLET #:WITHOUT-INTERRUPTS-BODY-26582 :IN SB-SYS:INVOKE-INTERRUPTION)) 11: (SB-SYS:INVOKE-INTERRUPTION #<CLOSURE (FLET SB-UNIX::INTERRUPTION :IN SB-SYS:ENABLE-INTERRUPT) {11FE81B}>) 12: (SB-SYS:INVOKE-INTERRUPTION #<CLOSURE (FLET SB-UNIX::INTERRUPTION :IN SB-SYS:ENABLE-INTERRUPT) {11FE81B}>)[:EXTERNAL] 13: ((FLET SB-UNIX::RUN-HANDLER :IN SB-SYS:ENABLE-INTERRUPT) 13 #.(SB-SYS:INT-SAP #X011FEDE8) #.(SB-SYS:INT-SAP #X011FEE50)) 14: ("foreign function: call_into_lisp") 15: ("foreign function: funcall3") 16: ("foreign function: interrupt_handle_now") 17: ("foreign function: maybe_now_maybe_later") 18: ("foreign function: _sigtramp") 19: ("bogus stack frame") 20: ((FLET SB-UNIX::SELECT :IN SB-UNIX:UNIX-FAST-SELECT) #.(SB-SYS:INT-SAP #X013FFEE8)) 21: (SB-IMPL::SUB-SUB-SERVE-EVENT 1 0) 22: (SB-IMPL::SUB-SERVE-EVENT 1 0 NIL) 23: (SB-SYS:SERVE-ALL-EVENTS 1) 24: (PROCESS-WAIT #<SB-IMPL::PROCESS 98462 :RUNNING> NIL) 25: ((FLET #:CLEANUP-FUN-1098 :IN RUN-PROGRAM))[:CLEANUP] 26: (RUN-PROGRAM "/local/src/lisp/sbcl/tests/../src/runtime/sbcl" ("/local/src/lisp/sbcl/tests/../src/runtime/sbcl" "--core" "/local/src/lisp/sbcl/tests/../output/sbcl.core" "--noinform" "--no-sysinit" "--no-userinit" "--noprint" "--disable-debugger" "--eval" "(LOAD \"test-util\")" "--eval" "(LOAD \"assertoid\")" "--eval" "(DEFPACKAGE :RUN-TESTS (:USE :CL :TEST-UTIL :SB-EXT))" "--eval" "(IN-PACKAGE :CL-USER)" "--eval" "(USE-PACKAGE :TEST-UTIL)" "--eval" "(USE-PACKAGE :ASSERTOID)" "--eval" "(SETF *BREAK-ON-FAILURE* NIL)" "--eval" "(SETF *BREAK-ON-EXPECTED-FAILURE* NIL)" "--eval" "(LET ((RUN-TESTS::FILE #P\"/local/src/lisp/sbcl/tests/threads.impure.lisp\") (RUN-TESTS::*BREAK-ON-ERROR* NIL)) (FORMAT T \"// Running ~a~%\" RUN-TESTS::FILE) (RESTART-CASE (HANDLER-BIND ((ERROR (LAMBDA (CONDITION) (PUSH (LIST :UNHANDLED-ERROR RUN-TESTS::FILE) *FAILURES*) (COND (RUN-TESTS::*BREAK-ON-ERROR* (REALLY-INVOKE-DEBUGGER CONDITION)) (T (FORMAT *ERROR-OUTPUT* \"~&Unhandled ~a: ~a~%\" (TYPE-OF CONDITION) CONDITION) (BACKTRACE))) (INVOKE-RESTART 'RUN-TESTS::SKIP-FILE)))) (LOAD #P\"/local/src/lisp/sbcl/tests/threads.impure.lisp\")) (RUN-TESTS::SKIP-FILE NIL (FORMAT T \">>>~a<<<~%\" *FAILURES*))) (REPORT-TEST-STATUS) (QUIT :UNIX-STATUS 104))") :ENV NIL :ENVIRONMENT NIL :WAIT T :SEARCH NIL :PTY NIL :INPUT #<SB-SYS:FD-STREAM for "file /dev/null" {1006F642A3}> :IF-INPUT-DOES-NOT-EXIST NIL :OUTPUT #<SB-SYS:FD-STREAM for "standard output" {100299B103}> :IF-OUTPUT-EXISTS :ERROR :ERROR :OUTPUT :IF-ERROR-EXISTS :ERROR :STATUS-HOOK NIL :EXTERNAL-FORMAT :DEFAULT) 27: (RUN-TESTS::RUN-IN-CHILD-SBCL ((LOAD "test-util") (LOAD "assertoid") (DEFPACKAGE :RUN-TESTS (:USE :CL :TEST-UTIL :SB-EXT))) ((IN-PACKAGE :CL-USER) (USE-PACKAGE :TEST-UTIL) (USE-PACKAGE :ASSERTOID) (SETF *BREAK-ON-FAILURE* NIL) (SETF *BREAK-ON-EXPECTED-FAILURE* NIL) (LET ((RUN-TESTS::FILE #P"/local/src/lisp/sbcl/tests/threads.impure.lisp") (RUN-TESTS::*BREAK-ON-ERROR* NIL)) (FORMAT T "// Running ~a~%" RUN-TESTS::FILE) (RESTART-CASE (HANDLER-BIND ((ERROR (LAMBDA (CONDITION) (PUSH (LIST :UNHANDLED-ERROR RUN-TESTS::FILE) *FAILURES*) (COND (RUN-TESTS::*BREAK-ON-ERROR* (REALLY-INVOKE-DEBUGGER CONDITION)) (T (FORMAT *ERROR-OUTPUT* "~&Unhandled ~a: ~a~%" (TYPE-OF CONDITION) CONDITION) (BACKTRACE))) (INVOKE-RESTART 'RUN-TESTS::SKIP-FILE)))) (LOAD #P"/local/src/lisp/sbcl/tests/threads.impure.lisp")) (RUN-TESTS::SKIP-FILE NIL (FORMAT T ">>>~a<<<~%" *FAILURES*))) (REPORT-TEST-STATUS) (QUIT :UNIX-STATUS 104)))) 28: (RUN-TESTS::IMPURE-RUNNER (#P"/local/src/lisp/sbcl/tests/alien.impure.lisp" #P"/local/src/lisp/sbcl/tests/arith.impure.lisp" #P"/local/src/lisp/sbcl/tests/backq.impure.lisp" #P"/local/src/lisp/sbcl/tests/bivalent-stream.impure.lisp" #P"/local/src/lisp/sbcl/tests/break-on-signals.impure.lisp" #P"/local/src/lisp/sbcl/tests/callback.impure.lisp" #P"/local/src/lisp/sbcl/tests/clos-1.impure.lisp" #P"/local/src/lisp/sbcl/tests/clos-add-remove-method.impure.lisp" #P"/local/src/lisp/sbcl/tests/clos-cache.impure.lisp" #P"/local/src/lisp/sbcl/tests/clos-interrupts.impure.lisp" #P"/local/src/lisp/sbcl/tests/clos-typechecking.impure.lisp" #P"/local/src/lisp/sbcl/tests/clos.impure.lisp" #P"/local/src/lisp/sbcl/tests/compare-and-swap.impure.lisp" #P"/local/src/lisp/sbcl/tests/compiler.impure.lisp" #P"/local/src/lisp/sbcl/tests/compound-cons.impure.lisp" #P"/local/src/lisp/sbcl/tests/condition.impure.lisp" #P"/local/src/lisp/sbcl/tests/ctor.impure.lisp" #P"/local/src/lisp/sbcl/tests/deadline.impure.lisp" #P"/local/src/lisp/sbcl/tests/debug.impure.lisp" #P"/local/src/lisp/sbcl/tests/defglobal.impure.lisp" #P"/local/src/lisp/sbcl/tests/define-compiler-macro.impure.lisp" #P"/local/src/lisp/sbcl/tests/defstruct.impure.lisp" #P"/local/src/lisp/sbcl/tests/deftype.impure.lisp" #P"/local/src/lisp/sbcl/tests/destructure.impure.lisp" #P"/local/src/lisp/sbcl/tests/dynamic-extent.impure.lisp" #P"/local/src/lisp/sbcl/tests/enc-cn.impure.lisp" #P"/local/src/lisp/sbcl/tests/enc-jpn.impure.lisp" #P"/local/src/lisp/sbcl/tests/eval.impure.lisp" #P"/local/src/lisp/sbcl/tests/exhaust.impure.lisp" #P"/local/src/lisp/sbcl/tests/external-format.impure.lisp" #P"/local/src/lisp/sbcl/tests/float.impure.lisp" #P"/local/src/lisp/sbcl/tests/foreign-stack-alignment.impure.lisp" #P"/local/src/lisp/sbcl/tests/full-eval.impure.lisp" #P"/local/src/lisp/sbcl/tests/gc.impure.lisp" #P"/local/src/lisp/sbcl/tests/gray-streams.impure.lisp" #P"/local/src/lisp/sbcl/tests/hash.impure.lisp" #P"/local/src/lisp/sbcl/tests/info.impure.lisp" #P"/local/src/lisp/sbcl/tests/interface.impure.lisp" #P"/local/src/lisp/sbcl/tests/kill-non-lisp-thread.impure.lisp" #P"/local/src/lisp/sbcl/tests/load.impure.lisp" #P"/local/src/lisp/sbcl/tests/loop.impure.lisp" #P"/local/src/lisp/sbcl/tests/macroexpand.impure.lisp" #P"/local/src/lisp/sbcl/tests/map-tests.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop-23.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop-24.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop-25.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop-26.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop-27.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop-28.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop-29.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop-30.impure.lisp" #P"/local/src/lisp/sbcl/tests/mop.impure.lisp" #P"/local/src/lisp/sbcl/tests/package-locks.impure.lisp" #P"/local/src/lisp/sbcl/tests/packages.impure.lisp" #P"/local/src/lisp/sbcl/tests/pathnames.impure.lisp" #P"/local/src/lisp/sbcl/tests/pprint.impure.lisp" #P"/local/src/lisp/sbcl/tests/print.impure.lisp" #P"/local/src/lisp/sbcl/tests/profile.impure.lisp" #P"/local/src/lisp/sbcl/tests/properties.impure.lisp" #P"/local/src/lisp/sbcl/tests/reader.impure.lisp" #P"/local/src/lisp/sbcl/tests/run-program.impure.lisp" #P"/local/src/lisp/sbcl/tests/seq.impure.lisp" #P"/local/src/lisp/sbcl/tests/setf.impure.lisp" #P"/local/src/lisp/sbcl/tests/signals.impure.lisp" #P"/local/src/lisp/sbcl/tests/smoke.impure.lisp" #P"/local/src/lisp/sbcl/tests/static-alloc.impure.lisp" #P"/local/src/lisp/sbcl/tests/step.impure.lisp" #P"/local/src/lisp/sbcl/tests/stream.impure.lisp" #P"/local/src/lisp/sbcl/tests/swap-lispobjs.impure.lisp" #P"/local/src/lisp/sbcl/tests/symbol.impure.lisp" #P"/local/src/lisp/sbcl/tests/threads.impure.lisp" #P"/local/src/lisp/sbcl/tests/timer.impure.lisp" #P"/local/src/lisp/sbcl/tests/type.impure.lisp" #P"/local/src/lisp/sbcl/tests/unwind-to-frame-and-call.impure.lisp" #P"/local/src/lisp/sbcl/tests/vector.impure.lisp" #P"/local/src/lisp/sbcl/tests/walk.impure.lisp" #P"/local/src/lisp/sbcl/tests/win32-foreign-stack-unwind.impure.lisp" #P"/local/src/lisp/sbcl/tests/with-compilation-unit.impure.lisp") #<FUNCTION RUN-TESTS::LOAD-TEST>) 29: (RUN-TESTS::RUN-ALL) 30: (SB-INT:SIMPLE-EVAL-IN-LEXENV (RUN-TESTS::RUN-ALL) #<NULL-LEXENV>) 31: (EVAL (RUN-TESTS::RUN-ALL)) 32: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:EVAL . "(with-compilation-unit () (load \"run-tests.lisp\"))") (:EVAL . "(run-tests::run-all)"))) 33: (SB-IMPL::TOPLEVEL-INIT) 34: ((FLET #:WITHOUT-INTERRUPTS-BODY-241595 :IN SAVE-LISP-AND-DIE)) 35: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE)) unhandled condition in --disable-debugger mode, quitting test failed, expected 104 return code, got 1 - Harald |
From: Harald Hanche-O. <ha...@ma...> - 2011-11-28 18:16:14
|
[Harald Hanche-Olsen <ha...@ma...> (2011-11-28 18:09:46 UTC)] > After more than an hour of > nothing happening, I interrupted the test by typing ^C, with the > following result: [...] Uh, that backtrace was clearly from the parent lisp, and hence worthless. The child lisp kept going in the background. I sent it a SIGSTOP for now, but suspect there is no more useful information to have from it? I can tell it has several threads, though. - Harald |
From: Nikodemus S. <nik...@ra...> - 2011-11-28 18:20:17
|
On 28 November 2011 20:09, Harald Hanche-Olsen <ha...@ma...> wrote: > The test fails on my system (OSX 10.7, threads enabled): Occasional hangs on OS X when threads are enabled is a known long-standing issue. When running the tests on OS X I typically see a hang in maybe 1 out of 5-10 runs. If this happens to you consistently, that's another matter. > The above line is 27130 characters long. After more than an hour of > nothing happening, I interrupted the test by typing ^C, with the > following result: FWIW, if tests seem to hang longer a few of minutes, they really are hung. (Assuming modern hardware.) After such a hang on OS X you can also easily be left with a stray child process madly spinning, so ps | grep sbcl, and a judicious bit of kill -9 may be in order. Cheers, -- nikodemus |
From: Harald Hanche-O. <ha...@ma...> - 2011-11-28 18:55:20
|
[Nikodemus Siivola <nik...@ra...> (2011-11-28 18:20:11 UTC)] > Occasional hangs on OS X when threads are enabled is a known > long-standing issue. When running the tests on OS X I typically see a > hang in maybe 1 out of 5-10 runs. > > If this happens to you consistently, that's another matter. Ah, okay, then. I am running the test again now, and it made it past that point this time. > > After more than an hour of > > nothing happening, I interrupted the test by typing ^C, with the > > following result: > > FWIW, if tests seem to hang longer a few of minutes, they really are > hung. (Assuming modern hardware.) Heh. Yeah, well, I didn't hang around for a whole hour. I was off doing something else, not knowing that the test was still going. Anyhow, before receiving your response I tried to run the test on its own: ; sh run-tests.sh --break-on-failure threads.impure.lisp and it failed with // Running impure tests (#<FUNCTION RUN-TESTS::LOAD-TEST>) unhandled SB-INT:C-STRING-DECODING-ERROR in thread #<SB-THREAD:THREAD "initial thread" RUNNING {100299A0C3}>: c-string decoding error (:external-format :ASCII): the octet sequence 1 cannot be decoded. [... I can provide the rest of the output if needed ...] This failure happens every time. Since it runs fine when all tests are being run, I assume this is a failure of the testing system itself and not of the lisp it runs on? - Harald |
From: Juho S. <js...@ik...> - 2011-11-28 21:35:19
|
On Mon, Nov 28, 2011 at 5:35 PM, Martin Cracauer <cra...@co...> wrote: > Juho Snellman wrote on Mon, Nov 28, 2011 at 05:29:29AM +0100: > > Hi, > > > > If nothing unexpected happens, I'll be releasing a new SBCL next weekend. > > Until then, testing and fixes for regressions will be appreciated more > than > > radical code changes :-) > > The new heap management doesn't quite work for me. I can't make sense > of this relation between bytes allocated and max heap size: > When saving a core you might need a dynamic space of up to double the high-water memory usage, due to the kludgy way the core is compacted. Basically at that point the gc will act like a non-conservative semi-space collector: first it does a gc into memory above last_free_page, and then another one to relocate the data to the start of the heap. This has been the case for a long time, so a failure to save a core with 2.7G of live data with a 4G heap should not be a new failure. The part about that output that looks dodgy is that in one place it claims 1.7G of heap in use, and in another 2.7G. How large a core do you expect this to produce? -- Juho Snellman |
From: Martin C. <cra...@co...> - 2011-11-28 21:40:25
|
Juho Snellman wrote on Mon, Nov 28, 2011 at 10:35:12PM +0100: > On Mon, Nov 28, 2011 at 5:35 PM, Martin Cracauer <cra...@co...> wrote: > > > Juho Snellman wrote on Mon, Nov 28, 2011 at 05:29:29AM +0100: > > > Hi, > > > > > > If nothing unexpected happens, I'll be releasing a new SBCL next weekend. > > > Until then, testing and fixes for regressions will be appreciated more > > than > > > radical code changes :-) > > > > The new heap management doesn't quite work for me. I can't make sense > > of this relation between bytes allocated and max heap size: > > > > When saving a core you might need a dynamic space of up to double the > high-water memory usage, due to the kludgy way the core is compacted. > Basically at that point the gc will act like a non-conservative semi-space > collector: first it does a gc into memory above last_free_page, and then > another one to relocate the data to the start of the heap. This has been > the case for a long time, so a failure to save a core with 2.7G of live > data with a 4G heap should not be a new failure. > > The part about that output that looks dodgy is that in one place it claims > 1.7G of heap in use, and in another 2.7G. How large a core do you expect > this to produce? Yeah I didn't spot this before I mailed. I expect 1.7 GB (so under the 2 GB limit) and there are no changes that would expect me to suddenly use 1 GB more than that. Martin -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer <cra...@co...> http://www.cons.org/cracauer/ |
From: Josh E. <jo...@el...> - 2011-11-29 16:23:19
|
On Mon, Nov 28, 2011 at 11:35:49AM -0500, Jim Wise wrote: > Juho Snellman <js...@ik...> writes: > > > Hi, > > > > If nothing unexpected happens, I'll be releasing a new SBCL next > > weekend. Until then, testing and fixes for regressions will be > > appreciated more than radical code changes :-) > > Current status on Solaris: > > Solaris x86 still cons'es madly in room.test.sh -- reverting this commit > > e7b2c507 'more robust backtraces for syscalls on x86' > > addresses this completely, but I'm not sure of the correct fix for this. > > Otherwise, tests look good. I've had this lying around for a while. With these optimizations turned off here, all tests pass cleanly on OpenBSD/i386 for the first time in a while. Nikodemus, was this intended to be a commit-able workaround or is it merely masking the real cause here? diff --git a/src/code/room.lisp b/src/code/room.lisp index 4f81df1..c56cd97 100644 --- a/src/code/room.lisp +++ b/src/code/room.lisp @@ -212,6 +212,7 @@ #!-sb-fluid (declaim (maybe-inline map-allocated-objects)) (defun map-allocated-objects (fun space &optional careful) (declare (type function fun) (type spaces space)) + (declare (optimize (sb!c:alien-funcall-saves-fp-and-pc 0))) (flet ((make-obj (tagged-address) (if careful (make-lisp-obj tagged-address nil) |
From: Nikodemus S. <nik...@ra...> - 2011-11-29 16:48:10
|
On 29 November 2011 18:23, Josh Elsasser <jo...@el...> wrote: > Nikodemus, was this intended to be a commit-able workaround or is it > merely masking the real cause here? A bit of both, maybe. We should really be able to cons those things on stack, which would remove the performance penalty and heap-consing... but as long as it does heap-cons fp and pc, I think that declaration makes perfect sense for MAP-ALLOCATED-OBJECTS. Cheers, - nikodemus |
From: Jim W. <jw...@dr...> - 2011-11-29 18:38:14
|
Nikodemus Siivola <nik...@ra...> writes: > On 29 November 2011 18:23, Josh Elsasser <jo...@el...> wrote: > >> Nikodemus, was this intended to be a commit-able workaround or is it >> merely masking the real cause here? > > A bit of both, maybe. > > We should really be able to cons those things on stack, which would > remove the performance penalty and heap-consing... but as long as it > does heap-cons fp and pc, I think that declaration makes perfect sense > for MAP-ALLOCATED-OBJECTS. Just to confirm, this does indeed address the issue on Solaris/x86 as well. Thanks, -- Jim Wise jw...@dr... |
From: Juho S. <js...@ik...> - 2011-11-30 17:17:32
|
On Mon, Nov 28, 2011 at 10:40 PM, Martin Cracauer <cra...@co...> wrote: > Yeah I didn't spot this before I mailed. > > I expect 1.7 GB (so under the 2 GB limit) and there are no changes > that would expect me to suddenly use 1 GB more than that. Ok. But it's still close enough to 2G that I could easily believe that the pre-core saving memory use + 1.7G doesn't fit in 4G. Are you explicitly setting bytes-consed-between-gcs? If not, perhaps the new larger default value is causing the memory use / high water mark to be a bit larger than before, and thus causing the failure. -- Juho Snellman |
From: Martin C. <cra...@co...> - 2011-11-30 18:32:48
|
Juho Snellman wrote on Wed, Nov 30, 2011 at 06:17:25PM +0100: > On Mon, Nov 28, 2011 at 10:40 PM, Martin Cracauer <cra...@co...> wrote: > > > Yeah I didn't spot this before I mailed. > > > > I expect 1.7 GB (so under the 2 GB limit) and there are no changes > > that would expect me to suddenly use 1 GB more than that. > > > Ok. But it's still close enough to 2G that I could easily believe that the > pre-core saving memory use + 1.7G doesn't fit in 4G. Are you explicitly > setting bytes-consed-between-gcs? If not, perhaps the new larger default > value is causing the memory use / high water mark to be a bit larger than > before, and thus causing the failure. Sorry guys it was just me being stupid. I reacted to the lowering of the heap by setting it back to 4 GB but that overrode our own (higher) setting during our image build. Then I picked up on the 2.7 versus 1.7 GB mismatch in the output and thought it's a new issue. But it isn't. I'm up and running with a 8 GB size now. [pats himself on shoulder] That also means I can probably give the head a full spin here before the weekend. Martin -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer <cra...@co...> http://www.cons.org/cracauer/ |
From: Martin C. <cra...@co...> - 2011-12-02 17:57:20
|
I thought this was fixed but I still have these spurious output messages in 07: (:REF #<SB-C::REF :%SOURCE-NAME SB-C::SEQ :LEAF #<SB-C::LAMBDA-VAR :%SOURCE-NAME SB-C::SEQ :TYPE #<SB-KERNEL:ARRAY-TYPE (SIMPLE-VECTOR 6)> {102A179B43}> {102A17C0F3}> #<SB-C::LVAR 1 {102A17D863}>) (:REF #<SB-C::REF :%SOURCE-NAME SB-C::SEQ :LEAF #<SB-C::LAMBDA-VAR :%SOURCE-NAME SB-C::SEQ :TYPE #<SB-KERNEL:ARRAY-TYPE (SIMPLE-VECTOR 6)> {102A217A33}> {102A21A023}> #<SB-C::LVAR 2 {102A21B793}>) (:REF #<SB-C::REF :%SOURCE-NAME SB-C::SEQ :LEAF #<SB-C::LAMBDA-VAR :%SOURCE-NAME SB-C::SEQ :TYPE #<SB-KERNEL:ARRAY-TYPE (SIMPLE-VECTOR 6)> {102A08DDF3}> {102A090C33}> #<SB-C::LVAR 3 {102A092473}>) (:REF #<SB-C::REF :%SOURCE-NAME SB-C::SEQ :LEAF #<SB-C::LAMBDA-VAR :%SOURCE-NAME SB-C::SEQ :TYPE #<SB-KERNEL:ARRAY-TYPE SIMPLE-VECTOR> {10212659B3}> {1021268783}> #<SB-C::LVAR 1 {1021269EF3}>) -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer <cra...@co...> http://www.cons.org/cracauer/ |
From: Nikodemus S. <nik...@ra...> - 2011-12-02 22:25:50
|
On 2 December 2011 19:57, Martin Cracauer <cra...@co...> wrote: > I thought this was fixed but I still have these spurious output > messages in 07: Wow. This has been there since 1.0.48.21. Pushed a fix. Cheers, -- nikodemus |