Steel Bank Common Lisp

The problem line here is
(rotatef (subseq first-parent crossover-point)
      (subseq second-parent crossover-point))

Here you change the hash keys after adding them to the hash.  Not nice.

Compare:

(defun dont-try-this-at-home ()
  (let ((k1 (list 1))(k2 (list 0)) (hash (make-hash-table :test 'equal)))
    (setf (gethash k1 hash) 1)
    (setf (gethash k2 hash) 1)
    (setf (first k1) 0)
    (loop for k being the hash-keys in hash collect k)))

DONT-TRY-THIS-AT-HOME
* (dont-try-this-at-home)

((0) (0))
*

Stig Hemmer

Thank you, Kevin Reid and Stig Hemmer,

So first-parent and second-parent are pointers to the actual keys of the 
hash-table, and not copies of the elements of the vector, which is in 
turn a copy of the keys?

How do I make this not happen?  I tried googling pointers in CL but 
couldn't find anything.  The closest thing I can come up with is to use 
COPY-LIST on the collected keys when making the vector.  I'd still like 
to learn more about when CL makes a copy and when it works as a pointer.

Thank you

As a general rule of thumb, in CL everything is a pointer (or
reference, if you prefer that word), and copies are only made when
asked for.

COPY-LIST is one way of requesting a copy.  You might also be
interested in APPEND.

If you read the Hyperspec (Ask Google for "Common Lisp Hyperspec") it
is very clear on what is copied when.  The Hyperspec is a bit dense
for a beginner to the language, but it is so wonderfully precise that
it should be your first source of information for all things Common
Lisp.

I am a bit vary of giving you more exact advice as it is not clear to
me what your code is trying to do.

Stig Hemmer

On 8/28/11, Ravi Desai <rd7190@...> wrote:
> Thank you, Kevin Reid and Stig Hemmer,
>
> So first-parent and second-parent are pointers to the actual keys of the
> hash-table, and not copies of the elements of the vector, which is in
> turn a copy of the keys?
>
> How do I make this not happen?  I tried googling pointers in CL but
> couldn't find anything.  The closest thing I can come up with is to use
> COPY-LIST on the collected keys when making the vector.  I'd still like
> to learn more about when CL makes a copy and when it works as a pointer.
>
> Thank you
>
> ------------------------------------------------------------------------------
> EMC VNX: the world's simplest storage, starting under $10K
> The only unified storage solution that offers unified management
> Up to 160% more powerful than alternatives and 25% more efficient.
> Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev
> _______________________________________________
> Sbcl-help mailing list
> Sbcl-help@...
> https://lists.sourceforge.net/lists/listinfo/sbcl-help
>

On 08/28/2011 01:38 PM, Stig Hemmer wrote:
> As a general rule of thumb, in CL everything is a pointer (or
> reference, if you prefer that word), and copies are only made when
> asked for.
Thanks for the tip.  I'll keep it in mind from now on.
> COPY-LIST is one way of requesting a copy.  You might also be
> interested in APPEND.
>
> If you read the Hyperspec (Ask Google for "Common Lisp Hyperspec") it
> is very clear on what is copied when.  The Hyperspec is a bit dense
> for a beginner to the language, but it is so wonderfully precise that
> it should be your first source of information for all things Common
> Lisp.
>
> I am a bit vary of giving you more exact advice as it is not clear to
> me what your code is trying to do.
The point of this function is to create different permutations (As 
represented by the list of 0s and 1s).  I'm using a hash-table to make 
sure that each permutation is unique.  The permutations are the keys 
because the hash-table will ensure that they are only unique 
permutations in that set.

Basically, I want to create a set of unique permutations of 1s and 0s.

I suppose I could also just use a list instead of a hash-table and use 
functions like MEMBER, PUSHNEW, etc.

Thanks

On 24 August 2011 18:28, Peter Keller <psilord@...> wrote:

> I find that often (maybe 100% reproducable), when I write a slot handler
> for a gtk signal with an incorrect lambda-list and run the software,
> I get what appears to be a debugger break in a thread with an error
> message like "wrong arguments 2"...and then nothing. The REPL thread
> doesn't seem to notice and if I break the REPL into the debugger again,
> I have no idea how to access the other threads. No matter what settings
> I use for debug/safty/optimize I can't get any better error message than
> that or a backtrace in the offending thread.

Is the in the terminal?

If so, the thread with the error is probably waiting for the terminal.
(The current mechanism that mediates this is ... not the nicest.)

Here's an example, ;;; are my comments.

* (defvar *th* (make-thread (lambda () (error "My bad!")) :name "Oops"))

*TH*
*
;;; This is the other thread barfing. It is now waiting for access to the
;;; terminal.
debugger invoked on a SIMPLE-ERROR in thread #<THREAD "Oops" RUNNING
                                                {1003112AB1}>:
  My bad!
;;; We're still in the original thread, though.
*th*

#<THREAD "Oops" RUNNING {1003112AB1}>
* *current-thread*

#<THREAD "initial thread" RUNNING {1002939071}>
;;; But we can release the terminal to allow us to debug the other thread.
* (release-foreground)

Resuming thread #<THREAD "Oops" RUNNING {1003112AB1}>

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [TERMINATE-THREAD] Terminate this thread (#<THREAD "Oops" RUNNING
                        {1003112AB1}>)

((LAMBDA ()))0
0] ba

0: ((LAMBDA ()))
1: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK443]448))
2: ((FLET SB-THREAD::WITH-MUTEX-THUNK))
3: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]309))
4: (SB-THREAD::CALL-WITH-MUTEX
    #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK) {322BD09}>
    #<MUTEX "thread result lock" owner: #<THREAD "Oops" RUNNING {1003112AB1}>>
    #<THREAD "Oops" RUNNING {1003112AB1}>
    T)
5: (SB-THREAD::INITIAL-THREAD-FUNCTION)
6: ("foreign function: call_into_lisp")
7: ("foreign function: new_thread_trampoline")
8: ("foreign function: _pthread_start")
9: ("foreign function: thread_start")

0] 0
Resuming thread #<THREAD "initial thread" RUNNING {1002939071}>

* *th*

#<THREAD "Oops" ABORTED {1003112AB1}>

>> 2) On Linux there's currently an undetermined SBCL bug which somehow
>> manifests itself very often in presence of semi-complex CL-GTK2 code.
>> The end result is that after the first time the offending code is
>> executed, thread handling fails silently and catastrophically,
>> including the complete inability to interrupt other threads (which
>> also precludes the debugger from noticing anything going wrong). It's
>> very possible that you're seeing this. In this case, there's not much
>> to be done except a long and torturous debugging session to fix SBCL.

This is the first I hear of this.

Information how how to reproduce this would be most welcome.

I have two theories as to what might cause something that vaguely
resembles the description:

 * GC deadlock. If CL-GTK uses WITHOUT-GCING anywhere, then all locks it
   obtains while in WITHOUT-GCING code must only ever be held while inside
   WITHOUT-GCING. (Moral: don't use WITHOUT-GCING unless you really have to,
   and then seriously try to minimize the amount of things done there.)

 * Foreign threads calling into Lisp. This is not currently supported at
   all. Only threads created using MAKE-THREAD can call into Lisp.

> What do the SBCL folks recommend I do so I can figure out how to debug these
> errors? How do I (in general) access different threads from the debugger?

I have no experience with CL-GTK2, but unless it is violently
Slime-unfriendly, using it with Slime might make your life easier.

Cheers,

 -- Nikodemus

Crowdfunding SBCL: http://www.indiegogo.com/SBCL-Threading-Improvements-1

On Wed, 2011-08-24 at 19:30 +0300, Nikodemus Siivola wrote:
> On 24 August 2011 18:28, Peter Keller <psilord@...> wrote:
> 
> >> 2) On Linux there's currently an undetermined SBCL bug which somehow
> >> manifests itself very often in presence of semi-complex CL-GTK2 code.
> >> The end result is that after the first time the offending code is
> >> executed, thread handling fails silently and catastrophically,
> >> including the complete inability to interrupt other threads (which
> >> also precludes the debugger from noticing anything going wrong). It's
> >> very possible that you're seeing this. In this case, there's not much
> >> to be done except a long and torturous debugging session to fix SBCL.
> 
> This is the first I hear of this.
> 
> Information how how to reproduce this would be most welcome.

Actually, it seems like an issue in cl-gtk2 that was fixed a long time
ago (I don't know why it reappeared - I haven't looked into it yet). The
issue was that Gtk+ changes signal handlers:

  "Since 2.18, GTK+ calls signal (SIGPIPE, SIG_IGN) during
initialization, to ignore SIGPIPE signals, since these are almost never
wanted in graphical applications. If you do need to handle SIGPIPE for
some reason, reset the handler after gtk_init(), but notice that other
libraries (e.g. libdbus or gvfs) might do similar things."

 (from http://developer.gnome.org/gtk/2.24/gtk-General.html#gtk-init )

Since SBCL uses SIGPIPE for thread interruption, setting SIGPIPE to
SIG_IGN breaks it. cl-gtk2 works around this by reenabling SIGPIPE after
gtk_init
(https://github.com/dmitryvk/cl-gtk2/commit/f6d4e6c74f4490bed63810a07587a0a63fd8c394)

> 
> I have two theories as to what might cause something that vaguely
> resembles the description:
> 
>  * GC deadlock. If CL-GTK uses WITHOUT-GCING anywhere, then all locks it
>    obtains while in WITHOUT-GCING code must only ever be held while inside
>    WITHOUT-GCING. (Moral: don't use WITHOUT-GCING unless you really have to,
>    and then seriously try to minimize the amount of things done there.)
> 
>  * Foreign threads calling into Lisp. This is not currently supported at
>    all. Only threads created using MAKE-THREAD can call into Lisp.

cl-gtk2 does none of this, so this should not be a problem.

On Wed, Aug 24, 2011 at 07:30:30PM +0300, Nikodemus Siivola wrote:
> Is the in the terminal?

yes. SBCL running in an xterm.

> ;;; But we can release the terminal to allow us to debug the other thread.
> * (release-foreground)

This works just fine and I am _finally_ able to get the backtrace I need.
Thank you very much!

> Information how how to reproduce this would be most welcome.

I've attached a reasonably small example which shows the problem.

The line of code with the error on it is near "XXX THIS IS THE BUG" comment.
You will see that the lambda-list is wrong.

To reproduce the problem, unpack the attached tarball, have quicklisp
installed.  Make sure these are installed and they work from quicklisp:

(The last one in this list might take some futzing to work since it isn't
enabled by default in quicklisp but is available in quicklisp's cl-gtk2)

#:cl-opengl #:cl-glu #:lispbuilder-sdl #:cl-gtk2-gtk #:cl-gtk2-gtkglext

Then, run the software by (require :test-1) (in-package :test-1)
(doit), you should see a rotating square. Put the mouse into the opengl
buffer, and hit a single key. Then you should see that the gtk main
thread emits an error, but there is no keyboard response. Then, Ctrl-C,
abort--returning to toplevel, then type (sb-thread:release-foreground),
and finally you get something very meaningful in terms of debugging
information and backtrace ability.

However, if you then terminate the thread, the busted gtk window stays up.
If you run (doit) again, it just sits there, doing nothing and not
consuming any CPU.

At this point in time, I have to tear down the SBCL session and restart
it since it just seems broken. As far as I am concerend, I aborted both
the gtk main thread and the original call to doit, so I should be able
to restart the program and have it work.

Thank you.

-pete

* 2011-08-18T17:36:00+02:00 * Arne Klein wrote:

> Is there any recommendation which one to use?

Yes, here's mine: use CFFI because then you write higher-level code. It
abstracts away implementation-specific code. Common language for all
Common Lisp programmers is good. Scattered communities is bad.

Arne Klein <arne.klein@...> writes:

> Hello,
>
> I started writing Lisp bindings for a C library with CFFI, but I'm not
> sure that this is the best way to go, or if I should use SBCL FFI
> instead. I couldn't find any good information about the advantages and
> disadvantages of CFFI compared to the SBCL FFI (except that these are
> limited to SBCL of course, but this isn't a problem for my use-case).
>
> Is there any recommendation which one to use? Is maybe one of them
> (probably SBCL FFI) only kept for backward compatibility and its use
> not recommended for new projects? I'd be very glad about any help on
> this topic.
CFFI uses SBCL's FFI. CFFI is a compatibility layer for FFIs across
different implementations.
So, if you choose to use SBCL's FFI, your code will run only
on SBCL, while when using CFFI it will be portable among other
implementations.

-- 
With best regards, Stas.

In article <87pqk2k64l.fsf@...>,
 Teemu Likonen <tlikonen@...> wrote:

> * 2011-08-18T17:36:00+02:00 * Arne Klein wrote:
> 
> > Is there any recommendation which one to use?
> 
> Yes, here's mine: use CFFI because then you write higher-level code. It
> abstracts away implementation-specific code. Common language for all
> Common Lisp programmers is good. Scattered communities is bad.

I'm all for portable code, but it is patently false to assert that 
smoothing over difference in systems until convergence to a common 
subset of features leads to a higher-level interface.  If anything, it 
clearly has the opposite effect.  There are multiple meanings to the 
word "abstract", and some programmers seem to harmfully conflate them, 
sometimes in the same sentence.

There are fewer SB-ALIEN users, and parts of it can be obscure.  That's 
a good reason to use CFFI, even if you're not worried about using other 
implementations.

On the other hand, SB-ALIEN hooks usefully in SBCL's type system, giving 
you more, more useful, diagnoses and warnings at compile-time.

Paul Khuong

Don Geddis <don@...> writes:

> I had been unable to find a list of possible external-formats in sbcl,
> so I couldn't just iterate through the choices.

For next time, you can find a list of possible external-formats in the
manual, at <http://www.sbcl.org/manual/index.html#External-Formats>.

Cheers,

Christophe

Don Geddis <don@...> writes:

> Paul Khuong <pvk@...> wrote on Fri, 12 Aug 2011:
>> In article <877h6i822c.fsf@...>, Don Geddis <don@...> wrote:
>>> Adding ":external-format :utf-8" doesn't seem to help (me).  CMUCL works
>>> with or without it.  SBCL fails, with or without it.
>>
>> Does that log include "F\"{u}r Elise"?
>
> Yes!  :-)
>
>> Looks like your file is in something like ISO-8859-1, not UTF-8.
>
> That's it!  You solved my problem.  Adding
>         :external-format :iso-8859-1
> to WITH-OPEN-FILE allows me to read strings from my data file.
>
> I had been unable to find a list of possible external-formats in sbcl,
> so I couldn't just iterate through the choices.

This is not a good way to determine the encoding of a file, because a
lot of byte sequence can be decoded with multiple encoding systems.
You could add an heuristic testing for the decoded characters, but this
would still leave ambiguous results, on non-word data.


> It also doesn't quite solve my secondary problem, of loading lisp code
> that has literal strings (which apparently are in ISO-8859-1 also).
> Although I guess LOAD takes an EXTERNAL-FORMAT keyword also, so I
> suppose I could change all my LOADs.  

The default encoding in sbcl is set from the locale environment
variables.

> Too bad there doesn't seem to be anything I can do inside the source
> code that is being loaded.

For files that have a non-default encoding, have a look at 
https://gitorious.org/com-informatimago/com-informatimago/blobs/master/tools/make-depends.lisp
there's code to read the Emacs File Local Variables, such as
"-*- coding:utf-8 -*-" (or "Local Variables:" at the end), using 
com.informatimago.common-lisp.cesarum.file:safe-text-file-to-string-list
and
com.informatimago.common-lisp.cesarum.character-sets:emacs-encoding-to-lisp-external-format
which allows scan-source-file to open the source files with the correct
encoding.

Notably, see the definition of
com.informatimago.common-lisp.cesarum.character-sets::*lisp-encodings* 
for the implementation specific code used to collect the coding systems
(external formats) in various implementations.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Christophe Rhodes <csr21@...> wrote on Sat, 13 Aug 2011:
> Don Geddis <don@...> writes:
>>   decoding error on stream
>>   #<SB-SYS:FD-STREAM for "file /home/geddis/www/don/lisp/asciify.lisp"
>>     {AA7C181}>
>>   (:EXTERNAL-FORMAT :ASCII):
>>     the octet sequence (225) cannot be decoded.
> Your external format for this stream is :ASCII, meaning that code
> points (bytes) 0-127 are mapped to characters, and everything else is
> an error: ASCII is a 128-character repertoire.  The first step in
> fixing this problem is going to be opening asciify.lisp in a non-ASCII
> external format.  (I'm surprised that you're getting ASCII as the
> default, even after you did something explicitly to select a UTF-8
> locale; checking with "locale -k LC_CTYPE | grep charmap" might be
> worthwhile).  SBCL takes its default external format from the
> environment; you can check that it's doing that correctly by looking
> at sb-impl::*default-external-format*, once your Unix environment is
> sorted out.

Just to complete this report:

        unix:~> echo $LANG
        en_US.UTF-8

        unix:~> locale -k LC_CTYPE | grep charmap
        charmap="ANSI_X3.4-1968"

        unix> sbcl
        This is SBCL 1.0.50.0.debian, an implementation of ANSI Common Lisp.
        More information about SBCL is available at <http://www.sbcl.org/>.
        
        SBCL is free software, provided as is, with absolutely no warranty.
        It is mostly in the public domain; some portions are provided under
        BSD-style licenses.  See the CREDITS and COPYING files in the
        distribution for more information.
        * sb-impl::*default-external-format*
        
        :ANSI_X3.4-1968
        * (quit)

> The second step is going to be to identify the actual encoding of your
> source file: is it stored on the disk in ISO-8859-1, UTF-8 or something
> else?  From looking at the file that's served up from your webserver, I
> suspect ISO-8859-1 (an 8-bit encoding that I think we can just about
> call "legacy" these days), which means that even if you have a UTF-8
> locale you will have to tell sbcl explicitly to load it with that
> external format -- or else you'll have to set up your Unix to use an
> ISO-8859-1 LC_CTYPE.  (I don't recommend this unless you really know
> what you're doing).

And, obviously, I barely know what I'm doing.

It seems like I would have more luck if my source file were UTF-8
instead of ISO-8859-1.  I think it came from typing a text file into
emacs, perhaps copying particular characters from usenet postings read
in GNUS (again, with emacs).  Or possibly from a web browser, into an
emacs text file.

Does't seem to help, though.  I did
        iconv -f ISO-8859-1 -t UTF-8 asciify.lisp > a2.lisp
but an attempt to (load "a2.lisp") in sbcl fails in the same way, for
the obvious reason: as you noted above, my sbcl seems to default to
        :external-format :ascii
so all of these encodings are going to fail (by default).

> I hope that the above gives you some ideas; the important thing is to
> understand character encodings, at which point it should all become
> blindingly obvious.

Thanks for everybody's help.  I now understand this topic (a little)
better.

It still seems ... unhelpful? ... for sbcl to default to :ASCII as an
external format.  You seem surprised by this.  But I have the same
behavior on both a Debian box and also a Ubuntu box (running different
sbcl versions), so it seems intentional.

Nonetheless, I have a workaround: I can add an :EXTERNAL-FORMAT argument
to all my LOADs and COMPILE-FILEs.

Thanks, all.

        -- Don
_______________________________________________________________________________
Don Geddis                  http://don.geddis.org/               don@...
Parents seem to think their kids are like clay, that you mould them into the
right shape when they're wet.  A better metaphor is that kids are like
flexible plastic -- they respond to pressure, but when you release the
pressure they tend to pop back to their original shape.  -- Bryan Caplan

Don Geddis <don@...> writes:

> Just to complete this report:
>
>         unix:~> echo $LANG
>         en_US.UTF-8
>
>         unix:~> locale -k LC_CTYPE | grep charmap
>         charmap="ANSI_X3.4-1968"

ANSI_X3.4-1968 is code for ASCII -- I suspect that this is telling you
that although you've set that LANG on your system, that locale isn't in
fact supported.  What does locale -a say?  FWIW, on my Debian system it
says

  csr21@...:~$ locale -a
  C
  en_GB.utf8
  fr_FR.utf8
  POSIX
  
  csr21@...:~$ echo $LANG 
  en_GB.UTF-8
  
  csr21@...:~$ locale -k LC_CTYPE | grep charmap
  charmap="UTF-8"

And my sbcl defaults to a utf-8 external format.  If you're root, and if
no plausible locales are showing up for "locale -a", you could run
"dpkg-reconfigure locales" to choose some plausible ones to generate for
your system.

> It still seems ... unhelpful? ... for sbcl to default to :ASCII as an
> external format.  You seem surprised by this.  But I have the same
> behavior on both a Debian box and also a Ubuntu box (running different
> sbcl versions), so it seems intentional.

What is intentional is to default to whatever the Unix/nl_langinfo()
LC_CTYPE character map is.  On your systems, that's ASCII, for an
unknown reason; on mine it's definitely not...

Good luck!

Cheers,

Christophe

Don Geddis <don@...> writes:

> Christophe Rhodes <csr21@...> wrote on Sat, 13 Aug 2011:
>> Don Geddis <don@...> writes:
>>>   decoding error on stream
>>>   #<SB-SYS:FD-STREAM for "file /home/geddis/www/don/lisp/asciify.lisp"
>>>     {AA7C181}>
>>>   (:EXTERNAL-FORMAT :ASCII):
>>>     the octet sequence (225) cannot be decoded.
>> Your external format for this stream is :ASCII, meaning that code
>> points (bytes) 0-127 are mapped to characters, and everything else is
>> an error: ASCII is a 128-character repertoire.  The first step in
>> fixing this problem is going to be opening asciify.lisp in a non-ASCII
>> external format.  (I'm surprised that you're getting ASCII as the
>> default, even after you did something explicitly to select a UTF-8
>> locale; checking with "locale -k LC_CTYPE | grep charmap" might be
>> worthwhile).  SBCL takes its default external format from the
>> environment; you can check that it's doing that correctly by looking
>> at sb-impl::*default-external-format*, once your Unix environment is
>> sorted out.
>
> Just to complete this report:
>
>         unix:~> echo $LANG
>         en_US.UTF-8
>
>         unix:~> locale -k LC_CTYPE | grep charmap
>         charmap="ANSI_X3.4-1968"
>
>         unix> sbcl
>         This is SBCL 1.0.50.0.debian, an implementation of ANSI Common Lisp.
>         More information about SBCL is available at <http://www.sbcl.org/>.
>         
>         SBCL is free software, provided as is, with absolutely no warranty.
>         It is mostly in the public domain; some portions are provided under
>         BSD-style licenses.  See the CREDITS and COPYING files in the
>         distribution for more information.
>         * sb-impl::*default-external-format*
>         
>         :ANSI_X3.4-1968
>         * (quit)
What is echo $LC_CTYPE?

-- 
With best regards, Stas.

Stas Boukarev <stassats@...> wrote on Sat, 13 Aug 2011:
> Don Geddis <don@...> writes:
>> Christophe Rhodes <csr21@...> wrote on Sat, 13 Aug 2011:
>>> Don Geddis <don@...> writes:
>>>>   decoding error on stream
>>>>   #<SB-SYS:FD-STREAM for "file /home/geddis/www/don/lisp/asciify.lisp"
>>>>     {AA7C181}>
>>>>   (:EXTERNAL-FORMAT :ASCII):
>>>>     the octet sequence (225) cannot be decoded.
>>> Your external format for this stream is :ASCII, meaning that code
>>> points (bytes) 0-127 are mapped to characters, and everything else is
>>> an error: ASCII is a 128-character repertoire.  The first step in
>>> fixing this problem is going to be opening asciify.lisp in a non-ASCII
>>> external format.  (I'm surprised that you're getting ASCII as the
>>> default, even after you did something explicitly to select a UTF-8
>>> locale; checking with "locale -k LC_CTYPE | grep charmap" might be
>>> worthwhile).  SBCL takes its default external format from the
>>> environment; you can check that it's doing that correctly by looking
>>> at sb-impl::*default-external-format*, once your Unix environment is
>>> sorted out.
>>
>> Just to complete this report:
>>
>>         unix:~> echo $LANG
>>         en_US.UTF-8
>>
>>         unix:~> locale -k LC_CTYPE | grep charmap
>>         charmap="ANSI_X3.4-1968"
>>
>>         unix> sbcl
>>         This is SBCL 1.0.50.0.debian, an implementation of ANSI Common Lisp.
>>         More information about SBCL is available at <http://www.sbcl.org/>.
>>         
>>         SBCL is free software, provided as is, with absolutely no warranty.
>>         It is mostly in the public domain; some portions are provided under
>>         BSD-style licenses.  See the CREDITS and COPYING files in the
>>         distribution for more information.
>>         * sb-impl::*default-external-format*
>>         
>>         :ANSI_X3.4-1968
>>         * (quit)
> What is echo $LC_CTYPE?

"Undefined variable", on both my Debian and Ubuntu servers (running tcsh).
_______________________________________________________________________________
Don Geddis                  http://don.geddis.org/               don@...

Stas Boukarev <stassats@...> wrote on Sat, 13 Aug 2011:
> Don Geddis <don@...> writes:
>>         unix:~> echo $LANG
>>         en_US.UTF-8
>>         unix:~> locale -k LC_CTYPE | grep charmap
>>         charmap="ANSI_X3.4-1968"
> What is echo $LC_CTYPE?

I don't seem to have this variable, but (reading more docs) I see that
the LC_* environment variables are supposed to be determining this
stuff.

On both my Debian and Ubuntu boxes, I get:

        unix> setenv | grep LC
        LC_ALL=C
        NLSPATH=/usr/share/locale/%L/LC_MESSAGES/%N.cat

Perhaps my problem has something to do with $LC_ALL being "C", and
$LC_CTYPE being missing?  Might that somehow override $LANG?

        -- Don
_______________________________________________________________________________
Don Geddis                  http://don.geddis.org/               don@...

Stas Boukarev <stassats@...> wrote on Sat, 13 Aug 2011:
> Don Geddis <don@...> writes:
>>         unix:~> echo $LANG
>>         en_US.UTF-8
>>         unix:~> locale -k LC_CTYPE | grep charmap
>>         charmap="ANSI_X3.4-1968"
> What is echo $LC_CTYPE?

Ah, that was the clue that solved it for me.

I didn't have an $LC_CTYPE variable.  But it turns out that, something
like ten years ago, I put the following in my .tcshrc:
        setenv LC_ALL C
The associated comment says
        # Fix the sort order of "ls"
Who knows what was happening at the time, and what kind of filename
sorting in "ls" I was seeing, and how I thought this would fix it.

However, that line has been in my .tcshrc shell init file ever since.
And, of course, I copied my old shell init file to my new server when I
built it too, a month ago.

I've commented out that line in the init file, and everything works now
just like everybody here has been saying.
        locale -k LC_CTYPE | grep charmap
now returns "UTF-8", sbcl correctly loads source files with such
encodings, etc.

(As you all explained to me earlier in this thread, I had a second
problem in that it appears my actual source files were ISO-8859-1.  But,
using "iconv", I changed them all to UTF-8, and now everything works
great.)

Thanks everyone, for the advice and suggestions, and for the patience to
help me track this down.

        -- Don
_______________________________________________________________________________
Don Geddis                  http://don.geddis.org/               don@...
Hard work never killed anybody, but why take a chance?
	-- Charlie McCarthy (ventriloquist puppet)

Christophe Rhodes <csr21@...> wrote on Sat, 13 Aug 2011:
> Don Geddis <don@...> writes:
>
>> Just to complete this report:
>>
>>         unix:~> echo $LANG
>>         en_US.UTF-8
>>
>>         unix:~> locale -k LC_CTYPE | grep charmap
>>         charmap="ANSI_X3.4-1968"
>
> ANSI_X3.4-1968 is code for ASCII -- I suspect that this is telling you
> that although you've set that LANG on your system, that locale isn't in
> fact supported.  What does locale -a say?  FWIW, on my Debian system it
> says
>
>   csr21@...:~$ locale -a
>   C
>   en_GB.utf8
>   fr_FR.utf8
>   POSIX
>   
>   csr21@...:~$ echo $LANG 
>   en_GB.UTF-8
>   
>   csr21@...:~$ locale -k LC_CTYPE | grep charmap
>   charmap="UTF-8"
>
> And my sbcl defaults to a utf-8 external format.

My "locale -a" seems reasonable.  On my Debian server:

        debian:~> locale -a
        C
        C.UTF-8
        POSIX
        en_US.utf8

and on (newly installed) Ubuntu 11.04:

        ubuntu:~> locale -a
        C
        POSIX
        en_AG
        en_AG.utf8
        en_AU.utf8
        en_BW.utf8
        en_CA.utf8
        en_DK.utf8
        en_GB.utf8
        en_HK.utf8
        en_IE.utf8
        en_IN
        en_IN.utf8
        en_NG
        en_NG.utf8
        en_NZ.utf8
        en_PH.utf8
        en_SG.utf8
        en_US.utf8
        en_ZA.utf8
        en_ZW.utf8
        zh_CN.utf8
        zh_SG.utf8

And, just to repeat, on both systems I get the same results for the
previous tests:

        unix:~> echo $LANG
        en_US.UTF-8
        unix:~> locale -k LC_CTYPE | grep charmap
        charmap="ANSI_X3.4-1968"

> If you're root, and if no plausible locales are showing up for "locale
> -a", you could run "dpkg-reconfigure locales" to choose some plausible
> ones to generate for your system.

The plausible locales seem to be already there.  Nonetheless, I have
done the dpkg-reconfigure.  That's how I (yesterday) set my $LANG to
UTF-8 (on the Debian server).

> What is intentional is to default to whatever the Unix/nl_langinfo()
> LC_CTYPE character map is.  On your systems, that's ASCII, for an
> unknown reason; on mine it's definitely not...

Very odd.  So apparently, the behavior I'm seeing with SBCL is not the
behavior that everybody else is seeing, because something is strange
about my locale setup?  And yet, one of the systems is a brand-new
Ubuntu install within the last month.  I don't recall doing anything
strange with configuring locales.

I wonder why I seem to be getting an ASCII charmap, but nobody else is?

        -- Don
_______________________________________________________________________________
Don Geddis                  http://don.geddis.org/               don@...

Don Geddis <don@...> writes:

> Ah, that was the clue that solved it for me.

\o/ Hooray!
 |
/ \

> I didn't have an $LC_CTYPE variable.  But it turns out that, something
> like ten years ago, I put the following in my .tcshrc:
>         setenv LC_ALL C
> The associated comment says
>         # Fix the sort order of "ls"
> Who knows what was happening at the time, and what kind of filename
> sorting in "ls" I was seeing, and how I thought this would fix it.

Well, another aspect of the Unix locale is collation: how lists are
alphabetized.  In the C/POSIX locale, you basically go in codepoint
order, so lower-case and uppercase letters are separate; in more
"natural language" locales, you go alphabetically.  Compare:

csr21@...:~$ ls
Desktop  Documents  Downloads  games  Music  Pictures  Public  src  teclo  Templates  Videos  VirtualBox VMs
csr21@...:~$ LC_COLLATE=C ls
Desktop  Documents  Downloads  Music  Pictures  Public  Templates  Videos  VirtualBox VMs  games  src  teclo

(In other words, you can preserve both a UTF-8 default character set and
`standard' collation, using LANG=en_US.UTF-8, as you currently do, and
LC_COLLATE=C.  Again, there are excitingly exotic possible collation
orders to do with different alphabetical orders in different natural
languages, none of which should be enabled unless you really want them.)

Cheers,

Christophe

* 2011-08-13T10:48:45-07:00 * Don Geddis wrote:

> I didn't have an $LC_CTYPE variable.  But it turns out that, something
> like ten years ago, I put the following in my .tcshrc:
>         setenv LC_ALL C
> The associated comment says
>         # Fix the sort order of "ls"
> Who knows what was happening at the time, and what kind of filename
> sorting in "ls" I was seeing, and how I thought this would fix it.

Glibc checks LC_ALL first and uses its value if it exists. If LC_ALL is
empty then the appropriate LC_* (e.g., LC_CTYPE, LC_COLLATE) variable is
inspected. If that is empty then LANG variable is inspected.

>         locale -k LC_CTYPE | grep charmap
> now returns "UTF-8", sbcl correctly loads source files with such
> encodings, etc.

You can just type "locale charmap" to get the effective character
encoding.