[clisp-list] character sets

[Pascal J.B. <pj...@in...>]

Don Cohen writes:
> Pascal J.Bourguignon writes:
> 
>  > You could invent your own mapping though.
> I thought so, but have not yet succeeded.
> 
>  > > But make-encoding should be able to make an encoding that does
>  > > whatever I want, shouldn't it?
>  > 
>  > You did not pass it a :charset argument with a set of 256
>  > characters.  Why should it not return an error?
>
> The code I sent did not attempt to generate a 1-1 mapping, just
> a mapping that would not generate an error.  And even that did
> not work.
> Notice that make-encoding did not complain.  It was when I read
> an illegal character that it complained, and I thought that's what
> :ignore was supposed to prevent.  If I supply a default character
> I still get an error.

There is no invalid characters.  Characters are.  There may be invalid
codes, codes that correspond to no character, and there may be invalid
byte sequences, byte sequences that encode no character.



>  > Note that since there exist no character with a code equal to 141,
>  > YOU will have to choose an existing character to put in the slot 141
>  > of the :charset argument.
>
> And is this not what the argument
> (make-encoding :input-error-action #\space) [or :ignore]
> is supposed to do?
> 
>  > Remember that  in C, char  is actually an  integer type!  So  it's not
>  > surprizing that you  can store 141 in a char  variable.  But that does
>  > not mean that there exist any  character that have a code equal to 141
>  > in any official encoding.  You still can invent your own encoding, but
>  > you have to specify it (with the :charset key)!
> 
> Which one should I use?  I've now tried a few - same result.

Now, I must excuse me, I thought that the string that can be passed as
:charset argument to  make-encoding was a set of  character, but it is
not.  It  is expected  to be  the name of  an existing  character set.
There  does not  appear  to be  any public  (implementation-dependend)
function to create your own character set.

That  means  that you'll  have  to go  the  portable  way, write  pure
Common-Lisp code,  and implement your own encoding  yourself.  This is
not difficult at all:


(defvar *my-encoding*
  (let ((cs (make-string 256  :initial-element (character "?"))))
    (loop for i from 32 upto 127
          do (setf (aref cs i) (code-char i)))
    cs));;*my-encoding*


(defun read-char-with-my-encoding
  (stream &optional (eof-error-p t) eof-value)
  (assert (equal '(unsigned-byte 8) (stream-element-type stream)))
  (let ((byte (read-byte stream eof-error-p eof-value)))
    (if (typep byte '(unsigned-byte 8))
      (aref *my-encoding* byte)
      byte)));;read-char-with-my-encoding



(defun write-char-with-my-encoding
  (character &optional (output-stream *standard-output*))
  (assert (equal '(unsigned-byte 8) (stream-element-type output-sstream)))
  (let ((byte (position character *my-encoding*
                        ;; A little hack depending on *my-encoding*:
                        :start 32)))
    (write-char (aref *my-encoding* byte) :output-stream output-stream))
  );;write-char-with-my-encoding



(with-open-file (in "/tmp/a" :element-type '(unsigned-byte 8))
  (do ((char (read-char-with-my-encoding in nil nil)
             (read-char-with-my-encoding in nil nil))
       (i 0 (1+ i)))
      ((null char))
    (when (= 0 (mod i 17)) (terpri))
    (format t "~C " char)))

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
?   ! " # $ % & ' ( ) * + , - . / 
? 0 1 2 3 4 5 6 7 8 9 : ; < = > ? 
? @ A B C D E F G H I J K L M N O 
? P Q R S T U V W X Y Z [ \ ] ^ _ 
? ` a b c d e f g h i j k l m n o 
? p q r s t u v w x y z { | } ~ _ 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 
? 
NIL


$ od -t x1 /tmp/a
0000000 0a 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e
0000020 0f 0a 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d
0000040 1e 1f 0a 20 21 22 23 24 25 26 27 28 29 2a 2b 2c
0000060 2d 2e 2f 0a 30 31 32 33 34 35 36 37 38 39 3a 3b
0000100 3c 3d 3e 3f 0a 40 41 42 43 44 45 46 47 48 49 4a
0000120 4b 4c 4d 4e 4f 0a 50 51 52 53 54 55 56 57 58 59
0000140 5a 5b 5c 5d 5e 5f 0a 60 61 62 63 64 65 66 67 68
0000160 69 6a 6b 6c 6d 6e 6f 0a 70 71 72 73 74 75 76 77
0000200 78 79 7a 7b 7c 7d 7e 7f 0a 80 81 82 83 84 85 86
0000220 87 88 89 8a 8b 8c 8d 8e 8f 0a 90 91 92 93 94 95
0000240 96 97 98 99 9a 9b 9c 9d 9e 9f 0a a0 a1 a2 a3 a4
0000260 a5 a6 a7 a8 a9 aa ab ac ad ae af 0a b0 b1 b2 b3
0000300 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf 0a c0 c1 c2
0000320 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf 0a d0 d1
0000340 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df 0a e0
0000360 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef 0a
0000400 f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff
0000420 0a
0000421


-- 
__Pascal_Bourguignon__                     http://www.informatimago.com/
There is no worse tyranny than to force a man to pay for what he doesn't
want merely because you think it would be good for him.--Robert Heinlein
http://www.theadvocates.org/