Re: [Maxima-discuss] Handling files with strange headers

[serge de m. <sde...@gm...>]

You could also set the encoding-format to the correct value:
:lisp (setf sb-impl::*default-external-format* <the-correct-value>)

But I have no idea what the correct value could be.
You can get a list of the accepted encodings like this:
:lisp (loop for key being the hash-key of sb-impl::*external-formats*
collect key)

Maybe :EUC-JP (since you tell us the customer is japanese)?

This is sbcl specific.

  serge


On Thu, Sep 15, 2016 at 9:03 AM, André Fettouhi <a.f...@gm...>
wrote:

> Yeah I noticed that the line seems to hold hidden characters. If I
> remove thos then it works. The data is originally from a japanese
> company we do business with. They extract their data via excel and then
> save it to a csv file for us. Guess the easiest this is to remove the
> line before I read it into maxima.
>
> Regards
>
> André
>
> Den 15-09-2016 kl. 08:56 skrev Gunter Königsmann:
> > There are many ways to express non-standard-ASCII characters the local
> > language can contain: One can just grab the 256 most importent
> > characters for a given country and use them as character set. This is
> > what the windows codepages do. One can use more than 8 bits for every
> > character (but this will make even standard english text incompatible
> > with your default text editor) or one can use UTF-8 that expresses
> > standard-ASCII characters (ASCII code 0-127) using the characters one is
> > used to, but characters 128-255 in a more elaborate scheme that allows
> > to use up to 4 bytes to gather bits for expressing mandarin, emoticons,
> > mathematical symbols, playing cards or others.
> >
> > Generally most programs tend to switch to unicode as it is the way to
> > handle characters that aren't used in the local languague.
> >
> > Maxima threads every character with the Most Significant Bit set as an
> > ordinary letter - which means it handles both unicode and Codepage-xxx
> > encoding exactly in the way an user would expect. But unfortunately lisp
> > tends to verify if a string is encoded in the Right Way for the encoding
> > that has been selected. Your lisp seems to be set up in a way that
> > handles unicode (and therefore throws an error if a character isn't
> > valid unicode even if maxima wouldn't have any problem handling it.
> >
> > Kind regards,
> >
> >  Gunter.
> >
> > ------------------------------------------------------------
> ------------------
> > _______________________________________________
> > Maxima-discuss mailing list
> > Max...@li...
> > https://lists.sourceforge.net/lists/listinfo/maxima-discuss
>
> --
> -------------------------------------------
> Dr. rer. nat. André Fettouhi
> Mejerivej 40, Hjallese
> DK-5260 Odense S
> Denmark
> Mobile Phone: +45-23269256
> E-Mail: A.F...@gm...
> -------------------------------------------
>
>
> ------------------------------------------------------------
> ------------------
> _______________________________________________
> Maxima-discuss mailing list
> Max...@li...
> https://lists.sourceforge.net/lists/listinfo/maxima-discuss
>