[Rest2web-develop] (pretty strange) encoding error with rest2web on a Mac

[Ben <bi...@ma...>]

Hello list,

I have a strange issue with file encoding. Actually, this problem only
occurs on my brand new MacBook -- running MacOSX on a HFS+ filesystem, but
everything used to run fine with my previous OS, Linux. I've investiguated
as much as I could but I can't find a solution, could you please tell me you
opinion about it?

First I had some weird behaviors with rest2web encoding handling, the 'file'
command returns a file as UTF8 encoded but the log of rest2web guesses
latin1.
I've added:
[uservalues]
__encoding__ = UTF-8
in my r2w and encoding: utf-8 in files restindex.
Just to let you know my env says: LANG=en_GB.UTF-8.

Things are still a bit weird (rest2web still guesses apparently a wrong
encoding) but kind of work. Except ONE and last file, an indexfile in a
directory.

Here is the error output r2w.py procudes (called by a Makefile):

-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<---
Processing indexfile.
[err] Traceback (most recent call last):
[err]   File "/Users/benoit/local/rest2web-0.5.1/r2w.py", line 170, in
<module>
[err]     count = main(options, config)
[err]   File "/Users/benoit/local/rest2web-0.5.1/r2w.py", line 103, in main
[err]     return processor.walk()
[err]   File "/Users/benoit/local/rest2web-0.5.1/rest2web/restprocessor.py",
line 473, in walk
[err]     errorcheck = self.execute_safely(self.buildsection)
[err]   File "/Users/benoit/local/rest2web-0.5.1/rest2web/restprocessor.py",
line 218, in execute_safely
[err]     val =  function(*args, **keywargs)
[err]   File "/Users/benoit/local/rest2web-0.5.1/rest2web/restprocessor.py",
line 1401, in buildsection
[err]     self.sections, final_encoding, self.dir_as_url, target)
[err]   File "/Users/benoit/local/rest2web-0.5.1/rest2web/restprocessor.py",
line 1735, in handle_sections
[err]     page['page-description'], encoding)
[err]   File "/Users/benoit/local/rest2web-0.5.1/rest2web/restutils.py",
line 240, in encode
[err]     return instring.encode(encoding)
[err] UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2019'
in position 138: ordinal not in range(256)
[err]
make: *** [build] Error 1

-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<---

And here is the faulty text:


-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<---

$ cat src/articles/index.txt
..
    restindex
        link-title: Resources
    encoding: utf-8
        page-description:
        List
        /description
    /restindex

========================================
List
========================================

test

-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<---

I first though it could be from my template but once again, other pages are
sucessfully processed. My locales seem ok...

Any idea? I'm lost.
Thank you very much in advance.

  -- Ben