Re: [Rest2web-develop] user-value not in ascii

[Michael F. <fuz...@vo...>]

Gael Varoquaux wrote:
>     Hi,
>
> r2w so far does a great job of dealing with non ASCII encodings. But I
> found a glitch: I have a user-value in a page that uses a non ascii
> character. Python isn't to happy about that: the line
>
> print node_title + ": " + title
>
> Now if node_tile contains, say "ë", I get this
>
> Traceback (most recent call last):
> [err]   File "./r2w.py", line 185, in ?
> [err]     count = main(options, config)
> [err]   File "./r2w.py", line 94, in main
> [err]     return processor.walk()
> [err]   File "/home/varoquau/www/rest2web/rest2web/restprocessor.py",
> line 385, in walk
> [err]     self.buildsection()
> [err]   File "/home/varoquau/www/rest2web/rest2web/restprocessor.py",
> line 1325, in buildsection
> [err]     uservalues = enc_uni_dict(page['uservalues'], final_encoding)
> [err]   File "/home/varoquau/www/rest2web/rest2web/restutils.py", line
> 227, in enc_uni_dict
> [err]     val = uni_dict[entry].encode(encoding)
> [err]   File "/usr/lib/python2.4/encodings/iso8859_1.py", line 18, in
> encode
> [err]     return codecs.charmap_encode(input,errors,encoding_map)
> [err] UnicodeDecodeError: 'ascii' codec can't decode byte 0xeb in
> position 2: ordinal not in range(128)
> [err]
>
> The encoding of this string is most probably the encoding of the page,
> therefore Latin1. A fix would probably to have the user-value be
> translated from Latin1 to unicode when it is read, as I recon the parser
> knows what the encoding of the page is, at this point.
>
>   
This is the same bug that was reported recently. It will be fixed in the 
next release.

In fact, uservalues are encoded to the final encoding early on.

If you are using the version from subversion (recommended) then the line 
to fix is line 1327 :

Change it from :

    uservalues = enc_uni_dict(page['uservalues'], final_encoding)

to :

    uservalues = page['uservalues']

This works in a simple test case, but I need to check that it is always
the right solution.

> On a side note, when r2w fails with such an error it still return a
> return value of 0, with means success in the Unix world, and is used in
> makefile, or building script a lot. On my website I check for the return
> value of r2w before propagating the website, but I cannot trap the rror,
> as it is not reported.
>   
This is a good point.

Many Thanks


Fuzzyman
http://www.voidspace.org.uk/python/index.shtml


> Cheers,
>
> Gaël
>  
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Rest2web-develop mailing list
> Res...@li...
> https://lists.sourceforge.net/lists/listinfo/rest2web-develop
>
>
>   



-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.407 / Virus Database: 268.12.10/459 - Release Date: 29/09/2006