[Rest2web-develop] Encoding & Uservalues

[Fuzzyman <fuz...@vo...>]

Hello all,

I've finished the first shot at verbosity levels. It should be fairly 
straightforward.

The next feature I'm working on is allowing you to set uservalues in the 
config file. These can be over-ridden in the individual pages by 
explicit uservalues.

This means you can supply values in your config file - and then have 
conditional logic in your templates/pages that use them. After this is 
implemented I will also allow the passing of uservalues at the command line.

I have to work out how to handle encodings of uservalues passed via the 
config file. Uservalues in the page are decoded to unicode (guessing the 
encoding if necessary), and then re-encoded in the output encoding.

This is useful because it means if you have a latin-1 document (for 
example) you can still specify that all your output HTML should be in UTF8.

So how should rest2web handle uservalues passed in the config file and 
at the command line ?

First of all the config file. There are three options :

1) Ignore encodings and assume that the config file values are ascii or 
in the appropriate encoding
2) Decode to unicode immediately and restrict values to ascii only
3) Guess the encoding of the config file, using the same technique as is 
used for guessing the encoding of pages (annoying to implement but well 
possible)
4) Allow a magic value '__encoding__' to specify the encoding in use

My personal preference is 4). I then need to choose between 1), 2) and 
3) as a fallback if that value is missing.

I have a similar problem with values passed at the command line - 
however in this case I can check ``sys.stdin.encoding``. This works (on 
windows) even if stdin has already been closed. I will probably have to 
force ascii only if that information isn't available.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml