Re: [Rest2web-develop] Re: ConfigObj 4 work

[Michael F. <mi...@pc...>]

Hello Nicola,

(and anyone else :-)

Nicola Larosa wrote:
>>I'll list the questions I have, answer any you can be bothered with :-)
> 
> 
> I'll try. :-)
> 

You did pretty well.

> 
> 
>>1) Part of the point of ConfigObj is that you can create config files
>>from a program. Something like :
>>
>>	config = ConfigObj()
>>	config.filename = 'new.ini'
>>	config['member1'] = value1
>>	config['section1'] = {}
>>	config['section1']['member1'] = value1
>>	...
>>
>>	config.write()
> 
> 
> I like this, I need this. :-)
> 

It's one of the nicest things about ConfigObj. It's also why I end up
using it for data persistence.

> 
> 
>>As it is to be written to a file - the only valid values to set are
>>strings, lists of strings, or dictionaries (containing strings or lists
>>of strings or...). Should we do type checking when the value is set ?
>>(not very difficult). This means the error would be raised when you *set
>>the value*, rather than when you try to write the file - which may be
>>another part of the program.
> 
> 
> Can't see the need for this limitation. The datatype may be any: a number,
> a flag, a date. Its *representation* is a string: it will be written to
> file, and when read back, the data will be converted back to its type, with
> validation.
> 
> Anything less than this is insufficient.
> 

Ok - we have either a confusion or a difficulty ;-)

Everything is written as a string - surprise, surprise. That means when
it's read back, it's a string again.

It sounds to me like you want the validation process to also do type
conversion ? (So that a value you set as an integer - although written
and read back in as a string, becomes an integer after validation).

Currently validation checks whether your read values (strings) conform
to whatever standard you set. It currently *doesn't* allow the
validation process to alter the stored values.

``validate.test`` returns a single True/False - pass/fail for each test.

I could also write a transforming test that returns a tuple of : (pass,
newvalue)

This would allow the validation process to also transform values for you.
It means two things :

    a) You must be allowed to set non string values on a ConfigObj
    b) You need a stringify option to transform all non-string values
when writing out

> 
> 
>>In ConfigObj 3 type checking was optional. There was also a
>>``stringify`` option. If set this automatically converts all values to
>>strings when writing. (So you can just assign integers etc without
>>having to do conversion yourself !).
>>
>>My preference is to do type checking and no longer provide the stringify
>>option.
> 
> 
> The type checking should be done when reading too: the data must conform,
> or be convertible, to the defined type, otherwise the config file is
> illegal. (Mmh, this all seems rather obvious; maybe you're talking about
> something else entirely?)
> 

The sort of type checking I'm talking about is when your program has a
bug and you accidentally set a value to be ``None`` or to be a class
instance by mistake. When you come to write out the file - we crash
because the value isn't a string.

If we *automatically* stringify non-string values, you won't see the
problem till much later.

If we allow the program to crash in the ``write`` method (as currently)
- this may actually be called from a totally different part of your
application that set the value - hard to track.

If we do type checking when you set the value - the code that sets the
value is the code that raises the error - easier to trace.

My view is that a ConfigObj represents the *config file*, and so it
makes sense to keep values as strings. Validation allows you to check
that your string does represent an integer, or represent a bool, or
whatever - but you do the conversion yourself.

You want (I think ?) the ConfigObj to represent the config data - so you
want the validate option to change the type of value from strings to
integers etc.

I'm quite happy to add to validate so it can transform values, add
*optional* type checking, and add a ``stringify`` option that is *on* by
default. This means ConfigObj will work with either model.

There will be a new keyword for the validate method - which will cause
it to *either* use ``val.test`` or ``val.transforming_test``.

> 
> 
>>2) Also in ConfigObj 3 - you could assign a new value to None - and it
>>would initialise the value to an empty section. You now do this with an
>>empty dictionary (which also worked in ConfigObj 3). I'm opting to lose
>>the option of setting a value to None to initialise it.
> 
> 
> I agree. "Explicit is better than implicit."
> 
> On the other hand, if you opt to lose the option, then you won't have the
> option anymore. ;-D
> 

It's hardly more work - and as you say, more explicit.

> 
> 
>>3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
>>Shall I reimplement this ?
> 
> 
> No, you shan't. ;-) Let them pound on the hash key. YAGNI rules. Less is
> better than more. All that jazz. ;-)
> 

YAGNI ?????
Anyway - cool.. I'll lose multiline comments.

> 
> 
>>4) ConfigObj 3 also preserved the comment at the start and end of a
>>file. These would be written out by the ``write`` method.
> 
> 
> That's useful, OTOH.
> 
> 

Ok - I'll keep it in the list.

> 
>>5) Should I attempt to preserve the amount of whitespace around the
>>divider, the type of quoting used, and the amount of whitespace before
>>the comment - and then re-use it in the write method ? I'd rather not....
> 
> 
> Absolutely not. Reformat mercilessly. After a while, the user will
> capitulate, and conform. ;-)
> 

lol - OTOH their nicely lined up config files will become all jagged,
but unavoidable really.

> 
> 
>>6) ConfigParser and ConfigObj allow some recursion in string
>>interpolation - a replaced value can also contain a value to replace.
>>This can be useful. In ConfigParser, you get a `Max recursion depth
>>exceeded' error if you get stuck in a loop. ConfigObj just ignores it.
>>Should it raise the error ? (I think it *should*)
> 
> 
> Yes, you should check it. OTOH, I'd rather drop the "feature". :-) But
> maybe that's taking YAGNI too far.
> 

Hmm... there's an option to turn it off.
It can be quite useful - and is in ConfigParser.

I'll put a maximum recursion depth error in.

> 
> 
>>7) In ConfigObj 3 you could specify the encoding of the file being read
>>and it would decode to unicode *before parsing*. I've removed support
>>for this and added ``decode`` and ``encode`` methods to convert to and
>>from unicode after parsing/before writing. This means you can only use
>>ascii compatible encodings in config files.
> 
> 
> Hooray, KISS rules too! ;-)
> 
> 
> 
>>Adding full unicode support
>>is possible - but requires the addition of two extra keywords (encoding
>>and backup_encoding). Seeing as UTF8 is a full unicode, ASCII compatible
>>encoding - I see no need to change yet.
> 
> 
> So leave it as it is. :-)
> 
> 
> 
>>>Then I'll revert to signing only, not may big secrets to protect anyway. ;-)
> 
> 
>>:-)
>>
>>Not yet.
> 
> 
> In Italy we say "Hope dies last." Do you have anything like that?
> 

Hmm... I can't think of anything directly equivalent. Their maybe
though. Ha - doesn't sound too hopeful though ;-)

(Their is a saying 'Hope springs eternal' which means something very
similar - but is quite old English).

> Better yet, I'm not even signing the email, the same as I (don't) do with
> all the other mailing lists.
> 

Ok

Cool - useful stuff.

Thanks

Fuzzyman
http://www.voidsapce.org.uk/python