Thread: [Rest2web-develop] Re: ConfigObj 4 work
Brought to you by:
mjfoord
Menu
▾
▴
rest2web-develop
|
From: Michael F. <mi...@pc...> - 2005-07-07 11:07:44
|
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Nicola Larosa wrote:
[snip..]
>
>>>I've added a ``SimpleVal`` object for doing dummy validation (achieves
>>>the same thing as the old 'partial configspec' - just checks members are
>>>present).
>>>
>>>I've added your iter methods into Section - so that it behaves minor an
>>>ordered dicitonary. A couple of slight changes you might want to use in
>>>odict - I use ``repr`` rather than ``str`` in the ``__repr__`` method,
>>>and ``popitem`` is also ordered.
>
>
> Rewrote __repr__, I like it better like this:
>
> def __repr__(self):
> items = []
> for key in self._keys:
> items.append('%s: %s' % (
> repr(key), repr(self[key])))
> return '{%s}' % ', '.join(items)
>
Yeah that's nicer.
>
>
>>>I've also started on the ``writein`` method - which will be a bit tricky
>>>to get right, but I think I know how I'll do it.
>>>
>>>If you get a chance - can you look at the ``TODO`` and ``ISSUES``
>>>section, and any questions in the source !
>
>
> That's next in line. :-) Let's switch all dev discussion to the
> rest2web-dev ML, I'm subscribed now.
>
Done :-)
I'll list the questions I have, answer any you can be bothered with :-)
1) Part of the point of ConfigObj is that you can create config files
from a program. Something like :
config = ConfigObj()
config.filename = 'new.ini'
config['member1'] = value1
config['section1'] = {}
config['section1']['member1'] = value1
...
config.write()
As it is to be written to a file - the only valid values to set are
strings, lists of strings, or dictionaries (containing strings or lists
of strings or...). Should we do type checking when the value is set ?
(not very difficult). This means the error would be raised when you *set
the value*, rather than when you try to write the file - which may be
another part of the program.
In ConfigObj 3 type checking was optional. There was also a
``stringify`` option. If set this automatically converts all values to
strings when writing. (So you can just assign integers etc without
having to do conversion yourself !).
My preference is to do type checking and no longer provide the stringify
option.
2) Also in ConfigObj 3 - you could assign a new value to None - and it
would initialise the value to an empty section. You now do this with an
empty dictionary (which also worked in ConfigObj 3). I'm opting to lose
the option of setting a value to None to initialise it.
3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
Shall I reimplement this ?
4) ConfigObj 3 also preserved the comment at the start and end of a
file. These would be written out by the ``write`` method.
5) Should I attempt to preserve the amount of whitespace around the
divider, the type of quoting used, and the amount of whitespace before
the comment - and then re-use it in the write method ? I'd rather not....
6) ConfigParser and ConfigObj allow some recursion in string
interpolation - a replaced value can also contain a value to replace.
This can be useful. In ConfigParser, you get a `Max recursion depth
exceeded' error if you get stuck in a loop. ConfigObj just ignores it.
Should it raise the error ? (I think it *should*)
7) In ConfigObj 3 you could specify the encoding of the file being read
and it would decode to unicode *before parsing*. I've removed support
for this and added ``decode`` and ``encode`` methods to convert to and
from unicode after parsing/before writing. This means you can only use
ascii compatible encodings in config files. Adding full unicode support
is possible - but requires the addition of two extra keywords (encoding
and backup_encoding). Seeing as UTF8 is a full unicode, ASCII compatible
encoding - I see no need to change yet.
That will do for the moment.....
>
>
>>>There are a couple of
>>>implementation decisions still to be made. e.g. should I prefix the
>>>private methods with a double underscore ?
>
>
> Single underscore is enough, I'd say.
>
Ok - I'll start that.
I've now added ``__all__`` and ``Section`` is not a public object.
>
>
>>>Once I've written ``writein`` - I'll preserve comments. I think I'll do
>>>this using a comments attribute for each section. I'm going to start by
>>>just preserving inline comments - you can preserve comments *above* a
>>>member by using the ``writein`` method ! Once these two features are
>>>done then it will basically be ready. Just the documentation !
>
>
> Are the docs for ConfigObj3, if any, still applicable, at least partially?
>
Probably large chunks are still applicable, but it *all* needs
rewriting. The ``validate.Validator`` class is poorly documented (and I
*didn't* write the example functions !).
>
>
>>>By the way - sending encrypted mail makes it difficult (not impossible)
>>>for me to receive mail at home (I use the PDA). When I leave on
>>>Saturday, it will make it impossible !
>
>
> Then I'll revert to signing only, not may big secrets to protect anyway. ;-)
>
:-)
Not yet.
Regards,
Fuzzy
http://www.voidspace.org.uk/python
>
> --
> Nicola Larosa - ni...@te...
>
> Adding things just because you can leads to monstrosities like Common LISP,
> PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
> (or better yet, by removing restrictions) leads to jewels like Python,
> Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFCzQz359Olk6wiv6IRAmkSAJ4l6ek5d2Yy7hWvshA8gUgZ+uo+2QCdETmY
UY1iW7V2i5nvZyIskgOaDv4=
=AZga
-----END PGP SIGNATURE-----
|
|
From: Michael F. <mi...@pc...> - 2005-07-07 11:53:24
|
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Oops - seems like administrator isn't automatically subscribed...
bizarre. Here's some stuff on ConfigObj 4.
Nicola Larosa wrote:
[snip..]
>
>>>I've added a ``SimpleVal`` object for doing dummy validation (achieves
>>>the same thing as the old 'partial configspec' - just checks members are
>>>present).
>>>
>>>I've added your iter methods into Section - so that it behaves minor an
>>>ordered dicitonary. A couple of slight changes you might want to use in
>>>odict - I use ``repr`` rather than ``str`` in the ``__repr__`` method,
>>>and ``popitem`` is also ordered.
>
>
> Rewrote __repr__, I like it better like this:
>
> def __repr__(self):
> items = []
> for key in self._keys:
> items.append('%s: %s' % (
> repr(key), repr(self[key])))
> return '{%s}' % ', '.join(items)
>
Yeah that's nicer.
>
>
>>>I've also started on the ``writein`` method - which will be a bit tricky
>>>to get right, but I think I know how I'll do it.
>>>
>>>If you get a chance - can you look at the ``TODO`` and ``ISSUES``
>>>section, and any questions in the source !
>
>
> That's next in line. :-) Let's switch all dev discussion to the
> rest2web-dev ML, I'm subscribed now.
>
Done :-)
I'll list the questions I have, answer any you can be bothered with :-)
1) Part of the point of ConfigObj is that you can create config files
from a program. Something like :
config = ConfigObj()
config.filename = 'new.ini'
config['member1'] = value1
config['section1'] = {}
config['section1']['member1'] = value1
...
config.write()
As it is to be written to a file - the only valid values to set are
strings, lists of strings, or dictionaries (containing strings or lists
of strings or...). Should we do type checking when the value is set ?
(not very difficult). This means the error would be raised when you *set
the value*, rather than when you try to write the file - which may be
another part of the program.
In ConfigObj 3 type checking was optional. There was also a
``stringify`` option. If set this automatically converts all values to
strings when writing. (So you can just assign integers etc without
having to do conversion yourself !).
My preference is to do type checking and no longer provide the stringify
option.
2) Also in ConfigObj 3 - you could assign a new value to None - and it
would initialise the value to an empty section. You now do this with an
empty dictionary (which also worked in ConfigObj 3). I'm opting to lose
the option of setting a value to None to initialise it.
3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
Shall I reimplement this ?
4) ConfigObj 3 also preserved the comment at the start and end of a
file. These would be written out by the ``write`` method.
5) Should I attempt to preserve the amount of whitespace around the
divider, the type of quoting used, and the amount of whitespace before
the comment - and then re-use it in the write method ? I'd rather not....
6) ConfigParser and ConfigObj allow some recursion in string
interpolation - a replaced value can also contain a value to replace.
This can be useful. In ConfigParser, you get a `Max recursion depth
exceeded' error if you get stuck in a loop. ConfigObj just ignores it.
Should it raise the error ? (I think it *should*)
7) In ConfigObj 3 you could specify the encoding of the file being read
and it would decode to unicode *before parsing*. I've removed support
for this and added ``decode`` and ``encode`` methods to convert to and
from unicode after parsing/before writing. This means you can only use
ascii compatible encodings in config files. Adding full unicode support
is possible - but requires the addition of two extra keywords (encoding
and backup_encoding). Seeing as UTF8 is a full unicode, ASCII compatible
encoding - I see no need to change yet.
That will do for the moment.....
>
>
>>>There are a couple of
>>>implementation decisions still to be made. e.g. should I prefix the
>>>private methods with a double underscore ?
>
>
> Single underscore is enough, I'd say.
>
Ok - I'll start that.
I've now added ``__all__`` and ``Section`` is not a public object.
>
>
>>>Once I've written ``writein`` - I'll preserve comments. I think I'll do
>>>this using a comments attribute for each section. I'm going to start by
>>>just preserving inline comments - you can preserve comments *above* a
>>>member by using the ``writein`` method ! Once these two features are
>>>done then it will basically be ready. Just the documentation !
>
>
> Are the docs for ConfigObj3, if any, still applicable, at least partially?
>
Probably large chunks are still applicable, but it *all* needs
rewriting. The ``validate.Validator`` class is poorly documented (and I
*didn't* write the example functions !).
>
>
>>>By the way - sending encrypted mail makes it difficult (not impossible)
>>>for me to receive mail at home (I use the PDA). When I leave on
>>>Saturday, it will make it impossible !
>
>
> Then I'll revert to signing only, not may big secrets to protect anyway. ;-)
>
:-)
Not yet.
Regards,
Fuzzy
http://www.voidspace.org.uk/python
>
> --
> Nicola Larosa - ni...@te...
>
> Adding things just because you can leads to monstrosities like Common LISP,
> PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
> (or better yet, by removing restrictions) leads to jewels like Python,
> Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFCzRem59Olk6wiv6IRAlWsAKDEaFrzneIZNfnjJGhViVVh9p+HHACghhof
chIKs5sIghLZ0psmN1lLmDA=
=///K
-----END PGP SIGNATURE-----
|
|
From: Michael F. <mi...@pc...> - 2005-07-07 12:08:58
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michael Foord wrote: > Oops - seems like administrator isn't automatically subscribed... > bizarre. Here's some stuff on ConfigObj 4. > [snip..] >>>>>There are a couple of >>>>>implementation decisions still to be made. e.g. should I prefix the >>>>>private methods with a double underscore ? >>> >>> >>>Single underscore is enough, I'd say. >>> > What about private attributes ? Like the regular expression class attributes in ConfigObj - should I prefix these with a single underscore as well ? Best Regards, Fuzzy http://www.voidspace.org.uk/python > > Ok - I'll start that. > I've now added ``__all__`` and ``Section`` is not a public object. > > > [snip..] >>>-- >>>Nicola Larosa - ni...@te... >>> >>>Adding things just because you can leads to monstrosities like Common LISP, >>>PL/I, Algol 68 and Perl 6. Adding features only when they add functionality >>>(or better yet, by removing restrictions) leads to jewels like Python, >>>Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005 >>> > - ------------------------------------------------------- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click _______________________________________________ Rest2web-develop mailing list Res...@li... https://lists.sourceforge.net/lists/listinfo/rest2web-develop -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCzRtU59Olk6wiv6IRAsXnAJ99/8S8wnGwpYdhkOrAqqIvWZQSxACeOzvj yt0gLClmIF2biGpr7p7dMqw= =ghfK -----END PGP SIGNATURE----- |
|
From: Nicola L. <ni...@te...> - 2005-07-07 12:58:59
|
> What about private attributes ? > Like the regular expression class attributes in ConfigObj - should I > prefix these with a single underscore as well ? Ideally, any instance member that is not to be externally and directly accessed should be, yes. It takes a fair amount of discipline to do that, but I think it's beneficial in the long run. -- Nicola Larosa - ni...@te... Adding things just because you can leads to monstrosities like Common LISP, PL/I, Algol 68 and Perl 6. Adding features only when they add functionality (or better yet, by removing restrictions) leads to jewels like Python, Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005 |
|
From: Nicola L. <ni...@te...> - 2005-07-07 12:57:03
|
> I'll list the questions I have, answer any you can be bothered with :-)
I'll try. :-)
> 1) Part of the point of ConfigObj is that you can create config files
> from a program. Something like :
>
> config = ConfigObj()
> config.filename = 'new.ini'
> config['member1'] = value1
> config['section1'] = {}
> config['section1']['member1'] = value1
> ...
>
> config.write()
I like this, I need this. :-)
> As it is to be written to a file - the only valid values to set are
> strings, lists of strings, or dictionaries (containing strings or lists
> of strings or...). Should we do type checking when the value is set ?
> (not very difficult). This means the error would be raised when you *set
> the value*, rather than when you try to write the file - which may be
> another part of the program.
Can't see the need for this limitation. The datatype may be any: a number,
a flag, a date. Its *representation* is a string: it will be written to
file, and when read back, the data will be converted back to its type, with
validation.
Anything less than this is insufficient.
> In ConfigObj 3 type checking was optional. There was also a
> ``stringify`` option. If set this automatically converts all values to
> strings when writing. (So you can just assign integers etc without
> having to do conversion yourself !).
>
> My preference is to do type checking and no longer provide the stringify
> option.
The type checking should be done when reading too: the data must conform,
or be convertible, to the defined type, otherwise the config file is
illegal. (Mmh, this all seems rather obvious; maybe you're talking about
something else entirely?)
> 2) Also in ConfigObj 3 - you could assign a new value to None - and it
> would initialise the value to an empty section. You now do this with an
> empty dictionary (which also worked in ConfigObj 3). I'm opting to lose
> the option of setting a value to None to initialise it.
I agree. "Explicit is better than implicit."
On the other hand, if you opt to lose the option, then you won't have the
option anymore. ;-D
> 3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
> Shall I reimplement this ?
No, you shan't. ;-) Let them pound on the hash key. YAGNI rules. Less is
better than more. All that jazz. ;-)
> 4) ConfigObj 3 also preserved the comment at the start and end of a
> file. These would be written out by the ``write`` method.
That's useful, OTOH.
> 5) Should I attempt to preserve the amount of whitespace around the
> divider, the type of quoting used, and the amount of whitespace before
> the comment - and then re-use it in the write method ? I'd rather not....
Absolutely not. Reformat mercilessly. After a while, the user will
capitulate, and conform. ;-)
> 6) ConfigParser and ConfigObj allow some recursion in string
> interpolation - a replaced value can also contain a value to replace.
> This can be useful. In ConfigParser, you get a `Max recursion depth
> exceeded' error if you get stuck in a loop. ConfigObj just ignores it.
> Should it raise the error ? (I think it *should*)
Yes, you should check it. OTOH, I'd rather drop the "feature". :-) But
maybe that's taking YAGNI too far.
> 7) In ConfigObj 3 you could specify the encoding of the file being read
> and it would decode to unicode *before parsing*. I've removed support
> for this and added ``decode`` and ``encode`` methods to convert to and
> from unicode after parsing/before writing. This means you can only use
> ascii compatible encodings in config files.
Hooray, KISS rules too! ;-)
> Adding full unicode support
> is possible - but requires the addition of two extra keywords (encoding
> and backup_encoding). Seeing as UTF8 is a full unicode, ASCII compatible
> encoding - I see no need to change yet.
So leave it as it is. :-)
>> Then I'll revert to signing only, not may big secrets to protect anyway. ;-)
> :-)
>
> Not yet.
In Italy we say "Hope dies last." Do you have anything like that?
Better yet, I'm not even signing the email, the same as I (don't) do with
all the other mailing lists.
--
Nicola Larosa - ni...@te...
Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005
|
|
From: Michael F. <mi...@pc...> - 2005-07-07 13:27:30
|
Hello Nicola,
(and anyone else :-)
Nicola Larosa wrote:
>>I'll list the questions I have, answer any you can be bothered with :-)
>
>
> I'll try. :-)
>
You did pretty well.
>
>
>>1) Part of the point of ConfigObj is that you can create config files
>>from a program. Something like :
>>
>> config = ConfigObj()
>> config.filename = 'new.ini'
>> config['member1'] = value1
>> config['section1'] = {}
>> config['section1']['member1'] = value1
>> ...
>>
>> config.write()
>
>
> I like this, I need this. :-)
>
It's one of the nicest things about ConfigObj. It's also why I end up
using it for data persistence.
>
>
>>As it is to be written to a file - the only valid values to set are
>>strings, lists of strings, or dictionaries (containing strings or lists
>>of strings or...). Should we do type checking when the value is set ?
>>(not very difficult). This means the error would be raised when you *set
>>the value*, rather than when you try to write the file - which may be
>>another part of the program.
>
>
> Can't see the need for this limitation. The datatype may be any: a number,
> a flag, a date. Its *representation* is a string: it will be written to
> file, and when read back, the data will be converted back to its type, with
> validation.
>
> Anything less than this is insufficient.
>
Ok - we have either a confusion or a difficulty ;-)
Everything is written as a string - surprise, surprise. That means when
it's read back, it's a string again.
It sounds to me like you want the validation process to also do type
conversion ? (So that a value you set as an integer - although written
and read back in as a string, becomes an integer after validation).
Currently validation checks whether your read values (strings) conform
to whatever standard you set. It currently *doesn't* allow the
validation process to alter the stored values.
``validate.test`` returns a single True/False - pass/fail for each test.
I could also write a transforming test that returns a tuple of : (pass,
newvalue)
This would allow the validation process to also transform values for you.
It means two things :
a) You must be allowed to set non string values on a ConfigObj
b) You need a stringify option to transform all non-string values
when writing out
>
>
>>In ConfigObj 3 type checking was optional. There was also a
>>``stringify`` option. If set this automatically converts all values to
>>strings when writing. (So you can just assign integers etc without
>>having to do conversion yourself !).
>>
>>My preference is to do type checking and no longer provide the stringify
>>option.
>
>
> The type checking should be done when reading too: the data must conform,
> or be convertible, to the defined type, otherwise the config file is
> illegal. (Mmh, this all seems rather obvious; maybe you're talking about
> something else entirely?)
>
The sort of type checking I'm talking about is when your program has a
bug and you accidentally set a value to be ``None`` or to be a class
instance by mistake. When you come to write out the file - we crash
because the value isn't a string.
If we *automatically* stringify non-string values, you won't see the
problem till much later.
If we allow the program to crash in the ``write`` method (as currently)
- this may actually be called from a totally different part of your
application that set the value - hard to track.
If we do type checking when you set the value - the code that sets the
value is the code that raises the error - easier to trace.
My view is that a ConfigObj represents the *config file*, and so it
makes sense to keep values as strings. Validation allows you to check
that your string does represent an integer, or represent a bool, or
whatever - but you do the conversion yourself.
You want (I think ?) the ConfigObj to represent the config data - so you
want the validate option to change the type of value from strings to
integers etc.
I'm quite happy to add to validate so it can transform values, add
*optional* type checking, and add a ``stringify`` option that is *on* by
default. This means ConfigObj will work with either model.
There will be a new keyword for the validate method - which will cause
it to *either* use ``val.test`` or ``val.transforming_test``.
>
>
>>2) Also in ConfigObj 3 - you could assign a new value to None - and it
>>would initialise the value to an empty section. You now do this with an
>>empty dictionary (which also worked in ConfigObj 3). I'm opting to lose
>>the option of setting a value to None to initialise it.
>
>
> I agree. "Explicit is better than implicit."
>
> On the other hand, if you opt to lose the option, then you won't have the
> option anymore. ;-D
>
It's hardly more work - and as you say, more explicit.
>
>
>>3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
>>Shall I reimplement this ?
>
>
> No, you shan't. ;-) Let them pound on the hash key. YAGNI rules. Less is
> better than more. All that jazz. ;-)
>
YAGNI ?????
Anyway - cool.. I'll lose multiline comments.
>
>
>>4) ConfigObj 3 also preserved the comment at the start and end of a
>>file. These would be written out by the ``write`` method.
>
>
> That's useful, OTOH.
>
>
Ok - I'll keep it in the list.
>
>>5) Should I attempt to preserve the amount of whitespace around the
>>divider, the type of quoting used, and the amount of whitespace before
>>the comment - and then re-use it in the write method ? I'd rather not....
>
>
> Absolutely not. Reformat mercilessly. After a while, the user will
> capitulate, and conform. ;-)
>
lol - OTOH their nicely lined up config files will become all jagged,
but unavoidable really.
>
>
>>6) ConfigParser and ConfigObj allow some recursion in string
>>interpolation - a replaced value can also contain a value to replace.
>>This can be useful. In ConfigParser, you get a `Max recursion depth
>>exceeded' error if you get stuck in a loop. ConfigObj just ignores it.
>>Should it raise the error ? (I think it *should*)
>
>
> Yes, you should check it. OTOH, I'd rather drop the "feature". :-) But
> maybe that's taking YAGNI too far.
>
Hmm... there's an option to turn it off.
It can be quite useful - and is in ConfigParser.
I'll put a maximum recursion depth error in.
>
>
>>7) In ConfigObj 3 you could specify the encoding of the file being read
>>and it would decode to unicode *before parsing*. I've removed support
>>for this and added ``decode`` and ``encode`` methods to convert to and
>>from unicode after parsing/before writing. This means you can only use
>>ascii compatible encodings in config files.
>
>
> Hooray, KISS rules too! ;-)
>
>
>
>>Adding full unicode support
>>is possible - but requires the addition of two extra keywords (encoding
>>and backup_encoding). Seeing as UTF8 is a full unicode, ASCII compatible
>>encoding - I see no need to change yet.
>
>
> So leave it as it is. :-)
>
>
>
>>>Then I'll revert to signing only, not may big secrets to protect anyway. ;-)
>
>
>>:-)
>>
>>Not yet.
>
>
> In Italy we say "Hope dies last." Do you have anything like that?
>
Hmm... I can't think of anything directly equivalent. Their maybe
though. Ha - doesn't sound too hopeful though ;-)
(Their is a saying 'Hope springs eternal' which means something very
similar - but is quite old English).
> Better yet, I'm not even signing the email, the same as I (don't) do with
> all the other mailing lists.
>
Ok
Cool - useful stuff.
Thanks
Fuzzyman
http://www.voidsapce.org.uk/python
|
|
From: Nicola L. <ni...@te...> - 2005-07-07 13:58:00
|
> Ok - we have either a confusion or a difficulty ;-) > > Everything is written as a string - surprise, surprise. That means when > it's read back, it's a string again. > > It sounds to me like you want the validation process to also do type > conversion ? (So that a value you set as an integer - although written > and read back in as a string, becomes an integer after validation). That's correct. > Currently validation checks whether your read values (strings) conform > to whatever standard you set. It currently *doesn't* allow the > validation process to alter the stored values. > > ``validate.test`` returns a single True/False - pass/fail for each test. > > I could also write a transforming test that returns a tuple of : (pass, > newvalue) > > This would allow the validation process to also transform values for you. > It means two things : > > a) You must be allowed to set non string values on a ConfigObj > b) You need a stringify option to transform all non-string values > when writing out A bidirectional to-from-string typing system. I'm still vague about the schema that describes types. Maybe we could take a page from Python and let the types be dynamic, but then how do we recognize the type of a value in a string read from the config file? > The sort of type checking I'm talking about is when your program has a > bug and you accidentally set a value to be ``None`` or to be a class > instance by mistake. When you come to write out the file - we crash > because the value isn't a string. > > If we *automatically* stringify non-string values, you won't see the > problem till much later. If the schema specifies the type for params, you catch the problem right away. The stringification should only happen later, when writing to file, and shouldn't be burdened with type checking concerns. > If we allow the program to crash in the ``write`` method (as currently) > - this may actually be called from a totally different part of your > application that set the value - hard to track. > > If we do type checking when you set the value - the code that sets the > value is the code that raises the error - easier to trace. Indeed. > My view is that a ConfigObj represents the *config file*, and so it > makes sense to keep values as strings. Validation allows you to check > that your string does represent an integer, or represent a bool, or > whatever - but you do the conversion yourself. > > You want (I think ?) the ConfigObj to represent the config data - so you > want the validate option to change the type of value from strings to > integers etc. The config file is just a kind of backend, an implementation detail. It could be a database connection, a link to cyberspace, or a time-space machine that reaches back to Pico de la Mirandola. ;-) What I'm interested in is the config *data*, in fact. The config data should be accurate, and consistently typed. Such concerns are better handled in the config system, instead of diluting them in the main program. This of course complicates the config system. ZConfig, for instance, goes to great lengths to embed the config schema and types in the config objects themselves, reaching unheard-of levels of complexity. :-| > I'm quite happy to add to validate so it can transform values, add > *optional* type checking, and add a ``stringify`` option that is *on* by > default. This means ConfigObj will work with either model. > > There will be a new keyword for the validate method - which will cause > it to *either* use ``val.test`` or ``val.transforming_test``. So, how do you specify what the value types should be? You have talked about this before, but it's still not clear to me. >>> 3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety. >>> Shall I reimplement this ? >> No, you shan't. ;-) Let them pound on the hash key. YAGNI rules. Less is >> better than more. All that jazz. ;-) > YAGNI ????? You Ain't Gonna Need It. A popular acronym around Python circles. > Anyway - cool.. I'll lose multiline comments. >>> 5) Should I attempt to preserve the amount of whitespace around the >>> divider, the type of quoting used, and the amount of whitespace before >>> the comment - and then re-use it in the write method ? I'd rather not.... >> Absolutely not. Reformat mercilessly. After a while, the user will >> capitulate, and conform. ;-) > lol - OTOH their nicely lined up config files will become all jagged, > but unavoidable really. Why? I was thinking about the opposite outcome instead, a lesson in tidying up. :-) >>>> Then I'll revert to signing only, not may big secrets to protect anyway. ;-) >>> :-) >>> >>> Not yet. >> In Italy we say "Hope dies last." Do you have anything like that? > Hmm... I can't think of anything directly equivalent. Their maybe > though. Ha - doesn't sound too hopeful though ;-) Italians often manage to be catholic and sceptic at the same time. ;-) > Cool - useful stuff. :-) -- Nicola Larosa - ni...@te... Adding things just because you can leads to monstrosities like Common LISP, PL/I, Algol 68 and Perl 6. Adding features only when they add functionality (or better yet, by removing restrictions) leads to jewels like Python, Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005 |
|
From: Michael F. <mi...@pc...> - 2005-07-07 15:01:45
|
Nicola Larosa wrote:
>>Ok - we have either a confusion or a difficulty ;-)
>>
>>Everything is written as a string - surprise, surprise. That means when
>>it's read back, it's a string again.
>>
>>It sounds to me like you want the validation process to also do type
>>conversion ? (So that a value you set as an integer - although written
>>and read back in as a string, becomes an integer after validation).
>
>
> That's correct.
>
>
>
>>Currently validation checks whether your read values (strings) conform
>>to whatever standard you set. It currently *doesn't* allow the
>>validation process to alter the stored values.
>>
>>``validate.test`` returns a single True/False - pass/fail for each test.
>>
>>I could also write a transforming test that returns a tuple of : (pass,
>>newvalue)
>>
>>This would allow the validation process to also transform values for you.
>>It means two things :
>>
>> a) You must be allowed to set non string values on a ConfigObj
>> b) You need a stringify option to transform all non-string values
>>when writing out
>
>
> A bidirectional to-from-string typing system. I'm still vague about the
> schema that describes types. Maybe we could take a page from Python and let
> the types be dynamic, but then how do we recognize the type of a value in a
> string read from the config file?
>
I don't think we need to specify the type of the value stored - it ought
to be unambiguous in the light of the *purpose* of the value. i.e. a
'port number' is always going to be an integer, a 'user name' is always
a string etc. This means that all the validation schema needs to do is
check that the stored value *makes sense* (can be sensibly converted to
the required type).
I think this is the stage to explain the 'schema' system - using
``validate.py``. You can decide whether this meets your needs - or can
be made to meet your needs.
The configspec is a file (list of lines, dictionary, whatever) that is
very like the config file. For each member of the config file, you
specify (effectively) a function to do the testing.
e.g. (from the tests)
test1='range(30,50)'
test2='lower()'
test3='integer'
test4='alphanumeric'
The function can be specified with or without parameters. If there are
no parameters then the '()' is optional. ``val.test`` then maps the
value in the configspec to a real function call. The function specified
is passed the value (as a string *or* a list of strings when read from
the config file) - *plus* any parameters specified in the config spec.
This means that, from the example above, the member 'test1' must be an
integer value between 30 and 50.
``lower`` checks the value is lowercase, ``integer`` just checks that
the value is an integer etc.
So I *don't* think we need to store any type information in the config
file. Once you have written a general purpose set of functions that
check the values make sense - all you need to do is write the
configspec. The validate method then checks each value individually.
One way of doing it would be to have the functions return the converted
value *or* raise an exception. They would look something like this :
def range(value, min, max):
if isinstance(value, list):
raise Exception
if not value.isdigit():
raise Exception
value = int(value)
if not min <= value <= max:
raise Exception
return value
If the member is missing, or the test raises an Exception, then it's a
fail. Otherwise the original value is replaced with the returned new value.
So we don't need to specify any type info in the config file - the type
should be obvious from context.
How does this sound ?
We can work together on a general set of functions that will cover most
of your needs (the syntax already allows for your configspec to pass
keyword arguments or specify multiple tests for a value).
A lot of are written already - but I think they could be clearer.
>
>
>>The sort of type checking I'm talking about is when your program has a
>>bug and you accidentally set a value to be ``None`` or to be a class
>>instance by mistake. When you come to write out the file - we crash
>>because the value isn't a string.
>>
>>If we *automatically* stringify non-string values, you won't see the
>>problem till much later.
>
>
> If the schema specifies the type for params, you catch the problem right
> away. The stringification should only happen later, when writing to file,
> and shouldn't be burdened with type checking concerns.
>
Hmm.. I only envisaged using the schema (configspec) to check a file
once you have read it, although there's no reason you couldn't set the
values and then validate.
The schema isn't involved when you *set* individual values. You would
have to validate specifically afterwards.... (allowing validation of
individual values at the point when they are set would be messy - I was
only suggesting type checking in the context of all values being strings
for writing to a file).
>
>
>>If we allow the program to crash in the ``write`` method (as currently)
>>- this may actually be called from a totally different part of your
>>application that set the value - hard to track.
>>
>>If we do type checking when you set the value - the code that sets the
>>value is the code that raises the error - easier to trace.
>
>
> Indeed.
>
>
>
>>My view is that a ConfigObj represents the *config file*, and so it
>>makes sense to keep values as strings. Validation allows you to check
>>that your string does represent an integer, or represent a bool, or
>>whatever - but you do the conversion yourself.
>>
>>You want (I think ?) the ConfigObj to represent the config data - so you
>>want the validate option to change the type of value from strings to
>>integers etc.
>
>
> The config file is just a kind of backend, an implementation detail. It
> could be a database connection, a link to cyberspace, or a time-space
> machine that reaches back to Pico de la Mirandola. ;-) What I'm interested
> in is the config *data*, in fact. The config data should be accurate, and
> consistently typed. Such concerns are better handled in the config system,
> instead of diluting them in the main program.
>
The nice thing about using ``validate.py`` is that you can use the same
system to check values wherever they come from. Mark Andrews - who I
designed this with, wanted to use it to validate the same values stored
in a database and a file.
> This of course complicates the config system. ZConfig, for instance, goes
> to great lengths to embed the config schema and types in the config objects
> themselves, reaching unheard-of levels of complexity. :-|
>
Ha....
>
>
>>I'm quite happy to add to validate so it can transform values, add
>>*optional* type checking, and add a ``stringify`` option that is *on* by
>>default. This means ConfigObj will work with either model.
>>
>>There will be a new keyword for the validate method - which will cause
>>it to *either* use ``val.test`` or ``val.transforming_test``.
>
>
> So, how do you specify what the value types should be? You have talked
> about this before, but it's still not clear to me.
>
*Hopefully* I've answered this above.
>
>
[snip..]
>
>>>>5) Should I attempt to preserve the amount of whitespace around the
>>>>divider, the type of quoting used, and the amount of whitespace before
>>>>the comment - and then re-use it in the write method ? I'd rather not....
>
>
>>>Absolutely not. Reformat mercilessly. After a while, the user will
>>>capitulate, and conform. ;-)
>
>
>>lol - OTOH their nicely lined up config files will become all jagged,
>>but unavoidable really.
>
>
> Why? I was thinking about the opposite outcome instead, a lesson in tidying
> up. :-)
>
Well, because if you add white space to your values to line up the
comments etc.... then the write method will simply add a standard amount
of whitespace after the value and before the comment.
It will also write a standard amount of whitespace between value and
divider, rather than padding it so that they line up.
Best Regards,
Fuzzy
http://www.voidspace.org.uk/python
|
|
From: Nicola L. <ni...@te...> - 2005-07-08 08:52:06
|
> I don't think we need to specify the type of the value stored - it ought > to be unambiguous in the light of the *purpose* of the value. i.e. a > 'port number' is always going to be an integer, a 'user name' is always > a string etc. This means that all the validation schema needs to do is > check that the stored value *makes sense* (can be sensibly converted to > the required type). In other words, "duck typing". Very Pythonic. :-) > I think this is the stage to explain the 'schema' system - using > ``validate.py``. You can decide whether this meets your needs - or can > be made to meet your needs. > > The configspec is a file (list of lines, dictionary, whatever) that is > very like the config file. For each member of the config file, you > specify (effectively) a function to do the testing. > > e.g. (from the tests) > > test1='range(30,50)' > test2='lower()' > test3='integer' > test4='alphanumeric' > > The function can be specified with or without parameters. If there are > no parameters then the '()' is optional. ``val.test`` then maps the > value in the configspec to a real function call. The function specified > is passed the value (as a string *or* a list of strings when read from > the config file) - *plus* any parameters specified in the config spec. > > This means that, from the example above, the member 'test1' must be an > integer value between 30 and 50. > > ``lower`` checks the value is lowercase, ``integer`` just checks that > the value is an integer etc. > > So I *don't* think we need to store any type information in the config > file. Once you have written a general purpose set of functions that > check the values make sense - all you need to do is write the > configspec. The validate method then checks each value individually. > > One way of doing it would be to have the functions return the converted > value *or* raise an exception. They would look something like this : > > def range(value, min, max): > if isinstance(value, list): > raise Exception > if not value.isdigit(): > raise Exception > value = int(value) > if not min <= value <= max: > raise Exception > return value > > If the member is missing, or the test raises an Exception, then it's a > fail. Otherwise the original value is replaced with the returned new value. > > So we don't need to specify any type info in the config file - the type > should be obvious from context. Nobody said that. ;-) I do implied that we could have a some kind of schema file, where the types would be specified. That's what I know, because that's what's ZConfig has. Your configspec file looks even better, though, since it contains constraints on the possible values. > How does this sound ? > We can work together on a general set of functions that will cover most > of your needs (the syntax already allows for your configspec to pass > keyword arguments or specify multiple tests for a value). > > A lot of are written already - but I think they could be clearer. >> If the schema specifies the type for params, you catch the problem right >> away. The stringification should only happen later, when writing to file, >> and shouldn't be burdened with type checking concerns. > Hmm.. I only envisaged using the schema (configspec) to check a file > once you have read it, although there's no reason you couldn't set the > values and then validate. > > The schema isn't involved when you *set* individual values. You would > have to validate specifically afterwards.... (allowing validation of > individual values at the point when they are set would be messy - I was > only suggesting type checking in the context of all values being strings > for writing to a file). OTOH, checking only when saving means disconnecting the check point from the set point. Unfortunately, checking at set point means keeping constraints in the same object holding the data, a firm step towards ZConfig-level complexity. :-) So yes, I think it's wise dropping such a requirement. >>> If we allow the program to crash in the ``write`` method (as currently) >>> - this may actually be called from a totally different part of your >>> application that set the value - hard to track. >>> >>> If we do type checking when you set the value - the code that sets the >>> value is the code that raises the error - easier to trace. >> Indeed. And this is what we just dropped. :-( > The nice thing about using ``validate.py`` is that you can use the same > system to check values wherever they come from. Or wherever they go to. :-) > Mark Andrews - who I > designed this with, wanted to use it to validate the same values stored > in a database and a file. >>> lol - OTOH their nicely lined up config files will become all jagged, >>> but unavoidable really. >> Why? I was thinking about the opposite outcome instead, a lesson in tidying >> up. :-) > Well, because if you add white space to your values to line up the > comments etc.... then the write method will simply add a standard amount > of whitespace after the value and before the comment. > > It will also write a standard amount of whitespace between value and > divider, rather than padding it so that they line up. I reckon there's a specific mention in GvR guidelines to not depend on such aligning. Not that I hope that many users of ConfigObj will have read them. ;-) -- Nicola Larosa - ni...@te... Adding things just because you can leads to monstrosities like Common LISP, PL/I, Algol 68 and Perl 6. Adding features only when they add functionality (or better yet, by removing restrictions) leads to jewels like Python, Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005 |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X