configobj-develop — Support and development of ConfigObj

Re: [Configobj-develop] Re: ConfigObj unicode Support in SVN

[Nicola L. <ni...@te...>]

>> My preference would be the ability to specify a file object to
>> cfg_obj.write().

> So you'd like to be able to pass a file like object to ``write()``...
> hmmm...
> 
> Again, this is something that you used to be able to do with ConfigObj
> 3, but I removed to reduce the level of complexity.
> 
> To be symmetric with initialisation, you ought then be able to pass in a
> filename as well. Seeing as you can already achieve this just be setting
> the ``filename`` attribute (which ok sounds a little weird for this
> purpose), I decided not to do it.
> 
> Hmmm... I guess it wouldn't hurt being able to pass an optional file
> object to write, it's just that the API keeps growing...
> 
> Nicola, do you have any take on this ?

Well, it actually feels a little awkward that ConfigObj is in the business
of opening files, and having to store the filename in an attribute. It
seems best that ``write`` just receives a file-like object, and be done
with it.

This seems a worthy addition to me. At this point you probably have to keep
the filename attribute for compatibility anyway, but maybe you could even
find a way to deprecate it? :-)

-- 
Nicola Larosa - http://www.tekNico.net/

Using threads is like having no backups: you stay in denial
of the problem until it bites you, and then it's too late.
 -- Nicola Larosa, November 2005

Re: [Configobj-develop] Re: ConfigObj unicode Support in SVN

[Michael F. <fuz...@vo...>]

Aaron Bentley wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Fuzzyman wrote:
>   
>> Aaron Bentley wrote:
>>
>> Michael Foord wrote:
>> | I would really appreciate it if you could check this out and try and
>> | break it. Unicode is notoriously fiddly to get right !
>>
>> This all looks good.  I will try converting bzr to the latest version
>> soon, and let you know.
>>
>>     
>>> Great.
>>>       
>
> I have done this, and I was rather surprised that the output of
> cfg_obj.write() is a list of unicode strings, rather than a list of
> bytestrings.
>
>   
Ok - so if an encoding is specified, then ``write()`` should always 
return byte strings. That is what is called an 'over-sight'. :-)

Will be fixed.

> We need to supply our own file objects, so we've been using the lines
> output from cfg_obj.write() up to this point.  The StringIO support
> looks awkward, but probably doable to me.  (Am I right that the public
> member for the StringIO is ConfigObj.filename?)
>
>   
Yes.

> My preference would be the ability to specify a file object to
> cfg_obj.write().
>   
So you'd like to be able to pass a file like object to ``write()``... 
hmmm...

Again, this is something that you used to be able to do with ConfigObj 
3, but I removed to reduce the level of complexity.

To be symmetric with initialisation, you ought then be able to pass in a 
filename as well. Seeing as you can already achieve this just be setting 
the ``filename`` attribute (which ok sounds a little weird for this 
purpose), I decided not to do it.

Hmmm... I guess it wouldn't hurt being able to pass an optional file 
object to write, it's just that the API keeps growing...

Nicola, do you have any take on this ?

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml
> Aaron
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
>
> iD8DBQFD4kxA0F+nu1YWqI0RAgoLAJ9HlNNJ0bTOm4xPEL3NXzeGzEvAeQCdEJw5
> FSyN2JEIBcKvT3lPHW2Zo/E=
> =gKuf
> -----END PGP SIGNATURE-----
>
>   

Re: [Configobj-develop] Re: ConfigObj unicode Support in SVN

[Aaron B. <aar...@ut...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Fuzzyman wrote:
> Aaron Bentley wrote:
> 
> Michael Foord wrote:
> | I would really appreciate it if you could check this out and try and
> | break it. Unicode is notoriously fiddly to get right !
> 
> This all looks good.  I will try converting bzr to the latest version
> soon, and let you know.
> 
>> Great.

I have done this, and I was rather surprised that the output of
cfg_obj.write() is a list of unicode strings, rather than a list of
bytestrings.

We need to supply our own file objects, so we've been using the lines
output from cfg_obj.write() up to this point.  The StringIO support
looks awkward, but probably doable to me.  (Am I right that the public
member for the StringIO is ConfigObj.filename?)

My preference would be the ability to specify a file object to
cfg_obj.write().

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFD4kxA0F+nu1YWqI0RAgoLAJ9HlNNJ0bTOm4xPEL3NXzeGzEvAeQCdEJw5
FSyN2JEIBcKvT3lPHW2Zo/E=
=gKuf
-----END PGP SIGNATURE-----

Re: [Configobj-develop] Re: ConfigObj unicode Support in SVN

[Fuzzyman <fuz...@vo...>]

Aaron Bentley wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Michael Foord wrote:
> | I would really appreciate it if you could check this out and try and
> | break it. Unicode is notoriously fiddly to get right !
>
> This all looks good.  I will try converting bzr to the latest version
> soon, and let you know.
>
Great.

> | There is still a question about automatic support for UTF8 files with a
> | BOM. The new position is that UTF8 files with a BOM will be
> | automatically decoded to unicode.
> |
> | I have also proposed a *better* way of handling the BOM.
>
> I like the idea of BOM being a boolean, if that's what you mean.
>
Ok, it's actually simpler than what I've currently implemented and is 
more sensible really.

The other question is, if ConfigObj is passed a UTF8 file with a BOM, 
should it automatically decode to unicode ?

The behaviour of ConfigObj 4.1.0 (and previous) was to preserve the BOM 
(UTF8 only) but not decode. On balance it is probably better *not* to 
decode (ConfigObj doesn't attempt to detect the encoding of other 
files). For UTF16 files it must automatically decode - otherwise it 
splits characters on byte boundaries which mangles them.

So if BOM is True, but no encoding is specified, then a UTF8 BOM will be 
written out. Otherwise a BOM will only be written out if BOM is True and 
the encoding is UTF8 or UTF16.


All the best,


Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> Aaron
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
>
> iD8DBQFD4YK10F+nu1YWqI0RApP8AJ0S+tbZ3V+oYUXL1+MrKwnQQpEenQCghLVJ
> 7TmgjVEsz55qNapITDocMD4=
> =UHuk
> -----END PGP SIGNATURE-----
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log 
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop
>

[Configobj-develop] Re: ConfigObj unicode Support in SVN

[Aaron B. <aar...@ut...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael Foord wrote:
| I would really appreciate it if you could check this out and try and
| break it. Unicode is notoriously fiddly to get right !

This all looks good.  I will try converting bzr to the latest version
soon, and let you know.

| There is still a question about automatic support for UTF8 files with a
| BOM. The new position is that UTF8 files with a BOM will be
| automatically decoded to unicode.
|
| I have also proposed a *better* way of handling the BOM.

I like the idea of BOM being a boolean, if that's what you mean.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFD4YK10F+nu1YWqI0RApP8AJ0S+tbZ3V+oYUXL1+MrKwnQQpEenQCghLVJ
7TmgjVEsz55qNapITDocMD4=
=UHuk
-----END PGP SIGNATURE-----

[Configobj-develop] ConfigObj unicode Support in SVN

[Michael F. <fuz...@vo...>]

Hello all,

The update to ConfigObj is now in subversion. In 
https://svn.rest2web.python-hosting.com/branches/configobj4/pythonutils/

This contains the new methods :

    * as_bool
    * as_int
    * as_float

istrue is now deprecated with a warning.

It also contains the new unicode support. This was easy enough to 
implement, and works on my tests with a UTF16 config file.

I would really appreciate it if you could check this out and try and 
break it. Unicode is notoriously fiddly to get right !

There is still a question about automatic support for UTF8 files with a 
BOM. The new position is that UTF8 files with a BOM will be 
automatically decoded to unicode.

I have also proposed a *better* way of handling the BOM.

See my blog entry on the subject : 
http://www.voidspace.org.uk/python/weblog/arch_d7_2006_01_28.shtml#e204

Opinions and comments sought.

Thanks

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

[Configobj-develop] Unicode Support in ConfigObj

[Michael F. <fuz...@vo...>]

Hello all,

I'm adding unicode support to ConfigObj.

This is *basically* complete - just a couple of troublesome uses of  
``str()`` in validate to nail.

The current situation, is that if you supply a UTF8 or UTF16 file with a 
BOM, then ConfigObj will autodetect the encoding and decode appropriately.

For UTF16 this is essential, ConfigObj will mangle a byte string with a 
16bit encoding unless it decodes first.

For UTF8 this is possibly not what you want, you might want to keep UTF8 
byte-strings. (Some applications attempt to avoid unicode issues by 
retaining UTF8 encoded byte strings throughout.) You can get round this 
by parsing the config file, detecting that the encoding attribute has 
been set, and then calling ``encode``.

from configobj import ConfigObj
cfg = ConfigObj(cfg_file)
if cfg.encoding == 'utf_8':
    cfg.encode('utf_8')
    cfg.encoding = None

Is this adequate, or should the default behaviour for ConfigObj remain 
*ignoring* the encoding (in the case of UTF8) except to preserve the BOM ?

All the best,


Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Re: [Configobj-develop] Re: ConfigObj Changes

[Michael F. <fuz...@vo...>]

Fuzzyman wrote:
[snip..]
> Done. :-)
>
> I'll set about implementing :
>
>     as_bool
>     as_int
>     as_float
>
>   
Uhmm... that was embarrassingly easy, rename ``istrue`` to ``as_bool``
and new section methods :

    def as_int(self, key):
        return int(self[key])

    def as_float(self, key):
        return float(self[key])

Have I missed anything (they ought to raise a ValueError for invalid
values).

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml
> I'm part way through adding unicode support. Things like checking the
> compiled (ascii) regular expressions handle unicode fine and fuller BOM
> support make this not-quite-trivial-but-not-too-bad.
>
> All the best,
>
>
> Fuzzyman
> http://www.voidspace.org.uk/python/index.shtml
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop
>
>   

Re: [Configobj-develop] Re: ConfigObj Changes

[Fuzzyman <fuz...@vo...>]

Nicola Larosa wrote:
>>> I think I'll disagree here. I concur that having to specify None for no
>>> subsection is a bit inconvenient. However, breaking the rule that optional
>>> parameters should be last in the signature is worse, IMO. I seem to recall
>>> a couple such cases in the standard library, and they feel out of place and
>>> inconsistent with the rest.
>>>
>>> Moreover, with option 3) you're not really compatible with ConfigParser
>>> anymore, since you changed the signature, by allowing one to use just one
>>> param. That may cause surprises in case of mistakes.
>>>
>>> Maybe it would be better to have separate methods with just one param, and
>>> that would be option 1), if I'm not mistaken.
>>>       
>
>   
>> I can see your point.
>>
>> *However* there is a current discussion of ConfigObj going on in
>> python-dev. Ian Bicking mentions about having *too much* in the API.
>> Implementing two means of doing similar things seems like overkill.
>>     
>
> That's also true, I agree that the API should be as frugal as possible.
>
>
>   
>> Do you greatly prefer ``getboolean`` (allowing but requiring you to
>> specify a sub-section) to ``asboolean`` (not requiring or allowing you
>> to specify a sub-section) ?
>>     
>
> No, I don't. It only seems useful for ConfigParser compatibility,
> subsect.as_bool(key) (I'd like it shorter and clearer) is simpler and
> functional.
>
>
>   
>> Just implementing ``asboolean`` leaves the way open for ``getboolean``
>> in a subclass (implementation of either is trivial).
>>     
>
> Then option 2) it is.
>
>   
Done. :-)

I'll set about implementing :

    as_bool
    as_int
    as_float

I'm part way through adding unicode support. Things like checking the
compiled (ascii) regular expressions handle unicode fine and fuller BOM
support make this not-quite-trivial-but-not-too-bad.

All the best,


Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Re: [Configobj-develop] Re: ConfigObj Changes

[Nicola L. <ni...@te...>]

>> I think I'll disagree here. I concur that having to specify None for no
>> subsection is a bit inconvenient. However, breaking the rule that optional
>> parameters should be last in the signature is worse, IMO. I seem to recall
>> a couple such cases in the standard library, and they feel out of place and
>> inconsistent with the rest.
>> 
>> Moreover, with option 3) you're not really compatible with ConfigParser
>> anymore, since you changed the signature, by allowing one to use just one
>> param. That may cause surprises in case of mistakes.
>> 
>> Maybe it would be better to have separate methods with just one param, and
>> that would be option 1), if I'm not mistaken.

> I can see your point.
> 
> *However* there is a current discussion of ConfigObj going on in
> python-dev. Ian Bicking mentions about having *too much* in the API.
> Implementing two means of doing similar things seems like overkill.

That's also true, I agree that the API should be as frugal as possible.


> Do you greatly prefer ``getboolean`` (allowing but requiring you to
> specify a sub-section) to ``asboolean`` (not requiring or allowing you
> to specify a sub-section) ?

No, I don't. It only seems useful for ConfigParser compatibility,
subsect.as_bool(key) (I'd like it shorter and clearer) is simpler and
functional.


> Just implementing ``asboolean`` leaves the way open for ``getboolean``
> in a subclass (implementation of either is trivial).

Then option 2) it is.

-- 
Nicola Larosa - http://www.tekNico.net/

People just don't want to be pestered - they want shit to happen
without being bugged. If technology is to help, it must be seren-
dipitous rather than strident. -- Nathan Torkington, November 2005

Re: [Configobj-develop] Re: ConfigObj Changes

[Fuzzyman <fuz...@vo...>]

Nicola Larosa wrote:
>>> I am going to implement these as section methods. My initial idea was
>>> to implement them using the ConfigParser names and syntax. This means
>>> you would have to specify a subsection, but could specify None to get
>>> values in the section you are calling from.
>>>
>>> In order to avoid this there are three (reasonable) options :
>>>
>>> 1) Implement ``asbool`` *and* ``getbool`` methods - ``getbool``
>>> following ConfigParser signature.
>>> 2) Just implement ``asbool`` and leave ``getbool`` for a ConfigParser
>>> compatibility layer - if it ever happens.
>>> 3) Implement ``getbool`` but make the second argument optional (by
>>> having None as the default). If no second argument is specified, treat
>>> the first argument as a key *not* a section...
>>>
>>> I have a mild preference for 2) but quite like 3). 3) is weird because
>>> it changes the meaning of the first argument *if* you provide a second...
>>>
>>> Opinions please...
>>>       
>
>   
>> Actually I like 3. It keeps compatibility with ConfigParser (benefit of
>> familiarity), and achieves both aims.
>>
>> If you supply one argument it's a key, if you supply two they are
>> ``section, key``.
>>
>> Oh, except it's ``getboolean`` not ``getbool``. :-)
>>     
>
> I think I'll disagree here. I concur that having to specify None for no
> subsection is a bit inconvenient. However, breaking the rule that optional
> parameters should be last in the signature is worse, IMO. I seem to recall
> a couple such cases in the standard library, and they feel out of place and
> inconsistent with the rest.
>
> Moreover, with option 3) you're not really compatible with ConfigParser
> anymore, since you changed the signature, by allowing one to use just one
> param. That may cause surprises in case of mistakes.
>
> Maybe it would be better to have separate methods with just one param, and
> that would be option 1), if I'm not mistaken.
>
>   
I can see your point.

*However* there is a current discussion of ConfigObj going on in
python-dev. Ian Bicking mentions about having *too much* in the API.
Implementing two means of doing similar things seems like overkill.

Do you greatly prefer ``getboolean`` (allowing but requiring you to
specify a sub-section) to ``asboolean`` (not requiring or allowing you
to specify a sub-section) ?

Just implementing ``asboolean`` leaves the way open for ``getboolean``
in a subclass (implementation of either is trivial).

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Re: [Configobj-develop] Re: ConfigObj Changes

[Nicola L. <ni...@te...>]

>> I am going to implement these as section methods. My initial idea was
>> to implement them using the ConfigParser names and syntax. This means
>> you would have to specify a subsection, but could specify None to get
>> values in the section you are calling from.
>> 
>> In order to avoid this there are three (reasonable) options :
>> 
>> 1) Implement ``asbool`` *and* ``getbool`` methods - ``getbool``
>> following ConfigParser signature.
>> 2) Just implement ``asbool`` and leave ``getbool`` for a ConfigParser
>> compatibility layer - if it ever happens.
>> 3) Implement ``getbool`` but make the second argument optional (by
>> having None as the default). If no second argument is specified, treat
>> the first argument as a key *not* a section...
>>
>> I have a mild preference for 2) but quite like 3). 3) is weird because
>> it changes the meaning of the first argument *if* you provide a second...
>>
>> Opinions please...

> Actually I like 3. It keeps compatibility with ConfigParser (benefit of
> familiarity), and achieves both aims.
> 
> If you supply one argument it's a key, if you supply two they are
> ``section, key``.
> 
> Oh, except it's ``getboolean`` not ``getbool``. :-)

I think I'll disagree here. I concur that having to specify None for no
subsection is a bit inconvenient. However, breaking the rule that optional
parameters should be last in the signature is worse, IMO. I seem to recall
a couple such cases in the standard library, and they feel out of place and
inconsistent with the rest.

Moreover, with option 3) you're not really compatible with ConfigParser
anymore, since you changed the signature, by allowing one to use just one
param. That may cause surprises in case of mistakes.

Maybe it would be better to have separate methods with just one param, and
that would be option 1), if I'm not mistaken.

-- 
Nicola Larosa - http://www.tekNico.net/

People just don't want to be pestered - they want shit to happen
without being bugged. If technology is to help, it must be seren-
dipitous rather than strident. -- Nathan Torkington, November 2005

Re: [Configobj-develop] Re: ConfigObj Changes

[Fuzzyman <fuz...@vo...>]

Fuzzyman wrote:
> Aaron Bentley wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Fuzzyman wrote:
>>   
>>> * To reverse the order of the arguments, with section defaulting to
>>> None. This leads to confusion in the (theoretical) ConfigParser
>>> compatibility layer, which will of course  be reversed back. It is
>>> *slightly* more friendly for normal use though.
>>>     
>>
>> For values in a subsection, we'd also be able to do:
>> val=cfg['subsection'].getboolean('key'), right?  That's more in keeping
>> with your existing UI.  Perhaps you'd prefer to use different names from
>> the ConfigParser ones.  ('asbool', 'asfloat'?)
>>
>>   
> I'm copying this to the configobj-develop mailing list because I value
> Nicola's input on this. Perhaps you'd consider joining the list Aaron
> (low traffic - via sourceforge).
>
> I am going to implement these as section methods. My initial idea was
> to implement them using the ConfigParser names and syntax. This means
> you would have to specify a subsection, but could specify None to get
> values in the section you are calling from.
>
> In order to avoid this there are three (reasonable) options :
>
> 1) Implement ``asbool`` *and* ``getbool`` methods - ``getbool``
> following ConfigParser signature.
> 2) Just implement ``asbool`` and leave ``getbool`` for a ConfigParser
> compatibility layer - if it ever happens.
> 3) Implement ``getbool`` but make the second argument optional (by
> having None as the default). If no second argument is specified, treat
> the first argument as a key *not* a section...
>
> I have a mild preference for 2) but quite like 3). 3) is weird because
> it changes the meaning of the first argument *if* you provide a second...
>
> Opinions please...
>
Actually I like 3. It keeps compatibility with ConfigParser (benefit of
familiarity), and achieves both aims.

If you supply one argument it's a key, if you supply two they are
``section, key``.

Oh, except it's ``getboolean`` not ``getbool``. :-)

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

>> I don't think we'll be exposing ConfigObj directly, so I don't think
>> this is a big deal for us, either way.
>>
>>   
>>> Should I then remove ``istrue`` (added in 4.1.0) which is effectively
>>> the same as ``getboolean`` ? I would deprecate it and add a warning
>>> until 4.3.0.
>>>     
>>
>> I have no personal stake in istrue.
>>
>>   
> Don't start using it then. :-)
>
> All the best,
>
> Fuzzyman
> http://www.voidspace.org.uk/python/index.shtml

[Configobj-develop] Re: ConfigObj Changes

[Fuzzyman <fuz...@vo...>]

Aaron Bentley wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Fuzzyman wrote:
>   
>> * To reverse the order of the arguments, with section defaulting to
>> None. This leads to confusion in the (theoretical) ConfigParser
>> compatibility layer, which will of course  be reversed back. It is
>> *slightly* more friendly for normal use though.
>>     
>
> For values in a subsection, we'd also be able to do:
> val=cfg['subsection'].getboolean('key'), right?  That's more in keeping
> with your existing UI.  Perhaps you'd prefer to use different names from
> the ConfigParser ones.  ('asbool', 'asfloat'?)
>
>   
I'm copying this to the configobj-develop mailing list because I value 
Nicola's input on this. Perhaps you'd consider joining the list Aaron 
(low traffic - via sourceforge).

I am going to implement these as section methods. My initial idea was to 
implement them using the ConfigParser names and syntax. This means you 
would have to specify a subsection, but could specify None to get values 
in the section you are calling from.

In order to avoid this there are three (reasonable) options :

1) Implement ``asbool`` *and* ``getbool`` methods - ``getbool`` 
following ConfigParser signature.
2) Just implement ``asbool`` and leave ``getbool`` for a ConfigParser 
compatibility layer - if it ever happens.
3) Implement ``getbool`` but make the second argument optional (by 
having None as the default). If no second argument is specified, treat 
the first argument as a key *not* a section...

I have a mild preference for 2) but quite like 3). 3) is weird because 
it changes the meaning of the first argument *if* you provide a second...

Opinions please...

> I don't think we'll be exposing ConfigObj directly, so I don't think
> this is a big deal for us, either way.
>
>   
>> Should I then remove ``istrue`` (added in 4.1.0) which is effectively
>> the same as ``getboolean`` ? I would deprecate it and add a warning
>> until 4.3.0.
>>     
>
> I have no personal stake in istrue.
>
>   
Don't start using it then. :-)

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Re: [Configobj-develop] ConfigObj Changes

[Fuzzyman <fuz...@vo...>]

Nicola Larosa wrote:
> [snip..]
>> * To make it a method on ConfigObj (*not* sections) and allow section to
>> be a string *or* a reference to a subsection - in which case the
>> specified key would be taken from that.
>>     
>
> How would you specify nested sections in a string? Dotted notation? Section
> names are not restricted to Python identifier syntax, right? Anyway, having
> methods on section objects seems more versatile.
>
>   
It sounds like a section method with ``None`` for the section (if not 
referencing a key in a sub-section) is the way. (Consider it decided 
unless someone shouts.)

*However*, what I meant here is that you pass in a string to specify a 
section one level down. To specify a nested section you pass in a 
*reference* to the section - not a string at all.

I don't come from a static language background, so I'm agnostic about 
arguments that can be of more than one type. In some circles I believe 
this is frowned upon as bad practise...

>   
>> Should I then remove ``istrue`` (added in 4.1.0) which is effectively
>> the same as ``getboolean`` ? I would deprecate it and add a warning
>> until 4.3.0.
>>     
>
> Yes, better avoid multiple ways of doing the same thing, especially if it's
> a recent thing.
>
>   
Cool - I added it in 4.1.0, but ``getboolean`` is better anyway.

Thanks

Re: [Configobj-develop] ConfigObj Changes

[Nicola L. <ni...@te...>]

> I *won't* implement get because it clashes with a dictionary method. If
> I implement the ``ConfigParser`` compatibility layer then that will
> change (but be on a subclass).

OK.


> My intention is to make these methods *section* methods, so they are
> available on all subsections, and to keep the ConfigParser method
> signature. For values *not* in a subsection, this leads to the slightly
> awkward syntax :
> 
> cfg = ConfigObj(cfg_file)
> val = cfg.getboolean(None, 'key')

There's no explicit outer section, so None is an appropriate value, if a
bit quirky.


> The alternatives are :
> 
> * To reverse the order of the arguments, with section defaulting to
> None. This leads to confusion in the (theoretical) ConfigParser
> compatibility layer, which will of course  be reversed back. It is
> *slightly* more friendly for normal use though.

I don't like it, equivalent APIs should be consistent, otherwise it's like
setting up traps for the unwary user,


> * To make it a method on ConfigObj (*not* sections) and allow section to
> be a string *or* a reference to a subsection - in which case the
> specified key would be taken from that.

How would you specify nested sections in a string? Dotted notation? Section
names are not restricted to Python identifier syntax, right? Anyway, having
methods on section objects seems more versatile.


> Should I then remove ``istrue`` (added in 4.1.0) which is effectively
> the same as ``getboolean`` ? I would deprecate it and add a warning
> until 4.3.0.

Yes, better avoid multiple ways of doing the same thing, especially if it's
a recent thing.


> The unicode changes are reasonably straightforward. I'll re-implement
> the system used in ConfigObj 3. (Unless anyone raises any objections or
> suggests an alternative).
> 
>  You will be able to pass in an (optional)  'encoding' keyword. If
> present, this will be used to decode the input. This maps to an
> 'encoding' attribute which will be used used to encode when writing (and
> can obviously be changed).
> 
> There will also be a 'default_encoding' keyword/attribute to override
> the system default encoding. This is used (to decode) any values that
> are byte-strings. These need to be turned into unicode prior to writing
> out in the specified output encoding.
> 
> If encoding is None (the default) then no encoding or decoding is
> explicitly done. This means the default behaviour for ConfigObj is
> completely unchanged.

Sounds reasonable.

-- 
Nicola Larosa - http://www.tekNico.net/

People just don't want to be pestered - they want shit to happen
without being bugged. If technology is to help, it must be seren-
dipitous rather than strident. -- Nathan Torkington, November 2005

[Configobj-develop] ConfigObj Changes

[Fuzzyman <fuz...@vo...>]

Hello all,

I'm working on some changes to ConfigObj that add new functionality.
These add some of the ``get`` type methods from ConfigParser to
ConfigObj, as well as full unicode support.

Both are *relatively* easy and will become ConfigObj 4.2.0. (I may well
do a beta release first.)

There is a question about the method signature for the get methods. The
ConfigParser methods are :

    get( section, option)

    Get an option value for the named section.

    getint( section, option)

    A convenience method which coerces the option in the specified
section to an integer.

    getfloat( section, option)

    A convenience method which coerces the option in the specified
section to a floating point         number.

    getboolean( section, option)

    A convenience method which coerces the option in the specified
section to a Boolean value.         Note that the accepted values for
the option are "1", "yes", "true", and "on", which cause this        
method to return True, and "0", "no", "false", and "off", which cause it
to return False. These         string values are checked in a
case-insensitive manner. Any other value will cause it to raise        
ValueError.

I *won't* implement get because it clashes with a dictionary method. If
I implement the ``ConfigParser`` compatibility layer then that will
change (but be on a subclass).

My intention is to make these methods *section* methods, so they are
available on all subsections, and to keep the ConfigParser method
signature. For values *not* in a subsection, this leads to the slightly
awkward syntax :

cfg = ConfigObj(cfg_file)
val = cfg.getboolean(None, 'key')

The alternatives are :

* To reverse the order of the arguments, with section defaulting to
None. This leads to confusion in the (theoretical) ConfigParser
compatibility layer, which will of course  be reversed back. It is
*slightly* more friendly for normal use though.

* To make it a method on ConfigObj (*not* sections) and allow section to
be a string *or* a reference to a subsection - in which case the
specified key would be taken from that.

Should I then remove ``istrue`` (added in 4.1.0) which is effectively
the same as ``getboolean`` ? I would deprecate it and add a warning
until 4.3.0.


The unicode changes are reasonably straightforward. I'll re-implement
the system used in ConfigObj 3. (Unless anyone raises any objections or
suggests an alternative).

 You will be able to pass in an (optional)  'encoding' keyword. If
present, this will be used to decode the input. This maps to an
'encoding' attribute which will be used used to encode when writing (and
can obviously be changed).

There will also be a 'default_encoding' keyword/attribute to override
the system default encoding. This is used (to decode) any values that
are byte-strings. These need to be turned into unicode prior to writing
out in the specified output encoding.

If encoding is None (the default) then no encoding or decoding is
explicitly done. This means the default behaviour for ConfigObj is
completely unchanged.

Opinions/responses solicited.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/configobj.html

Re: [Configobj-develop] ConfigObj & a Terrible Idea

[Fuzzyman <fuz...@vo...>]

fuz...@vo... wrote:
> [snip..]
>
> Note
>
> I've had no reply to this suggestion on the *Python-Dev* list. However
> I have had some feedback from the *Bazaar* team on improvements I
> could make in this direction.
>
Note that this will almost certainly include *proper* unicode support,
as well as the other 'get' type methods of ConfigParser.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> Posted by *Fuzzyman* on 2006-01-26 20:53:21.
> Categories: Python
> <http://www.voidspace.org.uk/python/weblog/arch_Python.shtml>,
> Projects <http://www.voidspace.org.uk/python/weblog/arch_Projects.shtml>
> Visit the Voidspace Techie Blog
> <http://www.voidspace.org.uk/python/weblog/index.shtml> to read this
> entry and more.
>

[Configobj-develop] ConfigObj & a Terrible Idea

[<fuz...@vo...>]

{emo;drive} `ConfigObj </python/configobj.html>`_ has also been spotted in the wild in a few interesting situations. Open Source projects that I've seen using it now include :

* `Bazaar <http://bazaar-ng.org>`_.

    Bazaar is *the* Python  distributed {acro;VCS;Version Control System}.

    ConfigObj is used to read ``bazaar.conf`` and ``branches.conf``.

* `Planet Plus <http://planetplus.python-hosting.com/>`_

    A new web application version of `Planet <http://www.planetplanet.org/>`_, 
    the web aggregator.
    
* `NeuroImaging in Python <http://projects.scipy.org/neuroimaging/ni/wiki>`_

    BrainSTAT is a project with the ultimate goal to produce a
    platform-independent python environment for the analysis of brain
    imaging data.
    
* `Gruik <http://www.tracos.org/gruik/wiki>`_

    Gruik is a free software network packet sniffer.

This is cool because **ConfigObj** is my favourite project. Not only was it a lot of work, but I genuinely think it provides a simple interface to a lot of functionality. Some of these projects are also using the more advanced features of ConfigObj - like validation and the `walk <http://www.voidspace.org.uk/python/configobj.html#walking-a-section>`_ method.

It has occurred to me that the ConfigObj {acro;API;Application Programmers Interface} and syntax is *basically* compatible with `ConfigParser <http://docs.python.org/lib/module-ConfigParser.html>`_, which is the Python standard library.

Hence the terrible idea. {sm;:-o}

It would be a lot of work, but completely feasible, to implement a ``ConfigParser`` compatibility layer on top of ConfigObj. This could retain backwards compatibility with ConfigParser, but add all the ConfigObj features like nested-sections and list values. This would address a lot of the issues raised in the `ConfigParser Shootout <http://wiki.python.org/moin/ConfigParserShootout>`_.

So I've donned my flame-proof pants (foolhardy chap that I am) and suggested it on `python-dev <http://mail.python.org/mailman/listinfo/python-dev>`_.

I said something like :

In the past there has been some discussion about a new module to replace
ConfigParser. Most notably at
http://wiki.python.org/moin/ConfigParserShootout

Specific issues that could be addressed include :

* Simpler API
* Nested subsections
* List values
* Storing/converting datatypes
* Config file schema
* Keeps track of order of values

Plus other issues.

I'm the (co-)author of ConfigObj -
http://www.voidspace.org.uk/python/configobj.html

This is a reasonably mature project (now in it's fourth incarnation),
and is being used in projects like Bazaar_
and `Planet Plus`_.

It occurs to me that the ConfigObj API and syntax is *almost* fully
compatible with ConfigParser.

It would be possible to extend to the ConfigObj API to be backwards
compatible with ConfigParser. This would bring the added benefits of
ConfigObj, without needing to add an extra module to the standard library.

Well nearly. ConfigObj supports config file schema with (optional) type
conversion, through a companion module called `validate </python/validate.html>`_. This could be
included or left as an added option.

Anyway. If this stands a *chance* of acceptance, I'll write the {acro;PEP;Python Enhancement Proposal} (and if accepted, do the work - which is not inconsiderable).

Summary of ConfigObj
====================

ConfigObj is a Python 2.2 compatible config file parser. It's major
feature is simplicity of use.

It reads (and writes) INI file like config files and presents the
members using a dictionary interface.

The order of keys/sections is preserved, and it allows nested
subsections to any level :

e.g. ::

    key = value
        [section]
        key = value
          [[sub-section]]
          key = value

It is fully documented with a barrage of doctests.

All comments in the config file are also preserved as attributes of the
object, and will be written back out. This can be useful for including
comments in programatically generated config files.

It is integrated with a powerful validation system.

Difficulties & Differences
==========================

A ConfigObj instance is a sub-class of the dictionary datatpe. This
means that the ``get`` method of ConfigParser clashes.

ConfigObj allows values in the root section (why not ?).

ConfigObj doesn't support line continuations (it does allow multi-line
values through the use of triple quoted strings).

ConfigObj currently only allows '``=``' as a valid divider.

Creating ConfigParser (and related classes) compatibility is a big job.

Solution
========

All of these problems (if deemed necessary) can be resolved. Either
through options, or just by extending the ConfigObj API. I'm happy to
put the work in.

Comments ?

.. note::

    I've had no reply to this suggestion on the **Python-Dev** list. However I have had some feedback from the **Bazaar** team on improvements I could make in this direction.

[Configobj-develop] ConfigObj 4.1.0 and Validate 0.2.1

[<fuz...@vo...>]

{emo;carrot} **ConfigObj 4.1.0** and **Validate 0.2.1** are now  available. {sm;:-)}

ConfigObj and validate have both had moderately big updates.

**What are they ?**

`ConfigObj <http://www.voidspace.org.uk/python/configobj.html>`_ is a simple but powerful config file reader and writer: an *ini file round tripper*. Its main feature is that it is very easy to use, with a straightforward programmer's interface and a simple syntax for config files.

It supports nested sections, preserves the order of keys and sections, list values, multiple line values, etc.

`validate <http://www.voidspace.org.uk/python/validate.html>`_ integrates with ConfigObj (but can also be used standalone) to check that values meet a certain specification. This can be used to validate a config file, *and* convert values to the expected type.

**What Has Changed?**

Configobj:

Added a ``merge`` method. This allows easy merging together of several config files (e.g. merging user values on top of a default one).

A new ``flatten_errors`` function to turn the results dictionary from ``validate`` into a flat list of errors. 

     Added merge, a recursive update.

     Added preserve_errors to validate and the flatten_errors example function.

     Thanks to Matthew Brett for suggestions and helping me iron out bugs.

     Fixed bug where a config file is all comment, the comment will now be initial_comment rather than final_comment.

     Validation no longer done on the 'DEFAULT' section (only in the root level). This allows interpolation in configspecs.

     Also use the new list syntax in validate 0.2.1. (For configspecs).


Validate:

A new list syntax for checks.


    Fixed bug so we can handle keyword argument values with commas.

    We now use a list constructor for passing list values to keyword arguments (including default) : ::

        default=list("val", "val", "val")

Re: [Configobj-develop] interpolation supported in the specification file ?

[Fuzzyman <fuz...@vo...>]

Jim...@no... wrote:

> [snip....]
>
>>The answer is for validation to *not* be done on the 'DEFAULT' 
>>section.
>>    
>>
>
>Is it, therefore, correct to conclude that interpolations are best done
>in a config file because, if present in a configspec file, they can not
>be validated, but that this will change with the 4.1.0 release?
>
>  
>
That is currently the case. I'm sending you by private email an interim
version which has the fix made.

I'm targeting a 4.1.0 release within a week. The changes now to be made
are in the validate module - for it to support the ``default=list(..)``
constructor.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

>>I'm targeting all the discussed recent changes/bugfixes for a 4.1.0 
>>release.
>>All the best,
>>
>>Fuzzyman
>>
>>http://www.voidspace.org.uk/python/index.shtml
>>
>>    
>>
>>><output>
>>>sys.platform:            win32
>>>sys.version:             2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC
>>>v.1310 32 bit (Intel)]
>>>sys.getwindowsversion(): (5, 1, 2600, 2, 'Service Pack 2')
>>>sys.winver:              2.4
>>>
>>>configobj.__version__: 4.0.2
>>>validate.__version__:  0.2.0
>>>
>>>*** validation succeeded ***
>>>Traceback (most recent call last):
>>> File "C:\Documents and Settings\jim.vickroy\My
>>>Documents\Projects\_experimental_\pythonutils\configobj\test-
>>>      
>>>
>>reader.py",> line 55, in ?
>>    
>>
>>>   assert facts['interpolated string'] == 'fuzzy-man', 
>>>facts['interpolated string']
>>>AssertionError: fuzzy-string(default="man")
>>></output>
>>>
>>>
>>>Thanks,
>>>-- jv
>>>      
>>>
>>
>>
>>-------------------------------------------------------
>>This SF.net email is sponsored by: Splunk Inc. Do you grep through 
>>log files
>>for problems?  Stop!  Download the new AJAX search engine that makes
>>searching your log files as easy as surfing the  web.  DOWNLOAD 
>>SPLUNK!http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>>_______________________________________________
>>Configobj-develop mailing list
>>Con...@li...
>>https://lists.sourceforge.net/lists/listinfo/configobj-develop
>>
>>    
>>
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
>for problems?  Stop!  Download the new AJAX search engine that makes
>searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
>http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
>_______________________________________________
>Configobj-develop mailing list
>Con...@li...
>https://lists.sourceforge.net/lists/listinfo/configobj-develop
>
>  
>

Re: [Configobj-develop] interpolation supported in the specification file ?

[<Jim...@no...>]

----- Original Message -----
From: Fuzzyman <fuz...@vo...>
Date: Wednesday, December 14, 2005 2:29 am
Subject: Re: [Configobj-develop] interpolation supported in the
specification file ?

> Jim Vickroy wrote:
> 
> > Is interpolation supported in default specifications or is this a
> > total abuse of the package?  If interpolations are supported, 
> then I
> > do not understand how to make them work in the specification 
> file. 
> > Here is the output from the attached script:
> >
> Interpolation is working - there is a bug in your logic :-D
> 
> Your string containing the interpolation value is :
> 
>    string(default="fuzzy-%(man)s")
> 
> You have defined 'man' in your DEFAULT section to be :
> 
>    string(default="man")
> 
> So ConfigObj replaces ``%(man)s`` with ``string(default="man")``. The
> result is :
> 
>       fuzzy-string(default="man")
> 
> Try replacing the line in your DEFAULT section with :
> 
>    man = wuzzy
> 
> That should do something a bit more expected. :-)

OK, thanks for the correction.

> 
> *However* - that will then raise an error because the default section
> doesn't contain a valid check.
> 
> Your code will still print "validation succeeded" - because check 
> is now
> a dictionary, which doesn't evaluate to False.
> 
>    if check:
>       print '*** validation succeeded ***'
> 
> Should be changed to :
> 
>    if check == True::
>       print '*** validation succeeded ***'

Thanks again.

> 
> The answer is for validation to *not* be done on the 'DEFAULT' 
> section.

Is it, therefore, correct to conclude that interpolations are best done
in a config file because, if present in a configspec file, they can not
be validated, but that this will change with the 4.1.0 release?

> I'm targeting all the discussed recent changes/bugfixes for a 4.1.0 
> release.
> All the best,
> 
> Fuzzyman
> 
> http://www.voidspace.org.uk/python/index.shtml
> 
> > <output>
> > sys.platform:            win32
> > sys.version:             2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC
> > v.1310 32 bit (Intel)]
> > sys.getwindowsversion(): (5, 1, 2600, 2, 'Service Pack 2')
> > sys.winver:              2.4
> >
> > configobj.__version__: 4.0.2
> > validate.__version__:  0.2.0
> >
> > *** validation succeeded ***
> > Traceback (most recent call last):
> >  File "C:\Documents and Settings\jim.vickroy\My
> > Documents\Projects\_experimental_\pythonutils\configobj\test-
> reader.py",> line 55, in ?
> >    assert facts['interpolated string'] == 'fuzzy-man', 
> > facts['interpolated string']
> > AssertionError: fuzzy-string(default="man")
> > </output>
> >
> >
> > Thanks,
> > -- jv
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through 
> log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD 
> SPLUNK!http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop
> 

Re: [Configobj-develop] interpolation supported in the specification file ?

[Fuzzyman <fuz...@vo...>]

Jim Vickroy wrote:

> Is interpolation supported in default specifications or is this a
> total abuse of the package?  If interpolations are supported, then I
> do not understand how to make them work in the specification file. 
> Here is the output from the attached script:
>
Interpolation is working - there is a bug in your logic :-D

Your string containing the interpolation value is :

    string(default="fuzzy-%(man)s")

You have defined 'man' in your DEFAULT section to be :

    string(default="man")

So ConfigObj replaces ``%(man)s`` with ``string(default="man")``. The
result is :

       fuzzy-string(default="man")

Try replacing the line in your DEFAULT section with :

    man = wuzzy

That should do something a bit more expected. :-)

*However* - that will then raise an error because the default section
doesn't contain a valid check.

Your code will still print "validation succeeded" - because check is now
a dictionary, which doesn't evaluate to False.

    if check:
       print '*** validation succeeded ***'

Should be changed to :

    if check == True::
       print '*** validation succeeded ***'

The answer is for validation to *not* be done on the 'DEFAULT' section.

I'm targeting all the discussed recent changes/bugfixes for a 4.1.0 release.

All the best,

Fuzzyman

http://www.voidspace.org.uk/python/index.shtml

> <output>
> sys.platform:            win32
> sys.version:             2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC
> v.1310 32 bit (Intel)]
> sys.getwindowsversion(): (5, 1, 2600, 2, 'Service Pack 2')
> sys.winver:              2.4
>
> configobj.__version__: 4.0.2
> validate.__version__:  0.2.0
>
> *** validation succeeded ***
> Traceback (most recent call last):
>  File "C:\Documents and Settings\jim.vickroy\My
> Documents\Projects\_experimental_\pythonutils\configobj\test-reader.py",
> line 55, in ?
>    assert facts['interpolated string'] == 'fuzzy-man', 
> facts['interpolated string']
> AssertionError: fuzzy-string(default="man")
> </output>
>
>
> Thanks,
> -- jv

[Configobj-develop] interpolation supported in the specification file ?

[Jim V. <Jim...@no...>]

Is interpolation supported in default specifications or is this a total 
abuse of the package?  If interpolations are supported, then I do not 
understand how to make them work in the specification file.  Here is the 
output from the attached script:

<output>
sys.platform:            win32
sys.version:             2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 
32 bit (Intel)]
sys.getwindowsversion(): (5, 1, 2600, 2, 'Service Pack 2')
sys.winver:              2.4

configobj.__version__: 4.0.2
validate.__version__:  0.2.0

*** validation succeeded ***
Traceback (most recent call last):
  File "C:\Documents and Settings\jim.vickroy\My 
Documents\Projects\_experimental_\pythonutils\configobj\test-reader.py", 
line 55, in ?
    assert facts['interpolated string'] == 'fuzzy-man',  
facts['interpolated string']
AssertionError: fuzzy-string(default="man")
</output>


Thanks,
-- jv

Re: [Configobj-develop] Validate - List Default Values

[Fuzzyman <fuz...@vo...>]

Nicola Larosa wrote:

>>The alternative would be to indicate list values by surrounding with
>>square brackets :
>>
>>    checkname(default="[ 'val 1', 'val 2', 'val 3']")
>>    
>>
>
>What about using a pseudo-constructor?
>
>     checkname(default=list('val 1', 'val 2', 'val 3'))
>  
>

That's a good idea. Saves syntax confusion.

Makes the regular expression harder of course. :-)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml