configobj-develop

[Configobj-develop] List handling in Validator

[Stefan P. <pa...@ik...>]

I've noticed some issues with how list types are handled in Validate.
There are several different list types, including, int_list  for
integers, string_list for strings, etc. IMHO the list handling could
be generalized to be much more useful.

Right now there is no easy way to declare a list of e.g. integers with
min and max values. You could declare a completely new type,
"list_of_ints_min_0_max_10" but in practice it's not really feasible.
It's also not easily possible to declare a list containing some custom
type.

I think an optional argument, type, should be added to the list type,
so something like this would be possible:

list(type=int(min=0,max=10), min=1, max=2)

This would also solve problems with inconsistent naming (integer vs.
int_list types). The old types could be kept for backwards
compatibility.


I've reported one issue related to lists, and three other issues (some
with patches) to the Google Code issue tracker at
http://code.google.com/p/configobj/issues/ but have not gotten any
reaction to them. Is the Google Code tracker the correct place to
report bugs? If not, could this be clarified somewhere?

-- 
Stefan Parviainen

Re: [Configobj-develop] List handling in Validator

[Michael F. <fuz...@vo...>]

On 26/12/2010 11:49, Stefan Parviainen wrote:
> I've noticed some issues with how list types are handled in Validate.
> There are several different list types, including, int_list  for
> integers, string_list for strings, etc. IMHO the list handling could
> be generalized to be much more useful.
>
> Right now there is no easy way to declare a list of e.g. integers with
> min and max values. You could declare a completely new type,
> "list_of_ints_min_0_max_10" but in practice it's not really feasible.
> It's also not easily possible to declare a list containing some custom
> type.
>
> I think an optional argument, type, should be added to the list type,
> so something like this would be possible:
>
> list(type=int(min=0,max=10), min=1, max=2)
>
> This would also solve problems with inconsistent naming (integer vs.
> int_list types). The old types could be kept for backwards
> compatibility.

This sounds like a good change.

> I've reported one issue related to lists, and three other issues (some
> with patches) to the Google Code issue tracker at
> http://code.google.com/p/configobj/issues/ but have not gotten any
> reaction to them. Is the Google Code tracker the correct place to
> report bugs? If not, could this be clarified somewhere?
>
Google code tracker is the correct place. I'm not using ConfigObj a 
great deal myself at the moment so don't have much spare time to work on 
it. I will eventually get to it though.

I see you submitted a patch along with issue 26, but no tests. Patches 
with tests and documentation are likely to get applied *much* sooner. :-)

Note that in issue 26 values will *already* have been converted to lists 
if they are in list format - so doing an *additional* split on ',' is 
incorrect. For example this single string does *not* represent a list:

     value = "something, with, commas"

I'm really not sure about issue 19 - only supporting # for comments was 
a deliberate design decision and in general I'm against making the 
configobj API more complex. Issue 3 has been merged into the Python 3 
port. Other than these it looks like most issues could be resolved in a 
new release.

All the best,

Michael Foord

-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

Re: [Configobj-develop] List handling in Validator

[Stefan P. <pa...@ik...>]

On Mon, Jan 3, 2011 at 3:37 PM, Michael Foord <fuz...@vo...> wrote:
>> This would also solve problems with inconsistent naming (integer vs.
>> int_list types). The old types could be kept for backwards compatibility.
> This sounds like a good change.
I can take a stab at it myself if you are busy/not interested in doing
it yourself.

> Note that in issue 26 values will *already* have been converted to lists if
> they are in list format - so doing an *additional* split on ',' is
> incorrect. For example this single string does *not* represent a list:
>
>    value = "something, with, commas"

Well, I think that string does represent a list :-) And so does
ConfigObj in some cases. Consider the following:

import configobj, validate
spec_str = """
a = list
b = integer
"""
conf_str = """
a = 1, 2
b = 0
"""
spec = configobj.ConfigObj(spec_str.split('\n'))
conf = configobj.ConfigObj(conf_str.split('\n'), configspec=spec)

conf.validate(validate.Validator())

print type(conf['a']), conf['a']
print type(conf['b']), conf['b']

conf['a'] = '1, 2'
conf['b'] = '0'

res = conf.validate(validate.Validator())

print type(conf['a']), conf['a']
print type(conf['b']), conf['b']

---
The above outputs:
<type 'list'> ['1', '2']
<type 'int'> 0
<type 'str'> 1, 2
<type 'int'> 0

So when parsing the config file "1,2" is indeed interpreted as a list,
but when "1,2" is assigned as a string it fails. Note that with the
integer type, it succeeds in both cases. I expect that assigning a
string with the same content as I would write in the config file would
result in the same values after validation. It does not, which is
confusing at least to me. I don't see a problem with splitting a
string when it is assigned to something expecting a list. This is
probably what the user wants anyway. Currently nothing sensible at all
happens (you end up with a string where you are expecting a list!).

There is the force_list type, which kind of solves the problem (in a
bad way IMHO), by converting the string to a list with a single item.
But why is a string properly converted to an int, while a string is
only converted to a list if force_list is used? Why is there no
force_int?

In summary: It is terribly confusing when I can't use the same syntax
when assigning as I can when writing config files.

-- 
Stefan Parviainen

Re: [Configobj-develop] List handling in Validator

[Michael F. <fuz...@vo...>]

On 03/01/2011 14:41, Stefan Parviainen wrote:
> On Mon, Jan 3, 2011 at 3:37 PM, Michael Foord<fuz...@vo...>  wrote:
>>> This would also solve problems with inconsistent naming (integer vs.
>>> int_list types). The old types could be kept for backwards compatibility.
>> This sounds like a good change.
> I can take a stab at it myself if you are busy/not interested in doing
> it yourself.
>

Tested patch with documentation stands a much better chance of making it 
into a release than leaving it to me... :-)

>> Note that in issue 26 values will *already* have been converted to lists if
>> they are in list format - so doing an *additional* split on ',' is
>> incorrect. For example this single string does *not* represent a list:
>>
>>     value = "something, with, commas"
> Well, I think that string does represent a list :-) And so does
> ConfigObj in some cases. Consider the following:
>
> import configobj, validate
> spec_str = """
> a = list
> b = integer
> """
> conf_str = """
> a = 1, 2
> b = 0
> """
> spec = configobj.ConfigObj(spec_str.split('\n'))
> conf = configobj.ConfigObj(conf_str.split('\n'), configspec=spec)
>
> conf.validate(validate.Validator())
>
> print type(conf['a']), conf['a']
> print type(conf['b']), conf['b']
>
> conf['a'] = '1, 2'
> conf['b'] = '0'
>
> res = conf.validate(validate.Validator())
>
> print type(conf['a']), conf['a']
> print type(conf['b']), conf['b']
>
> ---
> The above outputs:
> <type 'list'>  ['1', '2']
> <type 'int'>  0
> <type 'str'>  1, 2
> <type 'int'>  0
>
> So when parsing the config file "1,2" is indeed interpreted as a list,
No. 1,2 without quotes is a list. "1,2" with quotes is a string that 
happens to contain a comma.

The specified behaviour of ConfigObj and validate interaction is that 
list values should have already been parsed into a list *before* being 
validated - so the validation code must honour this. Otherwise, as I 
indicated, string values that contain commas can be incorrectly split 
into lists.


> but when "1,2" is assigned as a string it fails. Note that with the
> integer type, it succeeds in both cases. I expect that assigning a
> string with the same content as I would write in the config file would
> result in the same values after validation. It does not, which is
> confusing at least to me. I don't see a problem with splitting a
> string when it is assigned to something expecting a list. This is
> probably what the user wants anyway. Currently nothing sensible at all
> happens (you end up with a string where you are expecting a list!).
>
The list validation when a list is not supplied is not ideal - that is 
why I said your API suggestion was a good one. However quoted strings 
can contain commas that don't represent lists which is why your 
suggestion of always splitting on commas is bad.

> There is the force_list type, which kind of solves the problem (in a
> bad way IMHO), by converting the string to a list with a single item.
> But why is a string properly converted to an int, while a string is
> only converted to a list if force_list is used? Why is there no
> force_int?
What would be the use case of force_int? I don't really understand your 
point here.

> In summary: It is terribly confusing when I can't use the same syntax
> when assigning as I can when writing config files.
>
This is a different issue though. A ConfigObj instance will have 
*already* parsed list values. If you want a list value then assign a 
list. (Note that if you assign a value to the string "1,2" (without the 
quotes in the string itself) then ConfigObj will put quotes round it 
when it writes so that it is *not* read back in as a list.

Alternatively, it sounds like a reasonable feature request to ask for a 
method to parse a string using the ConfigObj syntax rules - so that you 
can use ConfigObj syntax for assigning strings.

All the best,

Michael Foord

-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

Re: [Configobj-develop] List handling in Validator

[David H. <neg...@gm...>]

I had posted some thoughts awhile back regarding generalization of the list
API.  It doesn't necessarily directly address all of the things that Stefan
is saying, but I think it factors into the discussion:

On Fri, Jul 17, 2009 at 12:41, David Hostetler <neg...@gm...>
 wrote:

> [this discussion arises out of the recent post inquiring how to handle
> empty values in a list]
>
> I think in general the list validation features might benefit from being
> either parameterized or more comprehensive.
>
> By that I mean, for example, that 'force_list' isn't so much a different
> validation function as it is a behavior that the author may or may not wish
> to apply to any particular list.
>
> In fact, in hindsight I wonder whether instead of having the set of list
> functions (list, int_list, float_list, bool_list, etc..) there is simply and
> only 'list'.   In addition to the 'min' and 'max' parameters that the list
> function accepts, you could add to that things like:
>
> - 'valtype'   (to allow 'valtype=int', 'valtype=float', etc..)
>
> - 'force'  (replacing the 'force_list' function and allowing any kind of
> list to adopt the implicit-trailing-comma behavior)
>
> - 'allowgaps'  (to accommodate the missing value behavior Cameron inquired
> about)
>
>
> Thus list() accrues parameters as needed to customize how it validates the
> values in a particular instance.
>
> The nice thing here is that the backwards compatibility story is clean:
> usage of, say, int_list() is functionally equivalent to list(type=int).
> force_list() becomes list(force=True).
>
> mixed_list is still a bit of a black sheep.  But that may be fine anyway.
> Technically, all of the other list variants are implicitly for lists of
> arbitrary length.  mixed_list is only valid for lists of known length
> (otherwise how can you specify the type for each item in the mixed list?).
> mixed_list, by its very nature, cannot do implicit type conversions.
>
> In other words -- the set of available validation functions essentially
> remains defined as "what kind of thing you get back from the validation".
>
> 'integer' gives you an integer
> 'tuple' gives you a tuple
> 'list' gives you a homogeneous list of 0 or more items
> 'option' gives you one string from among a known list of strings
> 'mixed_list' gives you a (potentially) heterogeneous list of 0 or more
> items
> etc..
>
>
> There might still be opportunity to refine the relationship between list
> and mixed_list, but the trade-off is the complexity of the list() API.
>
> In fact this whole discussion is about the complexity vs. capability of
> list validation.  Having force_list() be a top-level validation function
> grows the set of functions by one, but keeps the average function complexity
> low.  But it reduces the capability of the functions because the 'force'
> behavior cannot be applied to lists obtained via the other functions -- it's
> an either/or choice for the programmer: you can use int_list to get a list
> of ints, or you can have force_list to get a forced list of strings, but you
> can't have a forced list of ints.
>
> Pushing some of the choices back into parameters increases both the
> complexity and the capability of the validation functions, while
> simultaneously reducing the number of validation functions (assuming the
> redundant functions were then removed or deprecated).
>
>
> Incidentally -- I would argue that the tuple() validation function has
> precisely the same needs for parameterization as does list().  The
> parameterization needs arise from the existence of more than one value per
> keyword, and that's the case for both lists and tuples.  The only difference
> between the two returned types is that one is mutable the other is not --
> they're both sequences.  For that matter, why can I get a mixed list but not
> a mixed tuple?
>
>
> Just some food for thought.



I do still have that on my queue of stuff to do.  Eventually.  :)

regards,

-hoss


David Hostetler
neg...@gm...

Re: [Configobj-develop] List handling in Validator

[Stefan P. <pa...@ik...>]

I see what the source of confusion might be. I am talking about values
after validation has been performed. So when I say that I assign
something and then the value is of type something I mean that it is
after validation. Maybe this clears up some issues?

On Mon, Jan 3, 2011 at 4:49 PM, Michael Foord <fuz...@vo...> wrote:
>> So when parsing the config file "1,2" is indeed interpreted as a list,
> No. 1,2 without quotes is a list. '1,2" with quotes is a string that happens
> to contain a comma.
My point is that I can do foo = 1,2  in a config file and it works
correctly, but if I do config["foo"] = '1,2' it does not work. I would
fully understand that "foo = '1,2'" and config["foo"] = "'1,2'" would
not work (Notice the quotes inside the quotes). I think it's confusing
that I can write one thing in a file, but not the exact same thing in
the source. In this case I assign 1,2 (n.b. without any quotes) in
both cases.

> Otherwise, as I indicated, string values that contain commas can be incorrectly split into lists.
Can you name a situation when splitting a string when something
expects a list would lead to unwanted results? Not splitting the
string, and instead assigning it directly _always_ leads to incorrect
results when a list is expected.

> However quoted strings can contain commas that don't represent lists
> which is why your suggestion of always splitting on commas is bad.
I'm certainly not suggesting that a string containing commas is
_always_ interpreted as a list. Only when a list is expected!
Maybe there is an error in my code, and it is always split, but that
is certainly not the intention.

> What would be the use case of force_int? I don't really understand your
> point here.
Assigning a string to something expecting an integer "just works".
Assigning a string to something expecting a list does not, instead
force_list is needed. Don't you see the inconsistent behavior? I don't
see why integers can be converted automatically (when needed) but
lists require a separate type for this to work.

>  If you want a list value then assign a list.
I could similarly say that if you want an int value then assign an
int. Yet integer conversion from a string happens automatically. Why
does not list conversion happen?

> Alternatively, it sounds like a reasonable feature request to ask for a
> method to parse a string using the ConfigObj syntax rules - so that you can
> use ConfigObj syntax for assigning strings.
I think that an assignment should work the same, regardless if it's in
a config file or in the source code.

Once again:
Assume the spec:
p = list()

Case 1, Conf file:
p = 1,2

Case 2, in source:
config['p'] = '1,2'

In both cases the right-hand side is: 1,2. The quotation marks in case
2 is there because of Python syntax. I am still assigning the value
1,2 without any quotes, just like in case 1.

Given the above spec, in both cases, after validation, I expect p to
be the list ['1','2']. Currently case 1 gives ['1','2'] while case 2
gives a validation error. I don't think it makes sense to get
different results given that the right-hand side is the same in both
cases.

-- 
Stefan Parviainen

Re: [Configobj-develop] List handling in Validator

[Michael F. <fuz...@vo...>]

On 04/01/2011 14:18, Stefan Parviainen wrote:
> I see what the source of confusion might be. I am talking about values
> after validation has been performed. So when I say that I assign
> something and then the value is of type something I mean that it is
> after validation. Maybe this clears up some issues?
>
> On Mon, Jan 3, 2011 at 4:49 PM, Michael Foord<fuz...@vo...>  wrote:
>>> So when parsing the config file "1,2" is indeed interpreted as a list,
>> No. 1,2 without quotes is a list. '1,2" with quotes is a string that happens
>> to contain a comma.
> My point is that I can do foo = 1,2  in a config file and it works
> correctly, but if I do config["foo"] = '1,2' it does not work. I would
> fully understand that "foo = '1,2'" and config["foo"] = "'1,2'" would
> not work (Notice the quotes inside the quotes). I think it's confusing
> that I can write one thing in a file, but not the exact same thing in
> the source. In this case I assign 1,2 (n.b. without any quotes) in
> both cases.

ConfigObj instance contain *parsed values*. If you want lists you must 
assign lists. Otherwise you would have to do your own quoting of any 
strings you assigned to a configobj instance yourself (including 
honouring the comment rules) just to allow you to assign lists as strings.

I suggested a "ConfigObj.parse(value)" API to allow you to do that though.
>> Otherwise, as I indicated, string values that contain commas can be incorrectly split into lists.
> Can you name a situation when splitting a string when something
> expects a list would lead to unwanted results? Not splitting the
> string, and instead assigning it directly _always_ leads to incorrect
> results when a list is expected.
>

I gave examples:

     value = "some value, that contains, commas"

>> However quoted strings can contain commas that don't represent lists
>> which is why your suggestion of always splitting on commas is bad.
> I'm certainly not suggesting that a string containing commas is
> _always_ interpreted as a list. Only when a list is expected!
> Maybe there is an error in my code, and it is always split, but that
> is certainly not the intention.
If a single value is passed and it contains commas your code would 
always split it which is incorrect. See example above.

>> What would be the use case of force_int? I don't really understand your
>> point here.
> Assigning a string to something expecting an integer "just works".
> Assigning a string to something expecting a list does not, instead
> force_list is needed. Don't you see the inconsistent behavior? I don't
> see why integers can be converted automatically (when needed) but
> lists require a separate type for this to work.

I did say that I *agreed* that the list handling was not ideal and even 
tentatively approved your api suggestions - minus the bug with respect 
to splitting strings into lists.

>>   If you want a list value then assign a list.
> I could similarly say that if you want an int value then assign an
> int.

Yep, that works. ;-)

>   Yet integer conversion from a string happens automatically. Why
> does not list conversion happen?
>

You seem to want to assign unparsed values *after* validation. What is 
your use case for this? (Why *not* just assign the correct values, why 
rely on ConfigObj to parse again?)

This is a slightly irrelevant question as I have agreed that list 
handling can be improved and suggested an API for parsing strings to 
lists where you want to assign them. I'm still *interested* in why you 
want to do this though.

>> Alternatively, it sounds like a reasonable feature request to ask for a
>> method to parse a string using the ConfigObj syntax rules - so that you can
>> use ConfigObj syntax for assigning strings.
> I think that an assignment should work the same, regardless if it's in
> a config file or in the source code.
>

It *isn't* the same though. A ConfigObj instance holds parsed (and 
possibly validated) values so you can't assign unparsed values and 
expect them to behave the same. How should ConfigObj know whether 
"config['foo'] = '1,2'" is a parsed or unparsed value being assigned? As 
I said earlier (we're just going over the same point again and again) if 
configobj behaved like this you would have to apply all the quoting 
rules manually instead of having configobj do it for you. Worse though: 
when parsing should configobj unquote values for you or not? If it does 
then "config['foo'] = config['foo']" could change the value when writing 
out! (A value quoted in the source would be unquoted when parsed - and 
if unquoted when assigned it would be written out unquoted.)

Either configobj would need to track all this state (which values have 
been parsed and which not) - with the "bug" above that assigning a value 
to itself could change it - or it would have to do no quoting and no 
unquoting. Nasty.

I don't think you really understand the consequences of what you're 
asking for. :-) Anyway, not gonna happen as you request. I would be open 
to patches implementing a "parse" method that allows you to do what you 
want though.

All the best,

Michael

> Once again:
> Assume the spec:
> p = list()
>
> Case 1, Conf file:
> p = 1,2
>
> Case 2, in source:
> config['p'] = '1,2'
>
> In both cases the right-hand side is: 1,2. The quotation marks in case
> 2 is there because of Python syntax. I am still assigning the value
> 1,2 without any quotes, just like in case 1.
>
> Given the above spec, in both cases, after validation, I expect p to
> be the list ['1','2']. Currently case 1 gives ['1','2'] while case 2
> gives a validation error. I don't think it makes sense to get
> different results given that the right-hand side is the same in both
> cases.
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

Re: [Configobj-develop] List handling in Validator

[David H. <neg...@gm...>]

Stefan, you're conflating the concepts of data format and programming
language syntax (i.e. code).

The entire purpose of ConfigObj, and it's config file format, rules, and
API, is to allow data to successfully and robustly make the transition
_between_ a text-based file/storage format and program code.

The purpose is NOT to muddy the distinction between data saved in a storage
format and the specification for literals in the Python data model.

My point is that I can do foo = 1,2  in a config file and it works

correctly, but if I do config["foo"] = '1,2' it does not work.


config["foo"] = '1,2'  is Python code!!   It works, and does precisely what
it should.  The specification for Python's lexical data model is
long-standing, proven, consistent, understood, and powerful.   And you're
proposing that ConfigObj should intentionally subvert it for convenience!?

A programmer should _never_ have to ask themselves, when typing out an
assignment in code, "oops... am I writing Python, or am I writing
ConfigObj?"   But that's exactly the kind of ambiguity your proposal would
be injecting.

One of the most useful concepts a programmer can embrace is that of
establishing and respecting a very firm and distinct boundary between stored
data and programmatic data.  This is one of the 'tricks' of successful
i18n/L10n programming, but it's equally effective as a general rule.

With respect to ConfigObj use, applying that concept means respecting the
boundary between the config storage syntax and the ConfigObj API in Python
space.  Once a blob of stored config text has been digested and converted
into programmatic/Python variables, you're squarely in Python land.  Write
Python.  You should neither want nor expect any of the various and sundry
details of the storage format itself to intrude upon your ability to
exercise and rely upon the Python data model.

In other words, once data has been processed and is in a ConfigObj instance,
a list is only ever list because it's a list.  type(x) == types.ListType.
To violate that is to invite madness and anarchy.  And note that the true
Python connoisseur wouldn't be doing explicit type checking anyway, they'd
be duck-hunting.  :)


cheers,

-hoss

David Hostetler
neg...@gm...



On Tue, Jan 4, 2011 at 09:18, Stefan Parviainen <pa...@ik...> wrote:

> I see what the source of confusion might be. I am talking about values
> after validation has been performed. So when I say that I assign
> something and then the value is of type something I mean that it is
> after validation. Maybe this clears up some issues?
>
> On Mon, Jan 3, 2011 at 4:49 PM, Michael Foord <fuz...@vo...>
> wrote:
> >> So when parsing the config file "1,2" is indeed interpreted as a list,
> > No. 1,2 without quotes is a list. '1,2" with quotes is a string that
> happens
> > to contain a comma.
> My point is that I can do foo = 1,2  in a config file and it works
> correctly, but if I do config["foo"] = '1,2' it does not work. I would
> fully understand that "foo = '1,2'" and config["foo"] = "'1,2'" would
> not work (Notice the quotes inside the quotes). I think it's confusing
> that I can write one thing in a file, but not the exact same thing in
> the source. In this case I assign 1,2 (n.b. without any quotes) in
> both cases.
>
> > Otherwise, as I indicated, string values that contain commas can be
> incorrectly split into lists.
> Can you name a situation when splitting a string when something
> expects a list would lead to unwanted results? Not splitting the
> string, and instead assigning it directly _always_ leads to incorrect
> results when a list is expected.
>
> > However quoted strings can contain commas that don't represent lists
> > which is why your suggestion of always splitting on commas is bad.
> I'm certainly not suggesting that a string containing commas is
> _always_ interpreted as a list. Only when a list is expected!
> Maybe there is an error in my code, and it is always split, but that
> is certainly not the intention.
>
> > What would be the use case of force_int? I don't really understand your
> > point here.
> Assigning a string to something expecting an integer "just works".
> Assigning a string to something expecting a list does not, instead
> force_list is needed. Don't you see the inconsistent behavior? I don't
> see why integers can be converted automatically (when needed) but
> lists require a separate type for this to work.
>
> >  If you want a list value then assign a list.
> I could similarly say that if you want an int value then assign an
> int. Yet integer conversion from a string happens automatically. Why
> does not list conversion happen?
>
> > Alternatively, it sounds like a reasonable feature request to ask for a
> > method to parse a string using the ConfigObj syntax rules - so that you
> can
> > use ConfigObj syntax for assigning strings.
> I think that an assignment should work the same, regardless if it's in
> a config file or in the source code.
>
> Once again:
> Assume the spec:
> p = list()
>
> Case 1, Conf file:
> p = 1,2
>
> Case 2, in source:
> config['p'] = '1,2'
>
> In both cases the right-hand side is: 1,2. The quotation marks in case
> 2 is there because of Python syntax. I am still assigning the value
> 1,2 without any quotes, just like in case 1.
>
> Given the above spec, in both cases, after validation, I expect p to
> be the list ['1','2']. Currently case 1 gives ['1','2'] while case 2
> gives a validation error. I don't think it makes sense to get
> different results given that the right-hand side is the same in both
> cases.
>
> --
> Stefan Parviainen
>
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment,
> and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop
>

Re: [Configobj-develop] List handling in Validator

[Michael F. <fuz...@vo...>]

On 04/01/2011 15:03, David Hostetler wrote:
> Stefan, you're conflating the concepts of data format and programming 
> language syntax (i.e. code).
>

What you've said is true David.

Reading Stefan's mind (because he hasn't explained his use case) I 
imagine that he has data input from the user (or some other source) that 
he wants to treat in the same way as if it had been written in a config 
file. He is annoyed that just setting it on a ConfigObj instance doesn't 
do that.

For all the reasons both you and I have explained it isn't a good idea 
for it to work that way anyway...

However, given that he *may* have a reasonable use case for taking 
string data and then treating it as unparsed config data, I'd be happy 
with an API that *allows* him to do that without making it the default 
behaviour.

All the best,

Michael Foord

> The entire purpose of ConfigObj, and it's config file format, rules, 
> and API, is to allow data to successfully and robustly make the 
> transition _between_ a text-based file/storage format and program code.
>
> The purpose is NOT to muddy the distinction between data saved in a 
> storage format and the specification for literals in the Python data 
> model.
>
>     My point is that I can do foo = 1,2  in a config file and it works 
>
>     correctly, but if I do config["foo"] = '1,2' it does not work.
>
>
> config["foo"] = '1,2'  is Python code!!   It works, and does precisely 
> what it should.  The specification for Python's lexical data model is 
> long-standing, proven, consistent, understood, and powerful.   And 
> you're proposing that ConfigObj should intentionally subvert it for 
> convenience!?
>
> A programmer should _never_ have to ask themselves, when typing out an 
> assignment in code, "oops... am I writing Python, or am I writing 
> ConfigObj?"   But that's exactly the kind of ambiguity your proposal 
> would be injecting.
>
> One of the most useful concepts a programmer can embrace is that of 
> establishing and respecting a very firm and distinct boundary between 
> stored data and programmatic data.  This is one of the 'tricks' of 
> successful i18n/L10n programming, but it's equally effective as a 
> general rule.
>
> With respect to ConfigObj use, applying that concept means respecting 
> the boundary between the config storage syntax and the ConfigObj API 
> in Python space.  Once a blob of stored config text has been digested 
> and converted into programmatic/Python variables, you're squarely in 
> Python land.  Write Python.  You should neither want nor expect any of 
> the various and sundry details of the storage format itself to intrude 
> upon your ability to exercise and rely upon the Python data model.
>
> In other words, once data has been processed and is in a ConfigObj 
> instance, a list is only ever list because it's a list.  type(x) == 
> types.ListType.   To violate that is to invite madness and anarchy. 
>  And note that the true Python connoisseur wouldn't be doing explicit 
> type checking anyway, they'd be duck-hunting.  :)
>
>
> cheers,
>
> -hoss
>
> David Hostetler
> neg...@gm... <mailto:neg...@gm...>
>
>
>
> On Tue, Jan 4, 2011 at 09:18, Stefan Parviainen <pa...@ik... 
> <mailto:pa...@ik...>> wrote:
>
>     I see what the source of confusion might be. I am talking about values
>     after validation has been performed. So when I say that I assign
>     something and then the value is of type something I mean that it is
>     after validation. Maybe this clears up some issues?
>
>     On Mon, Jan 3, 2011 at 4:49 PM, Michael Foord
>     <fuz...@vo... <mailto:fuz...@vo...>> wrote:
>     >> So when parsing the config file "1,2" is indeed interpreted as
>     a list,
>     > No. 1,2 without quotes is a list. '1,2" with quotes is a string
>     that happens
>     > to contain a comma.
>     My point is that I can do foo = 1,2  in a config file and it works
>     correctly, but if I do config["foo"] = '1,2' it does not work. I would
>     fully understand that "foo = '1,2'" and config["foo"] = "'1,2'" would
>     not work (Notice the quotes inside the quotes). I think it's confusing
>     that I can write one thing in a file, but not the exact same thing in
>     the source. In this case I assign 1,2 (n.b. without any quotes) in
>     both cases.
>
>     > Otherwise, as I indicated, string values that contain commas can
>     be incorrectly split into lists.
>     Can you name a situation when splitting a string when something
>     expects a list would lead to unwanted results? Not splitting the
>     string, and instead assigning it directly _always_ leads to incorrect
>     results when a list is expected.
>
>     > However quoted strings can contain commas that don't represent lists
>     > which is why your suggestion of always splitting on commas is bad.
>     I'm certainly not suggesting that a string containing commas is
>     _always_ interpreted as a list. Only when a list is expected!
>     Maybe there is an error in my code, and it is always split, but that
>     is certainly not the intention.
>
>     > What would be the use case of force_int? I don't really
>     understand your
>     > point here.
>     Assigning a string to something expecting an integer "just works".
>     Assigning a string to something expecting a list does not, instead
>     force_list is needed. Don't you see the inconsistent behavior? I don't
>     see why integers can be converted automatically (when needed) but
>     lists require a separate type for this to work.
>
>     >  If you want a list value then assign a list.
>     I could similarly say that if you want an int value then assign an
>     int. Yet integer conversion from a string happens automatically. Why
>     does not list conversion happen?
>
>     > Alternatively, it sounds like a reasonable feature request to
>     ask for a
>     > method to parse a string using the ConfigObj syntax rules - so
>     that you can
>     > use ConfigObj syntax for assigning strings.
>     I think that an assignment should work the same, regardless if it's in
>     a config file or in the source code.
>
>     Once again:
>     Assume the spec:
>     p = list()
>
>     Case 1, Conf file:
>     p = 1,2
>
>     Case 2, in source:
>     config['p'] = '1,2'
>
>     In both cases the right-hand side is: 1,2. The quotation marks in case
>     2 is there because of Python syntax. I am still assigning the value
>     1,2 without any quotes, just like in case 1.
>
>     Given the above spec, in both cases, after validation, I expect p to
>     be the list ['1','2']. Currently case 1 gives ['1','2'] while case 2
>     gives a validation error. I don't think it makes sense to get
>     different results given that the right-hand side is the same in both
>     cases.
>
>     --
>     Stefan Parviainen
>
>     ------------------------------------------------------------------------------
>     Learn how Oracle Real Application Clusters (RAC) One Node allows
>     customers
>     to consolidate database storage, standardize their database
>     environment, and,
>     should the need arise, upgrade to a full multi-node Oracle RAC
>     database
>     without downtime or disruption
>     http://p.sf.net/sfu/oracle-sfdevnl
>     _______________________________________________
>     Configobj-develop mailing list
>     Con...@li...
>     <mailto:Con...@li...>
>     https://lists.sourceforge.net/lists/listinfo/configobj-develop
>
>
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment, and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
>
>
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

Re: [Configobj-develop] List handling in Validator

[Stefan P. <pa...@ik...>]

On Tue, Jan 4, 2011 at 5:03 PM, David Hostetler <neg...@gm...> wrote:
> config["foo"] = '1,2'  is Python code!!
OK, it seems clear that my point is not getting across at all. It
seems difficult to convey what I'm trying to say over the internet, so
after this I'm giving up on this point, and will simply continue to
use a workaround in my own program.

Consider the spec:
myint = integer
mylist = list

Assume conf is of type ConfigObj using the above mentioned spec.

conf['myint'] = '1'
conf[mylist'] = '1,2,3'

After validation myint will be converter to integer while mylist will
_not_ be converted to a list. I simply do not understand why one is
converted but the other isn't. You may very well be right, and I'm
wrong, and I see little point in continuing to argue about this weird
behaviour, especially since there's a simple workaround that I already
use in my programs.

-- 
Stefan Parviainen

Re: [Configobj-develop] List handling in Validator

[Michael F. <fuz...@vo...>]

On 04/01/2011 19:01, Stefan Parviainen wrote:
> On Tue, Jan 4, 2011 at 5:03 PM, David Hostetler<neg...@gm...>  wrote:
>> config["foo"] = '1,2'  is Python code!!
> OK, it seems clear that my point is not getting across at all.
You have made your point you just don't seem to have understood the 
responses as to why it doesn't (and shouldn't) work like that. :-)

Michael


> It
> seems difficult to convey what I'm trying to say over the internet, so
> after this I'm giving up on this point, and will simply continue to
> use a workaround in my own program.
>
> Consider the spec:
> myint = integer
> mylist = list
>
> Assume conf is of type ConfigObj using the above mentioned spec.
>
> conf['myint'] = '1'
> conf[mylist'] = '1,2,3'
>
> After validation myint will be converter to integer while mylist will
> _not_ be converted to a list. I simply do not understand why one is
> converted but the other isn't. You may very well be right, and I'm
> wrong, and I see little point in continuing to argue about this weird
> behaviour, especially since there's a simple workaround that I already
> use in my programs.
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

Re: [Configobj-develop] List handling in Validator

[Entity R. <ent...@gm...>]

Wouldn't you mean:
conf['mylist'] = [1,2,3]
Otherwise you are feeding the object a string.

On Jan 4, 2011, at 11:01 AM, Stefan Parviainen <pa...@ik...> wrote:

> On Tue, Jan 4, 2011 at 5:03 PM, David Hostetler <neg...@gm...> wrote:
>> config["foo"] = '1,2'  is Python code!!
> OK, it seems clear that my point is not getting across at all. It
> seems difficult to convey what I'm trying to say over the internet, so
> after this I'm giving up on this point, and will simply continue to
> use a workaround in my own program.
>
> Consider the spec:
> myint = integer
> mylist = list
>
> Assume conf is of type ConfigObj using the above mentioned spec.
>
> conf['myint'] = '1'
> conf[mylist'] = '1,2,3'
>
> After validation myint will be converter to integer while mylist will
> _not_ be converted to a list. I simply do not understand why one is
> converted but the other isn't. You may very well be right, and I'm
> wrong, and I see little point in continuing to argue about this weird
> behaviour, especially since there's a simple workaround that I already
> use in my programs.
>
> --
> Stefan Parviainen
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment, and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop

Re: [Configobj-develop] List handling in Validator

[Stefan P. <pa...@ik...>]

On Tue, Jan 4, 2011 at 9:05 PM, Entity Reborn <ent...@gm...> wrote:
> Wouldn't you mean:
> conf['mylist'] = [1,2,3]
> Otherwise you are feeding the object a string.
If I meant that my entire post would make no sense :-) Yes, I would be
feeding it a string. Some say it shouldn't work that way. Fair enough,
I can accept that feeding a string to something not expecting a string
should produce an error. But why doesn't conf['myint'] = '1' produce
an error? I'm feeding it too a string. Why can I feed a string in one
case but not the other when in neither case a string is what is
expected. What I'm feeding is wrong in both cases: in one case
implicit conversion occurs, in the other it doesn't.

(This post is just to clarify my previous message, as promised I
stopped arguing my point :-)

-- 
Stefan Parviainen