rest2web-develop — A discussion list for rest2web development.

Re: [Rest2web-develop] Re: We forked validate.py

[Michael F. <mi...@pc...>]

Hello Again,

Nicola Larosa wrote:

> [snip..]
>
>  
>
>>Of course it means I have to rework the docs ! Bah.....
>>    
>>
>
>Yes, slightly. ;-) I tried to keep the docstrings up to date, though.
>Speaking of which, we could do worse than generating the API docs with
>epydoc, or something. I'll see what I can do about this too.
>
>
>  
>
Yes - API generation is an interesting subject which I'd like to work on
sometimes. Preferably an answer integrated with docutils in some way or
other.

There are Epydoc generated docs for ConfigObj 3 online (and in the
distro) - and I imagine we'll do the same for ConfigObj 4.

> [snip..]
>
>>In the meantime if you want to implement any more built in tests for
>>validate.py, do the timestamp and a regex test for example.....  feel
>>free :-)
>>    
>>
>
>More polishing, eh? I'll see what I can do...
>
>
>  
>
Well.... a regex test mechanism is more than polishing (it has to be
extensible).

Also I suspect timestamp is non trivial but possible.


>>It looks like we need another project to work on together. Maybe you
>>could look at Firedrop - there's still a lot to do for that. Or we could
>>pick up rest2web....
>>    
>>
>
>Of course we *will*. ConfigObj is useful in itself, and it's a simpler
>testbench for both our collaboration and the introduction of a few new
>tools and techniques, but once done with it I definitely intend to get back
>to rest2web. As far as Firedrop is concerned, we'll see. :-)
>
>  
>

Cool.

> [snip..]
>
>>>So it's easier to chat from Romania than from Britain. Bah! ;-) See what
>>>you can do.
>>>      
>>>
>>Unfortunately that seems to be true at the moment.
>>    
>>
>
>That's a pity, though. :-(
>
>
>  
>

Yes - I enjoyed chatting.

>P.S.: are you still using, and fixing, DrPython? ;-D
>
>  
>
No... SPE is my current IDE. I doubt I will do any development on it
though, just with it. I'm growing to appreciate it more and more..

All the best,

Fuzzball
http://www.voidspace.org.uk/python

Re: [Rest2web-develop] Re: We forked validate.py

[Nicola L. <ni...@te...>]

> Something like that - *sigh*. Staff problems, meetings, customer
> problems, customers..... I'm actually quite fond of the bricks... it's
> the people I struggle with ;-)

You're not the only one. ;-)


>> So, tell me what you think of my last commits to validate.py, before I go
>> on and tackle configreader.py . :-)

> Yes - very impressed. I generally like all the changes. The new errors,
> the neat way you've improved the Validator object and the list tests,
> the docutils tests.

Great. :-)


> Of course it means I have to rework the docs ! Bah.....

Yes, slightly. ;-) I tried to keep the docstrings up to date, though.
Speaking of which, we could do worse than generating the API docs with
epydoc, or something. I'll see what I can do about this too.


> I intend to rename configreader to configobj at some point - and have it
> as a standalone module. I wasn't intending to have an ``__init__.py`` in
> the final version. We've ended up with the wrong filename because my
> first implementation was a reader only.
> 
> In writing the documentation I've uncovered about 15 issues (including
> two bugfixes) that I *need* to resolve. This is along with implementing
> ``writein`` and the comments stuff. All that won't take too long, but
> can you hold off configreader until then ?

Good to know. ;-) Yes, of course. I'm itching to understand its code,
though, so I think I'll give it a read-only look. ;-)


> In the meantime if you want to implement any more built in tests for
> validate.py, do the timestamp and a regex test for example.....  feel
> free :-)

More polishing, eh? I'll see what I can do...


> It looks like we need another project to work on together. Maybe you
> could look at Firedrop - there's still a lot to do for that. Or we could
> pick up rest2web....

Of course we *will*. ConfigObj is useful in itself, and it's a simpler
testbench for both our collaboration and the introduction of a few new
tools and techniques, but once done with it I definitely intend to get back
to rest2web. As far as Firedrop is concerned, we'll see. :-)


>> Individually as opposed to what? I'm not sure I understand.

> Don't worry - my new way is better. It just means re-implementing some
> stuff and rewriting some docs :-)
> 
> Bugger.

What else is new? ;-)


>> So it's easier to chat from Romania than from Britain. Bah! ;-) See what
>> you can do.

> Unfortunately that seems to be true at the moment.

That's a pity, though. :-(


P.S.: are you still using, and fixing, DrPython? ;-D

-- 
Nicola Larosa - ni...@te...

Learning C after learning Python can be done via Pyrex. [...]
Learning Java after learning Python can be done via Jython. [...]
Learning Perl after learning Python can ... never mind. ;-)
 -- André Roberge, May 2005

Re: [Rest2web-develop] Re: We forked validate.py

[Michael F. <mi...@pc...>]

Hello Nicola,

Nicola Larosa wrote:

>>I've got meld - I'll have to merge back in what you've done with the
>>docs. Not today - it's a bit manic :-(
>>    
>>
>
>Bricks flying left and right, eh? ;-)
>
>
>  
>
Something like that - *sigh*. Staff problems, meetings, customer
problems, customers..... I'm actually quite fond of the bricks... it's
the people I struggle with ;-)

>>Obviously for any further work we need to agree specific areas that we
>>should each work on - then we can just discuss how we integrate the
>>different things we do, rather than both working on the same things at
>>the same time !
>>    
>>
>
>So, tell me what you think of my last commits to validate.py, before I go
>on and tackle configreader.py . :-)
>
>
>  
>

Yes - very impressed. I generally like all the changes. The new errors,
the neat way you've improved the Validator object and the list tests,
the docutils tests.

Of course it means I have to rework the docs ! Bah.....

I intend to rename configreader to configobj at some point - and have it
as a standalone module. I wasn't intending to have an ``__init__.py`` in
the final version. We've ended up with the wrong filename because my
first implementation was a reader only.

In writing the documentation I've uncovered about 15 issues (including
two bugfixes) that I *need* to resolve. This is along with implementing
``writein`` and the comments stuff. All that won't take too long, but
can you hold off configreader until then ?

In the meantime if you want to implement any more built in tests for
validate.py, do the timestamp and a regex test for example.....  feel
free :-)

It looks like we need another project to work on together. Maybe you
could look at Firedrop - there's still a lot to do for that. Or we could
pick up rest2web....

>>I do have an implementation question.
>>
>>My current implementation for ConfigObj puts an attribute called
>>'configspec' on the main ConfigObj. This is the parsed configspec with
>>all the tests, the structure follows the expected structure of your
>>config file (obviously).
>>
>>I'm wondering if it would be better to have a configspec attribute for
>>each Section - (as a dictionary). This would allow you to *individually*
>>test values as well.
>>    
>>
>
>Individually as opposed to what? I'm not sure I understand.
>
>  
>
Don't worry - my new way is better. It just means re-implementing some
stuff and rewriting some docs :-)

Bugger.

> [snip..]
>
>
>  
>
>>Anyway - I don't *think* I can blag IRC until I've put a proper effort
>>into poking a hole in our work firewall. (Requires me to construct a
>>website that looks like a building related website - I have registered a
>>domain specially !). *But* - I'm going to try and get Jabber working.
>>    
>>
>
>So it's easier to chat from Romania than from Britain. Bah! ;-) See what
>you can do.
>
>
>  
>
Unfortunately that seems to be true at the moment.


Regards,


Fuzzy
http://www.voidspace.org.uk/python

Re: [Rest2web-develop] Re: We forked validate.py

[Nicola L. <ni...@te...>]

> I've got meld - I'll have to merge back in what you've done with the
> docs. Not today - it's a bit manic :-(

Bricks flying left and right, eh? ;-)


> Obviously for any further work we need to agree specific areas that we
> should each work on - then we can just discuss how we integrate the
> different things we do, rather than both working on the same things at
> the same time !

So, tell me what you think of my last commits to validate.py, before I go
on and tackle configreader.py . :-)


> I do have an implementation question.
>
> My current implementation for ConfigObj puts an attribute called
> 'configspec' on the main ConfigObj. This is the parsed configspec with
> all the tests, the structure follows the expected structure of your
> config file (obviously).
>
> I'm wondering if it would be better to have a configspec attribute for
> each Section - (as a dictionary). This would allow you to *individually*
> test values as well.

Individually as opposed to what? I'm not sure I understand.


> The disadvantage of doing this is that you have to create each section
> at the point you parse the configspec, which is before you parse the
> config file. This means that sections that appear in the configspec, but
> *not* in the config file, will have to be created anyway.

This does not seem a big disadvantage, but again, I'm not sure I get it all.


> What do you think ?

Not much, I'm sure you're be able to explain it better, once the pressure
subsides.


> Anyway - I don't *think* I can blag IRC until I've put a proper effort
> into poking a hole in our work firewall. (Requires me to construct a
> website that looks like a building related website - I have registered a
> domain specially !). *But* - I'm going to try and get Jabber working.

So it's easier to chat from Romania than from Britain. Bah! ;-) See what
you can do.


-- 
Nicola Larosa - ni...@te...

Learning C after learning Python can be done via Pyrex. [...]
Learning Java after learning Python can be done via Jython. [...]
Learning Perl after learning Python can ... never mind. ;-)
 -- André Roberge, May 2005

[Rest2web-develop] Re: We forked validate.py

[Michael F. <mi...@pc...>]

Hello Nicola,

I've got meld - I'll have to merge back in what you've done with the
docs. Not today - it's a bit manic :-(

Obviously for any further work we need to agree specific areas that we
should each work on - then we can just discuss how we integrate the
different things we do, rather than both working on the same things at
the same time !

I do have an implementation question.

My current implementation for ConfigObj puts an attribute called
'configspec' on the main ConfigObj. This is the parsed configspec with
all the tests, the structure follows the expected structure of your
config file (obviously).

I'm wondering if it would be better to have a configspec attribute for
each Section - (as a dictionary). This would allow you to *individually*
test values as well.

The disadvantage of doing this is that you have to create each section
at the point you parse the configspec, which is before you parse the
config file. This means that sections that appear in the configspec, but
*not* in the config file, will have to be created anyway.

What do you think ?

Anyway - I don't *think* I can blag IRC until I've put a proper effort
into poking a hole in our work firewall. (Requires me to construct a
website that looks like a building related website - I have registered a
domain specially !). *But* - I'm going to try and get Jabber working.

Best Regards,

Fuzzy
http://www.voidspace.org.uk/python

Nicola Larosa wrote:
> 
>>>>Michael,
>>>>I'm looking (via the wonderful Meld program) at the differences between the
>>>>last validate.py I committed, and the one you sent me yesterday.
>>>>
>>>>They do not have much in common. It looks like we started from a common
>>>>point, and then worked indipendently from each other, going in two
>>>>different directions. Probably you took a copy for working on it during
>>>>your travel before my commit, that I did on Sunday 10.
> 
> 
>>>Yes - I left on Saturday the ninth.
>>>
>>>As soon as you said you'd worked on it my heart sank. I've appreciated
>>>your input so far and was looking forward to actually coding with you -
>>>*and* moving to doctests is a very good idea.
>>>
>>>*But* - we'd already discussed how the previous system for validate.py
>>>was insufficient - it needed to do type conversion instead of just
>>>returning True/False for pass/fail. I didn't see any alternative but
>>>reworking chunks of it.
> 
> 
> Of course. I'm not looking for someone to blame, mostly because that
> someone would be me. :-) I got enthusiastic and eager to use doctest, and
> somehow forgot our discussion.
> 
> I'll reuse most of what I did anyway, it was all good experience.
> 
> 
> 
>>>>We'll have to better coordinate our efforts in the future, to avoid
>>>>repeating such unfortunate occurrences. I now intend to adapt the tests in
>>>>your new version to doctest, dropping the one I committed, but this time
>>>>I'll wait for confirmation from you before proceeding.
> 
> 
>>>Ok - I'm pretty sure I've made no changes to validate.py since I sent it
>>>to you. Feel free to add an alphanumeric test (and/or url_safe test which
>>>might be a more useful set of characters to test for).
> 
> 
> Mmh... alphanumeric? url_safe? Not exactly the answer I was waiting for,
> but I'll go ahead and integrate doctest in the new validate.py, while
> crossing fingers at the same time (kind of hard programming this way,
> actually...). ;-)
> 
> --
> Nicola Larosa - ni...@te...
> 
> Learning C after learning Python can be done via Pyrex. [...]
> Learning Java after learning Python can be done via Jython. [...]
> Learning Perl after learning Python can ... never mind. ;-)
>  -- André Roberge, May 2005
> 

[Rest2web-develop] More work on validate.py

[Nicola L. <ni...@te...>]

Done three commits about validate.py .

First I committed the version you emailed me last week.

Then I moved (again ;-) ) the tests to the doctest format, and made some
refactoring and more code cleanup.

Finally I strengthened the value checks, introducing more ValidationError
subclasses, and adding many more tests, plus more refactoring and cleanup.

See the changelog for more details, only two external changes: renamed the
"date" value type to "timestamp" (no implementation yet), and the "bool"
one to "boolean". Easy to change them back, if you don't like them. :-)

I didn't yet add *all* the tests I would like to, wanted to leave some of
the fun to you too. ;-)

-- 
Nicola Larosa - ni...@te...

Learning C after learning Python can be done via Pyrex. [...]
Learning Java after learning Python can be done via Jython. [...]
Learning Perl after learning Python can ... never mind. ;-)
 -- André Roberge, May 2005

[Rest2web-develop] Re: rest2web

[<mi...@pc...>]

Hello Dirk,

I need to test the 'thispage' stuff - looks like a bug to me=2E
Sticking to the order of  'section-list' seems like a no-brainer to me=2E
I'll consider adding a 'page-order' variable=2E

On principle I'm *against* adding extra variables - in the name of
simplicity(I already have too many !)=2E *But* - I see your use case and i=
t
is something  I'd wondered if someone would want=2E

I'll be back in action on Monday and will think about it then=2E How would=

you reference pages =3F (by filename is the only reasonable way I guess=2E=
=2E=2E=2E=2E)

Feel free to move this to the rest2web-develop mailing list (link on the
sourceforge page http://sourceforge=2Enet/projects/rest2web ) if you *want=
*=2E

Currently most discussion is about the specification for ConfigObj 4- the
*fantastic* new config system that is nearly finished=2E=2E=2E ;-)

Best Regards,

Fuzzyman
http://www=2Evoidspace=2Eorg=2Euk/python

Original Message:
-----------------
From: Dirk Steenpass steenpaz@sdf-eu=2Eorg
Date: Wed, 13 Jul 2005 16:47:21 +0200
To: root@voidspace=2Eorg=2Euk
Subject: Re: rest2web


Dear Fuzzyman,

thanks for your quick reply and apologies for my late answer=2E

I see the point of not specifying everything because pages are generated
dynamically=2E I could use templates for pages when I want an ordered
sidebar menue=2E However, I'm a lazy guy and I would like to get away
using as few templates as possible=2E I am thinking about introducing an
additional restindex variable 'page-order', that get's evaluated if
present=2E Similar to 'section-list' 'page-order' has a meaning on index
pages only=2E I will probably take a look into this tomorrow =2E=2E=2E


The 'thispage' question: I tried to say the following:

root=5Fdir/
    index=5Fpage                 <- thispage !=3D None

    subdir/
        index=5Fpage             <- thispage !=3D None
        page=5F1
        page=5F2

   subdir/
        index=5Fpage            <- thispage =3D=3D None

In words: the 'thispage' variable is None for index pages that are leaves
of the indextree=2E As index pages are treated special several times I was=

wondering whether this is a bug or a feature=2E

best regards, dirk


On Sun, Jul 10, 2005 at 08:26:15AM -0700, root wrote:
> >On Sun, 10 Jul 2005 01:20:43  0200 Dirk Steenpass <steenpaz@sdf-eu=2Eor=
g>
> wrote=2E
> >Dear Mr=2E Foord,
> >
> >I have recently discovered rest2web (via ReStructuredText
> >documentation)=2E I really like the rest2web idea, as I am sick of
> >writing plain html and a full blown content managment is an overkill fo=
r
> >my needs=2E
> >
> >I would like to bother you with a few questions about rest2web 0=2E3=2E=
0
> >hoping that your answers save me from code diving ;-)=2E
> >
> >
>=20
> Hello Dirk,
>=20
> I'm glad that you might find rest2web helpful - and I'd be very pleased =
to
> answer your questions about it=2E=20
>=20
> I'm away in Romania for the next week - so some of your questions I migh=
t
> have to investigate when I return=2E
>=20
> >
> >
> >When using minibar, is there a way to control the order of pages within=

> >a section=3F
> >
> >
> >When using minibar, is there a way to control the order of sections=3F =
It
> >seems that the order of section specifications is not honored=3F
> >
>=20
> Not currently - although honouring the order they are specified in seems=
 a
> very sensible idea=2E You can see the code in the minibar function in (I=

> think) the file ``functions=2Epy``=2E
>=20
> The reason I didn't add too much code to specify the position of
everything
> is that it's supposed to be dynamic - you're supposed to be able to add
> whole sections *without* having to specify where they go=2E Otherwise yo=
u
> might as well just put the links in the template yourself !  =20
>=20
> However - your idea is a good one=2E When I can look at implementing it=2E=
 It
> should be quite easy=2E
>=20
> >
> >
> >The 'thispage' value is not None for every page that is build, with the=

> >exception of index pages that are leafs of the indextree=2E Is this is =
a
> >bug or a feature=3F
> >
>=20
> I don't fully understand the question - and I'm not able to test=2E
'thispage'
> *shouldn't* be None as it's a pointer to the current page in the
'indextree'
> structure=2E=20
>=20
> Are you saying that I document it as being None - or that *sometimes* it=

is
> None =3F
>=20
> Best Regards,
>=20
> Fuzzyman
>=20
>=20
>=20
>=20
> >
> >
> >
> >bis denn, dirk
> >
> >
>=20

--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web=2Ecom/ =2E

Re: [Rest2web-develop] Goodbye....

[Nicola L. <ni...@te...>]

> Fantastic the doctests are very nice. 

I'm glad you like'em. :-)


> Unfortunately all those test functions need rewriting - as they need to 
> not just test values (return True/False) but transform values as well. 

No problem, now that the tests work and are in-context, it should be easy
to update and use them to check the changes as they are done.


> I've done the changes to the Validator object that implement this on my 
> PDA !

Great, so attach them to an email and I'll commit them. ;-)


> I'd also thought of ripping out all the regex stuff and rethinking it.
> 
> The example functions as they stand aren't much use I don't think. 
> Written by Mark Andrews. I've rewritten them for basic data types - 
> more complex tests can be built on top of these !

I reworked some of them too, as you may have seen.


> We can discuss which of the other tests are worth keeping when I return.
> 
> Do what you want to configreader.py. (Within reason). My project for my 
> time away is to start on the docs for ConfigObj. (Using my PDA *grin*).

I'll convert the tests in ConfigObj to doctest format too, as soon as I can.


> It's probably best to leave validate.py for now - until you can see the 
> basic data types I've kept (do we really *need* to test complex 
> numbers !) - and we decide which of the other tests are worth keeping 
> (and changing so they work with the new system).

Ok, I'll leave it alone, until your changes show up.

-- 
Nicola Larosa - ni...@te...

Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005

Re: [Rest2web-develop] Goodbye....

[Nicola L. <ni...@te...>]

> Well, That's it. I'm off to Romania.

Great, enjoy your vacation.


> I will still have email though :-)

Maybe you'll wish you did not... ;-)


> I've now changed the validate support in ConfigObj - so I just need to
> make the corresponding changes in validate.py - not very difficult.
> It's just about return values and then coding some nice examples.

Upon your return, you'll find validate.py rather changed. :-)

I'm not confining myself to style cleanups any more, here's the changelog:

> 2005/07/10
> By Nicola Larosa
> Tests converted to doctest format and placed in the respective docstrings,
>   so they are automatically checked, and easier to update
> Added many (code) tests, and made test_* methods more robust
> Added test_title to Validator.functions dict
> log_err method added to Validator class, and used in re_set method; it
>   outputs to stdout instead of stderr, for testing reasons
> Validator.functions dict and test functions reordered alphabetically
> Some more code cleanup

Now when you run

$ python validate.py

there's no output anymore. But if you run

$ python validate.py -v

you get a long display ending with:

> 9 items had no tests:
>     __main__
>     __main__.Validator.__init__
>     __main__.Validator._none
>     __main__.Validator.function_parse
>     __main__.Validator.function_test
>     __main__.Validator.log_err
>     __main__.Validator.test
>     __main__._test
>     __main__._test_range
> 24 items passed all tests:
>    4 tests in __main__.Validator
>    1 tests in __main__.Validator.getargs
>    1 tests in __main__.Validator.re_set
>    9 tests in __main__.Validator.re_test
>    3 tests in __main__.Validator.test_multiple
>    1 tests in __main__.Validator.tests
>    1 tests in __main__.Validator.validate
>    2 tests in __main__.metatest
>    2 tests in __main__.test_alnum
>    2 tests in __main__.test_alpha
>    3 tests in __main__.test_complex
>    2 tests in __main__.test_digit
>    3 tests in __main__.test_float
>    2 tests in __main__.test_floatrange
>    4 tests in __main__.test_int
>    2 tests in __main__.test_intrange
>    1 tests in __main__.test_length
>    1 tests in __main__.test_list
>    2 tests in __main__.test_lower
>    1 tests in __main__.test_option
>    3 tests in __main__.test_size
>    2 tests in __main__.test_string
>    3 tests in __main__.test_title
>    5 tests in __main__.test_upper
> 60 tests in 33 items.
> 60 passed and 0 failed.
> Test passed.

Rather nice, isn't it? :-D

I know we didn't discuss this change before, and you stated, in the TODO
section, your intention of using unit testing, but the doctest approach
seems nicer to me, and it can be easily integrated in a larger unit testing
framework anyway.

(I'll leave the other points up for discussion when you're back.)


> Have a good time..... (I will)

Yes, I am, as you can see. Now on to tackle configreader.py...

-- 
Nicola Larosa - ni...@te...

Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005

[Rest2web-develop] Goodbye....

[Michael F. <mi...@pc...>]

Well, That's it. I'm off to Romania.

I will still have email though :-)

I've now changed the validate support in ConfigObj - so I just need to
make the corresponding changes in validate.py - not very difficult.
It's just about return values and then coding some nice examples.

Have a good time..... (I will)

Best Regards,

Fuzzyman
http://www.voidspace.org.uk/python

[Rest2web-develop] Stringify

[Michael F. <mi...@pc...>]

By the way... I've just implemented the ``stringify`` option in
ConfigObj 4 and committed.

It's on by default - values are converted to strings when written. If
you switch it off, type checking is done when you set values. (Only
strings or lists of strings).

At the moment the ``str`` function is used to convert values to
strings. For somethings (e.g. datetime instances) this *might* not be
suitable - we might need to special case.

Best Regards,

Fuzzy
http://www.voidspace.org.uk/python

Re: [Rest2web-develop] ConfigObj Commit

[Nicola L. <ni...@te...>]

> But seeing as you *specify* what you want, in these cases you probably
> want to leave it as a string. See below about the fact that I'm not
> *really* suggesting *automatic* type conversion.
> 
> As you say in the other email - it's a bit like duck typing, the
> conversion is only done if you actually need (specify) it.

And that's as it should be. :-)


> I can write enough general case functions that will cover all the above
> examples - and anything else that you will need.

I can write something too sometimes, you know, even if it doesn't look like
that. ;-)


> According to YAGNI I shouldn't write anything you don't actually want -
> so give me your use cases and I'll work to those..... The above ones are
> obvious though I guess.....

You shouldn't write anything that *you* don't actually want. I'll take care
of my use cases in due course, your hands are full enough already. :-)

-- 
Nicola Larosa - ni...@te...

Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005

Re: [Rest2web-develop] ConfigObj Commit

[Michael F. <mi...@pc...>]

Nicola Larosa wrote:
[snip..]
>>I *might* have some time during the day to look at the validate issues.
>>(type checking and type conversion).
>>First of all I'll add stringify and transforming validation.
>>This will do the type conversion when you write files and when you validate.
>>
>>Can you think of any config information you might want to store where
>>the *type* of the value will be ambiguous ?
>>For example any config options where you might want a string or a number ?
> 
> 
> A phone number, or zip code, could be ambiguous.
> 

But seeing as you *specify* what you want, in these cases you probably
want to leave it as a string. See below about the fact that I'm not
*really* suggesting *automatic* type conversion.

As you say in the other email - it's a bit like duck typing, the
conversion is only done if you actually need (specify) it.

> 
> 
>>The key thing is having a good standard set of validate functions that
>>take care of all the common cases .
>>Basic types :
>>
>>    float
>>    decimal
>>    integer
>>    string
>>    date/time
>>    boolean   
>>    None
>>    lists of any of these
>>
>>All these basic sorts it would be *very* simple to detect and
>>automatically convert.
> 
> 
> Simple, yes. Not *very* so in the date/time case, I fear.
> 

Sure - there are modules that will attempt to cover the most common ways
of specifying dates though.

I *don't* think I'm really suggesting *automatic* type conversion - you
still use the ``validate`` method and *specify* in your configspec what
conversion/check to apply to each value.

I can write enough general case functions that will cover all the above
examples - and anything else that you will need.

According to YAGNI I shouldn't write anything you don't actually want -
so give me your use cases and I'll work to those..... The above ones are
obvious though I guess.....

Best Regards,

Fuzzball
http://www.voidspace.org.uk/python

Re: [Rest2web-develop] Re: ConfigObj 4 work

[Nicola L. <ni...@te...>]

> I don't think we need to specify the type of the value stored - it ought
> to be unambiguous in the light of the *purpose* of the value. i.e. a
> 'port number' is always going to be an integer, a 'user name' is always
> a string etc. This means that all the validation schema needs to do is
> check that the stored value *makes sense* (can be sensibly converted to
> the required type).

In other words, "duck typing". Very Pythonic. :-)


> I think this is the stage to explain the 'schema' system - using
> ``validate.py``. You can decide whether this meets your needs - or can
> be made to meet your needs.
> 
> The configspec is a file (list of lines, dictionary, whatever) that is
> very like the config file. For each member of the config file, you
> specify (effectively) a function to do the testing.
> 
> e.g. (from the tests)
> 
>         test1='range(30,50)'
>         test2='lower()'
>         test3='integer'
>         test4='alphanumeric'
> 
> The function can be specified with or without parameters. If there are
> no parameters then the '()' is optional. ``val.test`` then maps the
> value in the configspec to a real function call. The function specified
> is passed the value (as a string *or* a list of strings when read from
> the config file) - *plus* any parameters specified in the config spec.
> 
> This means that, from the example above, the member 'test1' must be an
> integer value between 30 and 50.
> 
> ``lower`` checks the value is lowercase, ``integer`` just checks that
> the value is an integer etc.
> 
> So I *don't* think we need to store any type information in the config
> file. Once you have written a general purpose set of functions that
> check the values make sense - all you need to do is write the
> configspec. The validate method then checks each value individually.
> 
> One way of doing it would be to have the functions return the converted
> value *or* raise an exception. They would look something like this :
> 
> def range(value, min, max):
>     if isinstance(value, list):
>         raise Exception
>     if not value.isdigit():
>         raise Exception
>     value = int(value)
>     if not min <= value <= max:
>         raise Exception
>     return value
> 
> If the member is missing, or the test raises an Exception, then it's a
> fail. Otherwise the original value is replaced with the returned new value.
> 
> So we don't need to specify any type info in the config file - the type
> should be obvious from context.

Nobody said that. ;-) I do implied that we could have a some kind of schema
file, where the types would be specified. That's what I know, because
that's what's ZConfig has. Your configspec file looks even better, though,
since it contains constraints on the possible values.


> How does this sound ?
> We can work together on a general set of functions that will cover most
> of your needs (the syntax already allows for your configspec to pass
> keyword arguments or specify multiple tests for a value).
> 
> A lot of are written already - but I think they could be clearer.


>> If the schema specifies the type for params, you catch the problem right
>> away. The stringification should only happen later, when writing to file,
>> and shouldn't be burdened with type checking concerns.

> Hmm.. I only envisaged using the schema (configspec) to check a file
> once you have read it, although there's no reason you couldn't set the
> values and then validate.
> 
> The schema isn't involved when you *set* individual values. You would
> have to validate specifically afterwards.... (allowing validation of
> individual values at the point when they are set would be messy - I was
> only suggesting type checking in the context of all values being strings
> for writing to a file).

OTOH, checking only when saving means disconnecting the check point from
the set point. Unfortunately, checking at set point means keeping
constraints in the same object holding the data, a firm step towards
ZConfig-level complexity. :-) So yes, I think it's wise dropping such a
requirement.


>>> If we allow the program to crash in the ``write`` method (as currently)
>>> - this may actually be called from a totally different part of your
>>> application that set the value - hard to track.
>>>
>>> If we do type checking when you set the value - the code that sets the
>>> value is the code that raises the error - easier to trace.

>> Indeed.

And this is what we just dropped. :-(


> The nice thing about using ``validate.py`` is that you can use the same
> system to check values wherever they come from.

Or wherever they go to. :-)


> Mark Andrews - who I
> designed this with, wanted to use it to validate the same values stored
> in a database and a file.


>>> lol - OTOH their nicely lined up config files will become all jagged,
>>> but unavoidable really.

>> Why? I was thinking about the opposite outcome instead, a lesson in tidying
>> up. :-)

> Well, because if you add white space to your values to line up the
> comments etc.... then the write method will simply add a standard amount
> of whitespace after the value and before the comment.
> 
> It will also write a standard amount of whitespace between value and
> divider, rather than padding it so that they line up.

I reckon there's a specific mention in GvR guidelines to not depend on such
aligning. Not that I hope that many users of ConfigObj will have read them. ;-)

-- 
Nicola Larosa - ni...@te...

Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005

[Rest2web-develop] ConfigObj Commit

[Michael F. <mi...@pc...>]

Hello Nicola,

Just committed more changes to ConfigObj.

Renamed private attributes with a single underscore prefix.
Changes to interpolation - exceeding recursion depth,
    or specifying a missing value, now raise errors.
Changes for Python 2.2 compatibility.
    (changed boolean tests - removed ``is True`` and ``is False``)
Added test for duplicate section and member (and fixed bug)


I *might* have some time during the day to look at the validate issues.
(type checking and type conversion).
First of all I'll add stringify and transforming validation.
This will do the type conversion when you write files and when you validate.

Can you think of any config information you might want to store where
the *type* of the value will be ambiguous ?
For example any config options where you might want a string or a number ?

The key thing is having a good standard set of validate functions that
take care of all the common cases .
Basic types :

    float
    decimal
    integer
    string
    date/time
    boolean   
    None
    lists of any of these

All these basic sorts it would be *very* simple to detect and
automatically convert.

Mark Andrews also favoured a regex approach as well - for things like
post codes (zip codes), phone numbers, IP addresses. Obviously this will
only check that the value matches the spec - not do any transformation.

Can you tell that I'm procrastinating writing the ``writein`` method...

Best Regards,

Fuzzy
http://www.voidspace.org.uk/python

RE: [Rest2web-develop] Heresy!

[Michael F. <mi...@pc...>]

Hmmmm.... Syck is also a C extension. That may not be a show stopper, =
but it's a factor. I'd definitely like to try it.=20

Looks like ``syck.load``returns a dictionary.

It's certainly going to be safer than pickle (known security =
vulnerabilities). I don't know how easy it is to write YAML by hand. =
Doesn't look too bad.

Fuzz

-----Original Message-----
   >From: "Nicola Larosa"<ni...@te...>
   >Sent: 07/07/05 21:40:22
   >To: =
"res...@li..."<res...@li...urcefor=
ge.net>
   >Subject: [Rest2web-develop] Heresy!
   >
   >A blasphemous thought: what about YAML ( http://yaml.org/start.html =
)?
   >
   >Syck ( http://whytheluckystiff.net/syck/ ) seems to be fairly =
maintained.
   >
   >Trying it now.
   >
   >--=20
   >Nicola Larosa - ni...@te...
   >
   >Adding things just because you can leads to monstrosities like =
Common LISP,
   >PL/I, Algol 68 and Perl 6. Adding features only when they add =
functionality
   >(or better yet, by removing restrictions) leads to jewels like =
Python,
   >Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005
   >
   >
   >
   >
   >-------------------------------------------------------
   >This SF.Net email is sponsored by the 'Do More With Dual!' webinar =
happening
   >July 14 at 8am PDT/11am EDT. We invite you to explore the latest in =
dual
   >core and dual graphics technology at this free one hour event hosted =
by HP,
   >AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
   >_______________________________________________
   >Rest2web-develop mailing list
   >Res...@li...
   >
   >

[Message truncated. Tap Edit->Mark for Download to get remaining =
portion.]

RE: [Rest2web-develop] Heresy!

[Michael F. <mi...@pc...>]

-----Original Message-----
   >From: "Nicola Larosa"<ni...@te...>
   >Sent: 07/07/05 21:40:22
   >To: =
"res...@li..."<res...@li...urcefor=
ge.net>
   >Subject: [Rest2web-develop] Heresy!
   >
   >A blasphemous thought: what about YAML ( http://yaml.org/start.html =
)?
   >
   >Syck ( http://whytheluckystiff.net/syck/ ) seems to be fairly =
maintained.
   >
   >Trying it now.
   >

Ha ! :-)
Well - I'll still create ConfigObj 4, not much to do, and I need nested =
sections for several of my own projects.

www.pyyaml.org doesn't seem to exist any more.
syck looks interesting, useful for data persistence. I'm not so sure =
about application config though. It also depends what programmers =
interface it provides.

You'd still have to implement some kind of validation scheme - even if =
syck looks like it retains some kind of type data.

Regards,

Fuzz


   >--=20
   >Nicola Larosa - ni...@te...
   >
   >Adding things just because you can leads to monstrosities like =
Common LISP,
   >PL/I, Algol 68 and Perl 6. Adding features only when they add =
functionality
   >(or better yet, by removing restrictions) leads to jewels like =
Python,
   >Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005
   >
   >
   >
   >
   >-------------------------------------------------------
   >This SF.Net email is sponsored by the 'Do More With Dual!' webinar =
happening
   >July 14 at 8am PDT/11am EDT. We invite you to explore the latest in =
dual
   >core and dual graphics technology at this free one hour event hosted =
by HP,
   >AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
   >_______________________________________________
   >Rest2web-develop mailing list
   >Res...@li...
   >
   >

[Message truncated. Tap Edit->Mark for Download to get remaining =
portion.]

[Rest2web-develop] Heresy!

[Nicola L. <ni...@te...>]

A blasphemous thought: what about YAML ( http://yaml.org/start.html )?

Syck ( http://whytheluckystiff.net/syck/ ) seems to be fairly maintained.

Trying it now.

-- 
Nicola Larosa - ni...@te...

Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005

Re: [Rest2web-develop] Re: ConfigObj 4 work

[Michael F. <mi...@pc...>]

Nicola Larosa wrote:
>>Ok - we have either a confusion or a difficulty ;-)
>>
>>Everything is written as a string - surprise, surprise. That means when
>>it's read back, it's a string again.
>>
>>It sounds to me like you want the validation process to also do type
>>conversion ? (So that a value you set as an integer - although written
>>and read back in as a string, becomes an integer after validation).
> 
> 
> That's correct.
> 
> 
> 
>>Currently validation checks whether your read values (strings) conform
>>to whatever standard you set. It currently *doesn't* allow the
>>validation process to alter the stored values.
>>
>>``validate.test`` returns a single True/False - pass/fail for each test.
>>
>>I could also write a transforming test that returns a tuple of : (pass,
>>newvalue)
>>
>>This would allow the validation process to also transform values for you.
>>It means two things :
>>
>>    a) You must be allowed to set non string values on a ConfigObj
>>    b) You need a stringify option to transform all non-string values
>>when writing out
> 
> 
> A bidirectional to-from-string typing system. I'm still vague about the
> schema that describes types. Maybe we could take a page from Python and let
> the types be dynamic, but then how do we recognize the type of a value in a
> string read from the config file?
> 

I don't think we need to specify the type of the value stored - it ought
to be unambiguous in the light of the *purpose* of the value. i.e. a
'port number' is always going to be an integer, a 'user name' is always
a string etc. This means that all the validation schema needs to do is
check that the stored value *makes sense* (can be sensibly converted to
the required type).

I think this is the stage to explain the 'schema' system - using
``validate.py``. You can decide whether this meets your needs - or can
be made to meet your needs.

The configspec is a file (list of lines, dictionary, whatever) that is
very like the config file. For each member of the config file, you
specify (effectively) a function to do the testing.

e.g. (from the tests)

        test1='range(30,50)'
        test2='lower()'
        test3='integer'
        test4='alphanumeric'

The function can be specified with or without parameters. If there are
no parameters then the '()' is optional. ``val.test`` then maps the
value in the configspec to a real function call. The function specified
is passed the value (as a string *or* a list of strings when read from
the config file) - *plus* any parameters specified in the config spec.

This means that, from the example above, the member 'test1' must be an
integer value between 30 and 50.

``lower`` checks the value is lowercase, ``integer`` just checks that
the value is an integer etc.

So I *don't* think we need to store any type information in the config
file. Once you have written a general purpose set of functions that
check the values make sense - all you need to do is write the
configspec. The validate method then checks each value individually.

One way of doing it would be to have the functions return the converted
value *or* raise an exception. They would look something like this :

def range(value, min, max):
    if isinstance(value, list):
        raise Exception
    if not value.isdigit():
        raise Exception
    value = int(value)
    if not min <= value <= max:
        raise Exception
    return value

If the member is missing, or the test raises an Exception, then it's a
fail. Otherwise the original value is replaced with the returned new value.

So we don't need to specify any type info in the config file - the type
should be obvious from context.

How does this sound ?
We can work together on a general set of functions that will cover most
of your needs (the syntax already allows for your configspec to pass
keyword arguments or specify multiple tests for a value).

A lot of are written already - but I think they could be clearer.

> 
> 
>>The sort of type checking I'm talking about is when your program has a
>>bug and you accidentally set a value to be ``None`` or to be a class
>>instance by mistake. When you come to write out the file - we crash
>>because the value isn't a string.
>>
>>If we *automatically* stringify non-string values, you won't see the
>>problem till much later.
> 
> 
> If the schema specifies the type for params, you catch the problem right
> away. The stringification should only happen later, when writing to file,
> and shouldn't be burdened with type checking concerns.
> 

Hmm.. I only envisaged using the schema (configspec) to check a file
once you have read it, although there's no reason you couldn't set the
values and then validate.

The schema isn't involved when you *set* individual values. You would
have to validate specifically afterwards.... (allowing validation of
individual values at the point when they are set would be messy - I was
only suggesting type checking in the context of all values being strings
for writing to a file).

> 
> 
>>If we allow the program to crash in the ``write`` method (as currently)
>>- this may actually be called from a totally different part of your
>>application that set the value - hard to track.
>>
>>If we do type checking when you set the value - the code that sets the
>>value is the code that raises the error - easier to trace.
> 
> 
> Indeed.
> 
> 
> 
>>My view is that a ConfigObj represents the *config file*, and so it
>>makes sense to keep values as strings. Validation allows you to check
>>that your string does represent an integer, or represent a bool, or
>>whatever - but you do the conversion yourself.
>>
>>You want (I think ?) the ConfigObj to represent the config data - so you
>>want the validate option to change the type of value from strings to
>>integers etc.
> 
> 
> The config file is just a kind of backend, an implementation detail. It
> could be a database connection, a link to cyberspace, or a time-space
> machine that reaches back to Pico de la Mirandola. ;-) What I'm interested
> in is the config *data*, in fact. The config data should be accurate, and
> consistently typed. Such concerns are better handled in the config system,
> instead of diluting them in the main program.
> 

The nice thing about using ``validate.py`` is that you can use the same
system to check values wherever they come from. Mark Andrews - who I
designed this with, wanted to use it to validate the same values stored
in a database and a file.


> This of course complicates the config system. ZConfig, for instance, goes
> to great lengths to embed the config schema and types in the config objects
> themselves, reaching unheard-of levels of complexity. :-|
> 

Ha....

> 
> 
>>I'm quite happy to add to validate so it can transform values, add
>>*optional* type checking, and add a ``stringify`` option that is *on* by
>>default. This means ConfigObj will work with either model.
>>
>>There will be a new keyword for the validate method - which will cause
>>it to *either* use ``val.test`` or ``val.transforming_test``.
> 
> 
> So, how do you specify what the value types should be? You have talked
> about this before, but it's still not clear to me.
> 


*Hopefully* I've answered this above.


> 
> 
[snip..]
> 
>>>>5) Should I attempt to preserve the amount of whitespace around the
>>>>divider, the type of quoting used, and the amount of whitespace before
>>>>the comment - and then re-use it in the write method ? I'd rather not....
> 
> 
>>>Absolutely not. Reformat mercilessly. After a while, the user will
>>>capitulate, and conform. ;-)
> 
> 
>>lol - OTOH their nicely lined up config files will become all jagged,
>>but unavoidable really.
> 
> 
> Why? I was thinking about the opposite outcome instead, a lesson in tidying
> up. :-)
> 

Well, because if you add white space to your values to line up the
comments etc.... then the write method will simply add a standard amount
of whitespace after the value and before the comment.

It will also write a standard amount of whitespace between value and
divider, rather than padding it so that they line up.

Best Regards,


Fuzzy
http://www.voidspace.org.uk/python

Re: [Rest2web-develop] Re: ConfigObj 4 work

[Nicola L. <ni...@te...>]

> Ok - we have either a confusion or a difficulty ;-)
> 
> Everything is written as a string - surprise, surprise. That means when
> it's read back, it's a string again.
> 
> It sounds to me like you want the validation process to also do type
> conversion ? (So that a value you set as an integer - although written
> and read back in as a string, becomes an integer after validation).

That's correct.


> Currently validation checks whether your read values (strings) conform
> to whatever standard you set. It currently *doesn't* allow the
> validation process to alter the stored values.
> 
> ``validate.test`` returns a single True/False - pass/fail for each test.
> 
> I could also write a transforming test that returns a tuple of : (pass,
> newvalue)
> 
> This would allow the validation process to also transform values for you.
> It means two things :
> 
>     a) You must be allowed to set non string values on a ConfigObj
>     b) You need a stringify option to transform all non-string values
> when writing out

A bidirectional to-from-string typing system. I'm still vague about the
schema that describes types. Maybe we could take a page from Python and let
the types be dynamic, but then how do we recognize the type of a value in a
string read from the config file?


> The sort of type checking I'm talking about is when your program has a
> bug and you accidentally set a value to be ``None`` or to be a class
> instance by mistake. When you come to write out the file - we crash
> because the value isn't a string.
> 
> If we *automatically* stringify non-string values, you won't see the
> problem till much later.

If the schema specifies the type for params, you catch the problem right
away. The stringification should only happen later, when writing to file,
and shouldn't be burdened with type checking concerns.


> If we allow the program to crash in the ``write`` method (as currently)
> - this may actually be called from a totally different part of your
> application that set the value - hard to track.
> 
> If we do type checking when you set the value - the code that sets the
> value is the code that raises the error - easier to trace.

Indeed.


> My view is that a ConfigObj represents the *config file*, and so it
> makes sense to keep values as strings. Validation allows you to check
> that your string does represent an integer, or represent a bool, or
> whatever - but you do the conversion yourself.
> 
> You want (I think ?) the ConfigObj to represent the config data - so you
> want the validate option to change the type of value from strings to
> integers etc.

The config file is just a kind of backend, an implementation detail. It
could be a database connection, a link to cyberspace, or a time-space
machine that reaches back to Pico de la Mirandola. ;-) What I'm interested
in is the config *data*, in fact. The config data should be accurate, and
consistently typed. Such concerns are better handled in the config system,
instead of diluting them in the main program.

This of course complicates the config system. ZConfig, for instance, goes
to great lengths to embed the config schema and types in the config objects
themselves, reaching unheard-of levels of complexity. :-|


> I'm quite happy to add to validate so it can transform values, add
> *optional* type checking, and add a ``stringify`` option that is *on* by
> default. This means ConfigObj will work with either model.
> 
> There will be a new keyword for the validate method - which will cause
> it to *either* use ``val.test`` or ``val.transforming_test``.

So, how do you specify what the value types should be? You have talked
about this before, but it's still not clear to me.


>>> 3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
>>> Shall I reimplement this ?

>> No, you shan't. ;-) Let them pound on the hash key. YAGNI rules. Less is
>> better than more. All that jazz. ;-)

> YAGNI ?????

You Ain't Gonna Need It. A popular acronym around Python circles.


> Anyway - cool.. I'll lose multiline comments.

>>> 5) Should I attempt to preserve the amount of whitespace around the
>>> divider, the type of quoting used, and the amount of whitespace before
>>> the comment - and then re-use it in the write method ? I'd rather not....

>> Absolutely not. Reformat mercilessly. After a while, the user will
>> capitulate, and conform. ;-)

> lol - OTOH their nicely lined up config files will become all jagged,
> but unavoidable really.

Why? I was thinking about the opposite outcome instead, a lesson in tidying
up. :-)


>>>> Then I'll revert to signing only, not may big secrets to protect anyway. ;-)

>>> :-)
>>>
>>> Not yet.

>> In Italy we say "Hope dies last." Do you have anything like that?

> Hmm... I can't think of anything directly equivalent. Their maybe
> though. Ha - doesn't sound too hopeful though ;-)

Italians often manage to be catholic and sceptic at the same time. ;-)


> Cool - useful stuff.

:-)


-- 
Nicola Larosa - ni...@te...

Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005

Re: [Rest2web-develop] Re: ConfigObj 4 work

[Michael F. <mi...@pc...>]

Hello Nicola,

(and anyone else :-)

Nicola Larosa wrote:
>>I'll list the questions I have, answer any you can be bothered with :-)
> 
> 
> I'll try. :-)
> 

You did pretty well.

> 
> 
>>1) Part of the point of ConfigObj is that you can create config files
>>from a program. Something like :
>>
>>	config = ConfigObj()
>>	config.filename = 'new.ini'
>>	config['member1'] = value1
>>	config['section1'] = {}
>>	config['section1']['member1'] = value1
>>	...
>>
>>	config.write()
> 
> 
> I like this, I need this. :-)
> 

It's one of the nicest things about ConfigObj. It's also why I end up
using it for data persistence.

> 
> 
>>As it is to be written to a file - the only valid values to set are
>>strings, lists of strings, or dictionaries (containing strings or lists
>>of strings or...). Should we do type checking when the value is set ?
>>(not very difficult). This means the error would be raised when you *set
>>the value*, rather than when you try to write the file - which may be
>>another part of the program.
> 
> 
> Can't see the need for this limitation. The datatype may be any: a number,
> a flag, a date. Its *representation* is a string: it will be written to
> file, and when read back, the data will be converted back to its type, with
> validation.
> 
> Anything less than this is insufficient.
> 

Ok - we have either a confusion or a difficulty ;-)

Everything is written as a string - surprise, surprise. That means when
it's read back, it's a string again.

It sounds to me like you want the validation process to also do type
conversion ? (So that a value you set as an integer - although written
and read back in as a string, becomes an integer after validation).

Currently validation checks whether your read values (strings) conform
to whatever standard you set. It currently *doesn't* allow the
validation process to alter the stored values.

``validate.test`` returns a single True/False - pass/fail for each test.

I could also write a transforming test that returns a tuple of : (pass,
newvalue)

This would allow the validation process to also transform values for you.
It means two things :

    a) You must be allowed to set non string values on a ConfigObj
    b) You need a stringify option to transform all non-string values
when writing out

> 
> 
>>In ConfigObj 3 type checking was optional. There was also a
>>``stringify`` option. If set this automatically converts all values to
>>strings when writing. (So you can just assign integers etc without
>>having to do conversion yourself !).
>>
>>My preference is to do type checking and no longer provide the stringify
>>option.
> 
> 
> The type checking should be done when reading too: the data must conform,
> or be convertible, to the defined type, otherwise the config file is
> illegal. (Mmh, this all seems rather obvious; maybe you're talking about
> something else entirely?)
> 

The sort of type checking I'm talking about is when your program has a
bug and you accidentally set a value to be ``None`` or to be a class
instance by mistake. When you come to write out the file - we crash
because the value isn't a string.

If we *automatically* stringify non-string values, you won't see the
problem till much later.

If we allow the program to crash in the ``write`` method (as currently)
- this may actually be called from a totally different part of your
application that set the value - hard to track.

If we do type checking when you set the value - the code that sets the
value is the code that raises the error - easier to trace.

My view is that a ConfigObj represents the *config file*, and so it
makes sense to keep values as strings. Validation allows you to check
that your string does represent an integer, or represent a bool, or
whatever - but you do the conversion yourself.

You want (I think ?) the ConfigObj to represent the config data - so you
want the validate option to change the type of value from strings to
integers etc.

I'm quite happy to add to validate so it can transform values, add
*optional* type checking, and add a ``stringify`` option that is *on* by
default. This means ConfigObj will work with either model.

There will be a new keyword for the validate method - which will cause
it to *either* use ``val.test`` or ``val.transforming_test``.

> 
> 
>>2) Also in ConfigObj 3 - you could assign a new value to None - and it
>>would initialise the value to an empty section. You now do this with an
>>empty dictionary (which also worked in ConfigObj 3). I'm opting to lose
>>the option of setting a value to None to initialise it.
> 
> 
> I agree. "Explicit is better than implicit."
> 
> On the other hand, if you opt to lose the option, then you won't have the
> option anymore. ;-D
> 

It's hardly more work - and as you say, more explicit.

> 
> 
>>3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
>>Shall I reimplement this ?
> 
> 
> No, you shan't. ;-) Let them pound on the hash key. YAGNI rules. Less is
> better than more. All that jazz. ;-)
> 

YAGNI ?????
Anyway - cool.. I'll lose multiline comments.

> 
> 
>>4) ConfigObj 3 also preserved the comment at the start and end of a
>>file. These would be written out by the ``write`` method.
> 
> 
> That's useful, OTOH.
> 
> 

Ok - I'll keep it in the list.

> 
>>5) Should I attempt to preserve the amount of whitespace around the
>>divider, the type of quoting used, and the amount of whitespace before
>>the comment - and then re-use it in the write method ? I'd rather not....
> 
> 
> Absolutely not. Reformat mercilessly. After a while, the user will
> capitulate, and conform. ;-)
> 

lol - OTOH their nicely lined up config files will become all jagged,
but unavoidable really.

> 
> 
>>6) ConfigParser and ConfigObj allow some recursion in string
>>interpolation - a replaced value can also contain a value to replace.
>>This can be useful. In ConfigParser, you get a `Max recursion depth
>>exceeded' error if you get stuck in a loop. ConfigObj just ignores it.
>>Should it raise the error ? (I think it *should*)
> 
> 
> Yes, you should check it. OTOH, I'd rather drop the "feature". :-) But
> maybe that's taking YAGNI too far.
> 

Hmm... there's an option to turn it off.
It can be quite useful - and is in ConfigParser.

I'll put a maximum recursion depth error in.

> 
> 
>>7) In ConfigObj 3 you could specify the encoding of the file being read
>>and it would decode to unicode *before parsing*. I've removed support
>>for this and added ``decode`` and ``encode`` methods to convert to and
>>from unicode after parsing/before writing. This means you can only use
>>ascii compatible encodings in config files.
> 
> 
> Hooray, KISS rules too! ;-)
> 
> 
> 
>>Adding full unicode support
>>is possible - but requires the addition of two extra keywords (encoding
>>and backup_encoding). Seeing as UTF8 is a full unicode, ASCII compatible
>>encoding - I see no need to change yet.
> 
> 
> So leave it as it is. :-)
> 
> 
> 
>>>Then I'll revert to signing only, not may big secrets to protect anyway. ;-)
> 
> 
>>:-)
>>
>>Not yet.
> 
> 
> In Italy we say "Hope dies last." Do you have anything like that?
> 

Hmm... I can't think of anything directly equivalent. Their maybe
though. Ha - doesn't sound too hopeful though ;-)

(Their is a saying 'Hope springs eternal' which means something very
similar - but is quite old English).

> Better yet, I'm not even signing the email, the same as I (don't) do with
> all the other mailing lists.
> 

Ok

Cool - useful stuff.

Thanks

Fuzzyman
http://www.voidsapce.org.uk/python

Re: [Rest2web-develop] Re: ConfigObj 4 work

[Nicola L. <ni...@te...>]

> What about private attributes ?
> Like the regular expression class attributes in ConfigObj - should I
> prefix these with a single underscore as well ?

Ideally, any instance member that is not to be externally and directly
accessed should be, yes. It takes a fair amount of discipline to do that,
but I think it's beneficial in the long run.


-- 
Nicola Larosa - ni...@te...

Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005

Re: [Rest2web-develop] Re: ConfigObj 4 work

[Nicola L. <ni...@te...>]

> I'll list the questions I have, answer any you can be bothered with :-)

I'll try. :-)


> 1) Part of the point of ConfigObj is that you can create config files
> from a program. Something like :
> 
> 	config = ConfigObj()
> 	config.filename = 'new.ini'
> 	config['member1'] = value1
> 	config['section1'] = {}
> 	config['section1']['member1'] = value1
> 	...
> 
> 	config.write()

I like this, I need this. :-)


> As it is to be written to a file - the only valid values to set are
> strings, lists of strings, or dictionaries (containing strings or lists
> of strings or...). Should we do type checking when the value is set ?
> (not very difficult). This means the error would be raised when you *set
> the value*, rather than when you try to write the file - which may be
> another part of the program.

Can't see the need for this limitation. The datatype may be any: a number,
a flag, a date. Its *representation* is a string: it will be written to
file, and when read back, the data will be converted back to its type, with
validation.

Anything less than this is insufficient.


> In ConfigObj 3 type checking was optional. There was also a
> ``stringify`` option. If set this automatically converts all values to
> strings when writing. (So you can just assign integers etc without
> having to do conversion yourself !).
> 
> My preference is to do type checking and no longer provide the stringify
> option.

The type checking should be done when reading too: the data must conform,
or be convertible, to the defined type, otherwise the config file is
illegal. (Mmh, this all seems rather obvious; maybe you're talking about
something else entirely?)


> 2) Also in ConfigObj 3 - you could assign a new value to None - and it
> would initialise the value to an empty section. You now do this with an
> empty dictionary (which also worked in ConfigObj 3). I'm opting to lose
> the option of setting a value to None to initialise it.

I agree. "Explicit is better than implicit."

On the other hand, if you opt to lose the option, then you won't have the
option anymore. ;-D


> 3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
> Shall I reimplement this ?

No, you shan't. ;-) Let them pound on the hash key. YAGNI rules. Less is
better than more. All that jazz. ;-)


> 4) ConfigObj 3 also preserved the comment at the start and end of a
> file. These would be written out by the ``write`` method.

That's useful, OTOH.


> 5) Should I attempt to preserve the amount of whitespace around the
> divider, the type of quoting used, and the amount of whitespace before
> the comment - and then re-use it in the write method ? I'd rather not....

Absolutely not. Reformat mercilessly. After a while, the user will
capitulate, and conform. ;-)


> 6) ConfigParser and ConfigObj allow some recursion in string
> interpolation - a replaced value can also contain a value to replace.
> This can be useful. In ConfigParser, you get a `Max recursion depth
> exceeded' error if you get stuck in a loop. ConfigObj just ignores it.
> Should it raise the error ? (I think it *should*)

Yes, you should check it. OTOH, I'd rather drop the "feature". :-) But
maybe that's taking YAGNI too far.


> 7) In ConfigObj 3 you could specify the encoding of the file being read
> and it would decode to unicode *before parsing*. I've removed support
> for this and added ``decode`` and ``encode`` methods to convert to and
> from unicode after parsing/before writing. This means you can only use
> ascii compatible encodings in config files.

Hooray, KISS rules too! ;-)


> Adding full unicode support
> is possible - but requires the addition of two extra keywords (encoding
> and backup_encoding). Seeing as UTF8 is a full unicode, ASCII compatible
> encoding - I see no need to change yet.

So leave it as it is. :-)


>> Then I'll revert to signing only, not may big secrets to protect anyway. ;-)

> :-)
> 
> Not yet.

In Italy we say "Hope dies last." Do you have anything like that?

Better yet, I'm not even signing the email, the same as I (don't) do with
all the other mailing lists.

-- 
Nicola Larosa - ni...@te...

Adding things just because you can leads to monstrosities like Common LISP,
PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
(or better yet, by removing restrictions) leads to jewels like Python,
Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005

Re: [Rest2web-develop] Re: ConfigObj 4 work

[Michael F. <mi...@pc...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael Foord wrote:
> Oops - seems like administrator isn't automatically subscribed...
> bizarre. Here's some stuff on ConfigObj 4.
> 
[snip..]
>>>>>There are a couple of
>>>>>implementation decisions still to be made. e.g. should I prefix the
>>>>>private methods with a double underscore ?
>>>
>>>
>>>Single underscore is enough, I'd say.
>>>
> 

What about private attributes ?
Like the regular expression class attributes in ConfigObj - should I
prefix these with a single underscore as well ?

Best Regards,

Fuzzy
http://www.voidspace.org.uk/python


> 
> Ok - I'll start that.
> I've now added ``__all__`` and ``Section`` is not a public object.
> 
> 
> 
[snip..]
>>>--
>>>Nicola Larosa - ni...@te...
>>>
>>>Adding things just because you can leads to monstrosities like Common LISP,
>>>PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
>>>(or better yet, by removing restrictions) leads to jewels like Python,
>>>Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005
>>>
> 

- -------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
Rest2web-develop mailing list
Res...@li...
https://lists.sourceforge.net/lists/listinfo/rest2web-develop




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCzRtU59Olk6wiv6IRAsXnAJ99/8S8wnGwpYdhkOrAqqIvWZQSxACeOzvj
yt0gLClmIF2biGpr7p7dMqw=
=ghfK
-----END PGP SIGNATURE-----

[Rest2web-develop] Re: ConfigObj 4 work

[Michael F. <mi...@pc...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Oops - seems like administrator isn't automatically subscribed...
bizarre. Here's some stuff on ConfigObj 4.

Nicola Larosa wrote:
[snip..]
>
>>>I've added a ``SimpleVal`` object for doing dummy validation (achieves
>>>the same thing as the old 'partial configspec' - just checks members are
>>>present).
>>>
>>>I've added your iter methods into Section - so that it behaves minor an
>>>ordered dicitonary. A couple of slight changes you might want to use in
>>>odict - I use ``repr`` rather than ``str`` in the ``__repr__`` method,
>>>and ``popitem`` is also ordered.
>
>
> Rewrote __repr__, I like it better like this:
>
>     def __repr__(self):
>         items = []
>         for key in self._keys:
>             items.append('%s: %s' % (
>                 repr(key), repr(self[key])))
>         return '{%s}' % ', '.join(items)
>

Yeah that's nicer.

>
>
>>>I've also started on the ``writein`` method - which will be a bit tricky
>>>to get right, but I think I know how I'll do it.
>>>
>>>If you get a chance - can you look at the ``TODO`` and ``ISSUES``
>>>section, and any questions in the source !
>
>
> That's next in line. :-) Let's switch all dev discussion to the
> rest2web-dev ML, I'm subscribed now.
>

Done :-)

I'll list the questions I have, answer any you can be bothered with :-)


1) Part of the point of ConfigObj is that you can create config files
from a program. Something like :

	config = ConfigObj()
	config.filename = 'new.ini'
	config['member1'] = value1
	config['section1'] = {}
	config['section1']['member1'] = value1
	...

	config.write()

As it is to be written to a file - the only valid values to set are
strings, lists of strings, or dictionaries (containing strings or lists
of strings or...). Should we do type checking when the value is set ?
(not very difficult). This means the error would be raised when you *set
the value*, rather than when you try to write the file - which may be
another part of the program.

In ConfigObj 3 type checking was optional. There was also a
``stringify`` option. If set this automatically converts all values to
strings when writing. (So you can just assign integers etc without
having to do conversion yourself !).

My preference is to do type checking and no longer provide the stringify
option.

2) Also in ConfigObj 3 - you could assign a new value to None - and it
would initialise the value to an empty section. You now do this with an
empty dictionary (which also worked in ConfigObj 3). I'm opting to lose
the option of setting a value to None to initialise it.

3) ConfigObj 3 allowed multiline comments of the \*.....*\ variety.
Shall I reimplement this ?

4) ConfigObj 3 also preserved the comment at the start and end of a
file. These would be written out by the ``write`` method.

5) Should I attempt to preserve the amount of whitespace around the
divider, the type of quoting used, and the amount of whitespace before
the comment - and then re-use it in the write method ? I'd rather not....

6) ConfigParser and ConfigObj allow some recursion in string
interpolation - a replaced value can also contain a value to replace.
This can be useful. In ConfigParser, you get a `Max recursion depth
exceeded' error if you get stuck in a loop. ConfigObj just ignores it.
Should it raise the error ? (I think it *should*)

7) In ConfigObj 3 you could specify the encoding of the file being read
and it would decode to unicode *before parsing*. I've removed support
for this and added ``decode`` and ``encode`` methods to convert to and
from unicode after parsing/before writing. This means you can only use
ascii compatible encodings in config files. Adding full unicode support
is possible - but requires the addition of two extra keywords (encoding
and backup_encoding). Seeing as UTF8 is a full unicode, ASCII compatible
encoding - I see no need to change yet.

That will do for the moment.....

>
>
>>>There are a couple of
>>>implementation decisions still to be made. e.g. should I prefix the
>>>private methods with a double underscore ?
>
>
> Single underscore is enough, I'd say.
>

Ok - I'll start that.
I've now added ``__all__`` and ``Section`` is not a public object.


>
>
>>>Once I've written ``writein`` - I'll preserve comments. I think I'll do
>>>this using a comments attribute for each section. I'm going to start by
>>>just preserving inline comments - you can preserve comments *above* a
>>>member by using the ``writein`` method ! Once these two features are
>>>done then it will basically be ready. Just the documentation !
>
>
> Are the docs for ConfigObj3, if any, still applicable, at least partially?
>

Probably large chunks are still applicable, but it *all* needs
rewriting. The ``validate.Validator`` class is poorly documented (and I
*didn't* write the example functions !).


>
>
>>>By the way - sending encrypted mail makes it difficult (not impossible)
>>>for me to receive mail at home (I use the PDA). When I leave on
>>>Saturday, it will make it impossible !
>
>
> Then I'll revert to signing only, not may big secrets to protect anyway. ;-)
>
:-)

Not yet.

Regards,

Fuzzy
http://www.voidspace.org.uk/python


>
> --
> Nicola Larosa - ni...@te...
>
> Adding things just because you can leads to monstrosities like Common LISP,
> PL/I, Algol 68 and Perl 6. Adding features only when they add functionality
> (or better yet, by removing restrictions) leads to jewels like Python,
> Scheme and Eiffel. -- Mike Meyer, comp.lang.python, April 2005
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCzRem59Olk6wiv6IRAlWsAKDEaFrzneIZNfnjJGhViVVh9p+HHACghhof
chIKs5sIghLZ0psmN1lLmDA=
=///K
-----END PGP SIGNATURE-----