rest2web-develop

[Rest2web-develop] rest2web and ConfigObj

[Michael F. <mi...@pc...>]

Hello All,

Updates to rest2web. I'm working towards a new release.

    I've removed some image files
    Also cleaned up the gallery files
    More work on the gallery/plugin system
    py2exe-setup.py now works (builds rest2web as a windoze executable)
    make_dist.py now works - builds a distribution from the SVN
repository files

The gallery is nearly ready. I also have some thorny questions about
encodings and the plugin system to resolve.

Issues to resolve before next release :

    finish gallery todo list
    finish plugin system
    finish gallery and plugins docs
    implement 'file' keyword
    make 'section' an ordered dictionary
    add keyword to specify ordering of pages in section

I've also done a bugfix in ConfigObj - I hope this doesn't cross over
with anything you've done Nicola. It was only one line I changed (line
805) :

    if os.path.isfile:

became :

    if os.path.isfile(self.filename):

Are you alright with the changes to validate/ConfigObj Nicola ? If not,
I can do them. I'd like to get that released as well. The docs need
updating - but it depends on the implementation details of the changes
we discussed.........

Best Regards,


Fuzzyman
http://www.voidspace.org.uk/python

      

Re: [Rest2web-develop] rest2web and ConfigObj

[Nicola L. <ni...@te...>]

> I've also done a bugfix in ConfigObj - I hope this doesn't cross over
> with anything you've done Nicola. It was only one line I changed (line
> 805) :
> 
>     if os.path.isfile:
> 
> became :
> 
>     if os.path.isfile(self.filename):
> 
> Are you alright with the changes to validate/ConfigObj Nicola ? If not,
> I can do them. I'd like to get that released as well. The docs need
> updating - but it depends on the implementation details of the changes
> we discussed.........

Sorry for disappearing during the last few days, Real Life Intruding Again
(tm).

Yes, the changes to ConfigObj we discussed are fine with me, and it would
be great if you would be able to make them.

In the meanwhile I'll try to write doctests for odict.py .

-- 
Nicola Larosa - ni...@te...

Abstraction is layering ignorance on top of reality.
 -- rpg, cited by Richard P. Gabriel, November 2002

Re: [Rest2web-develop] rest2web and ConfigObj

[Michael F. <mi...@pc...>]

Nicola Larosa wrote:
> [Is it intended, this not going to the list?]
> 

No - a mistake. Sorry

> 
> 
>>>I've implemented (and checked in) defaults as discussed. I've done one
>>>test - more appreciated ;-)
>>>
>>>It uses the 'default' keyword in checks, and adds a new attribute
>>>'defaults' on sections.
>>>
>>>If a keyword is missing but a default is supplied, then the default is
>>>used. (The default *is* still passed to the check so that type
>>>conversion can be done).
>>>
>>>If a keyword is missing and no default is supplied then a
>>>ValidateMissingValue error is raised. (Check fails).
>>>
>>>The write method doesn't write out any values that come from defaults.
>>>If a whole section is missing the empty marker is still written out.
>>>
>>>Setting a value that was previously obtained from a default, removes it
>>>from the list of defaults - so it will be written out.
> 
> 
> Wonderful, I'll check this ASAP.
> 
> 
> 
>>>I *haven't* implemented a way of repeating whole sections in the configspec.
>>>
>>>If I *was* to do that I would probably do it by allowing you to name
>>>sections of a ConfigObj with the magic value ``'__name__' = name``
>>>
>>>You could then assign a section with (?) ``'__copy__' = name`` and it
>>>will copy the section (including recursive subsections ?).
>>>
>>>I don't like magic values though...... It's also a nuisance as you'd
>>>have to parse the whole thing and then go back through sorting out the
>>>magic values :-) What to do with the ``write`` method would also be
>>>interesting. (I'd probably *not* write out sections with '__copy__' as a
>>>value, and delete '__copy__' if you assign to a value.. not easy though
>>>in case someone sets a value in a *parent* section - have to walk up the
>>>parents deleting the '__copy__' value as well....).
> 
> 
> Yuck, too complicated. I'll first try your suggestion to manually assign
> configspecs to sections, even if it's already been used in another one.
> 

No problem - this *will* work. The validate method just uses the
'configspec' attribute for checks - so you can assign to it easily. You
still have to do it manually of course... :-)

> The only thing really missing now is having "lists of sections", for the
> cases when their number cannot be determined in advance. Maybe some easy
> workaround can be found.

I don't understand what you want. If you give me an example I'll think
about it. It sounds like you need a way of referencing (naming)
individual sections. Is this just for the programmer - or for the system
administrator (the user).

If you just want to obtain a list of all the sections in a config file -
then that would be quite easy to obtain....

Regards,


Fuzzy
http://www.voidspace.org.uk/python

> 
> --
> Nicola Larosa - ni...@te...
> 
> When I thought life had some purpose - Then I thought I had some choice
>   (I was running blind)
> And I made some value judgments - In a self-important voice
>   (I was outa line)
> But then absurdity came over me - And I longed to lose control
>   (into no mind)
> Oh all I ever wanted - Was just to come in from the cold
>  -- Joni Mitchell, Come in from the cold, Night Ride Home, 1991
> 

Re: [Rest2web-develop] rest2web and ConfigObj

[Nicola L. <ni...@te...>]

>> The only thing really missing now is having "lists of sections", for the
>> cases when their number cannot be determined in advance. Maybe some easy
>> workaround can be found.

> I don't understand what you want. If you give me an example I'll think
> about it. It sounds like you need a way of referencing (naming)
> individual sections. Is this just for the programmer - or for the system
> administrator (the user).

My cousin has several cats and several dogs, and wants to "configure" them.
(I know, peculiar cousin.) I don't exactly know how many they are, and
moreover, their number could change with time.

All the cats have the same params among them, the same with dogs, but cats'
params are slightly different from dogs' params.

I write this configspec:

[ cats ]
    [[ cat ]]
        color = string
        whiskers = int
[ dogs ]
    [[ dog ]]
        color = string
        trained = boolean

Then I write this config file:

[ cats ]
    [[ felix ]]
        color = black
        whiskers = 5
    [[ miao ]]
        color = white
        whiskers = 7
[ dogs ]
    [[ fido ]]
        color = brown
        trained = yes
    [[ lassie ]]
        color = grey
        trained = no
    [[ rintintin ]]
        color = pink
        trained = no

How do I specify that the "cat" configspec has to be used for the "felix"
and "miao" sections, while the "dog" configspec has to be used for the
"fido", "lassie" and "rintintin" sections?

Or better, how do I specify that, when reading the config file, all
subsections of the "cats" section have to be validated against the "cat"
configspec, and that all subsections of the "dogs" section have to be
validated against the "dog" configspec?

-- 
Nicola Larosa - ni...@te...

When I thought life had some purpose - Then I thought I had some choice
  (I was running blind)
And I made some value judgments - In a self-important voice
  (I was outa line)
But then absurdity came over me - And I longed to lose control
  (into no mind)
Oh all I ever wanted - Was just to come in from the cold
 -- Joni Mitchell, Come in from the cold, Night Ride Home, 1991

Re: [Rest2web-develop] sections and configspec

[Michael F. <mi...@pc...>]

Nicola Larosa wrote:
>>>The only thing really missing now is having "lists of sections", for the
>>>cases when their number cannot be determined in advance. Maybe some easy
>>>workaround can be found.
> 
> 
>>I don't understand what you want. If you give me an example I'll think
>>about it. It sounds like you need a way of referencing (naming)
>>individual sections. Is this just for the programmer - or for the system
>>administrator (the user).
> 
> 
> My cousin has several cats and several dogs, and wants to "configure" them.
> (I know, peculiar cousin.) I don't exactly know how many they are, and
> moreover, their number could change with time.
> 
> All the cats have the same params among them, the same with dogs, but cats'
> params are slightly different from dogs' params.
> 
> I write this configspec:
> 
> [ cats ]
>     [[ cat ]]
>         color = string
>         whiskers = int
> [ dogs ]
>     [[ dog ]]
>         color = string
>         trained = boolean
> 
> Then I write this config file:
> 
> [ cats ]
>     [[ felix ]]
>         color = black
>         whiskers = 5
>     [[ miao ]]
>         color = white
>         whiskers = 7
> [ dogs ]
>     [[ fido ]]
>         color = brown
>         trained = yes
>     [[ lassie ]]
>         color = grey
>         trained = no
>     [[ rintintin ]]
>         color = pink
>         trained = no
> 
> How do I specify that the "cat" configspec has to be used for the "felix"
> and "miao" sections, while the "dog" configspec has to be used for the
> "fido", "lassie" and "rintintin" sections?
> 
> Or better, how do I specify that, when reading the config file, all
> subsections of the "cats" section have to be validated against the "cat"
> configspec, and that all subsections of the "dogs" section have to be
> validated against the "dog" configspec?
> 

You want named types of configspec sections and to be able to apply
those types in your config file.

In this particular example you are saying that all 'dogs' (sub-sections
in the dogs section) should use the [[dog]] configspec and 'cats'
(sub-sections in the cats section) should use the 'cat' configspec.

Two possible implementations spring to mind - named sections, or
repeated sections. Repeated sections would meet your specific example
here - but aren't a generic solution. I'll suggest an alternative
implementation of named sections, that is less complicated than yesterdays.

Named Sections
==============

The trouble is that with a hierarchical structure (rather than a flat
one) there is no unique identifier per section (sections have non unique
names).

The alternatives are either to allow sections to be named - a magic
value (either '__name__' or '__type__') or to use something like a
dotted syntax to reference sections 'section.sub-section.sub-sub-section'.

I think my previous explanation was overly complex. I favour this
solution - using a '__type__' magic value in sections. These can be
treated specially when parsing configspecs and are not treated specially
by the ``write`` method.

I write this configspec:

[ cats ]
    [[ cat ]]
	__type__ = cat
        color = string
        whiskers = int
[ dogs ]
    [[ dog ]]
	__type__ = dog
        color = string
        trained = boolean

Then I write this config file:

[ cats ]
    [[ felix ]]
	__type__ = cat
        color = black
        whiskers = 5
    [[ miao ]]
	__type__ = cat
        color = white
        whiskers = 7
[ dogs ]
    [[ fido ]]
	__type__ = dog
        color = brown
        trained = yes
    [[ lassie ]]
	__type__ = dog
        color = grey
        trained = no
    [[ rintintin ]]
	__type__ = dog
        color = pink
        trained = no

When parsing a configspec any __type__ values are not treated as checks,
but a *separate* dictionary of types is kept.

If the validate method encounters a section with a __type__ value it
looks for the configspec for this section in the type dictionary.

There are a couple of implementation questions if you like this
solution.....

Repeated sections could solve this particular example without the use of
 a magic value in the config file.

You could do something like

[ cats ]
    [[ __repeated__ ]]
        color = string
        whiskers = int
[ dogs ]
    [[ __repeated__ ]]
	__type__ = dog
        color = string
        trained = boolean

Effectively saying that any subsection of 'cats' or 'dogs' has to meet
the __repeated__ specification. This isn't useful if you want to be able
to have a 'dog' somewhere else of course.....

Using the dotted syntax allows you to avoid magic values in your
configspec (which is less useful).

You could do :

[ dogs ]
    [[ fido ]]
	__type__ = dogs.dog
        color = brown
        trained = yes
    [[ lassie ]]
	__type__ = dogs.dog
        color = grey
        trained = no
    [[ rintintin ]]
	__type__ = dogs.dog
        color = pink
        trained = no

I don't like this personally.....

Regards,

Fuzzy
http://www.voidspace.org.uk/python

Re: [Rest2web-develop] sections and configspec

[Nicola L. <ni...@te...>]

>> Or better, how do I specify that, when reading the config file, all
>> subsections of the "cats" section have to be validated against the "cat"
>> configspec, and that all subsections of the "dogs" section have to be
>> validated against the "dog" configspec?

> You want named types of configspec sections and to be able to apply
> those types in your config file.

Not necessarily "named". :-)


> In this particular example you are saying that all 'dogs' (sub-sections
> in the dogs section) should use the [[dog]] configspec and 'cats'
> (sub-sections in the cats section) should use the 'cat' configspec.

Yes.


> Two possible implementations spring to mind - named sections, or
> repeated sections. Repeated sections would meet your specific example
> here - but aren't a generic solution.

I don't care. I don't know if YAGNI, but surely INNIN (I'm Not Needing It
Now), where "it" is "named sections". Repeated sections are enough.


> I'll suggest an alternative implementation of named sections, that is
> less complicated than yesterdays.

But probably still too complicated. ;-)


> Named Sections
> ==============
> 
> The trouble is that with a hierarchical structure (rather than a flat
> one) there is no unique identifier per section (sections have non unique
> names).
> 
> The alternatives are either to allow sections to be named - a magic
> value (either '__name__' or '__type__') or to use something like a
> dotted syntax to reference sections 'section.sub-section.sub-sub-section'.

Dotted syntax would yield unique ids, so magic values are not needed, and
best avoided, since they're ugly. But even it is not needed anyway. :-)


> I think my previous explanation was overly complex. I favour this
> solution - using a '__type__' magic value in sections.

I don't, it's ugly.


> [ cats ]
>     [[ felix ]]
> 	__type__ = cat
>         color = black
>         whiskers = 5
>     [[ miao ]]
> 	__type__ = cat
>         color = white
>         whiskers = 7
> [ dogs ]
>     [[ fido ]]
> 	__type__ = dog
>         color = brown
>         trained = yes
>     [[ lassie ]]
> 	__type__ = dog
>         color = grey
>         trained = no
>     [[ rintintin ]]
> 	__type__ = dog
>         color = pink
>         trained = no

Magic values in the config file even, not just in the configspec? Oh, the
horrors. :-)


> When parsing a configspec any __type__ values are not treated as checks,
> but a *separate* dictionary of types is kept.
> 
> If the validate method encounters a section with a __type__ value it
> looks for the configspec for this section in the type dictionary.
> 
> There are a couple of implementation questions if you like this
> solution.....

I don't. :-) We don't really need to give users the ability to define
custom types. That way lies madness, and XML, and ZConfig. ;-)


> Repeated sections could solve this particular example without the use of
>  a magic value in the config file.
> 
> You could do something like
> 
> [ cats ]
>     [[ __repeated__ ]]
>         color = string
>         whiskers = int
> [ dogs ]
>     [[ __repeated__ ]]
> 	__type__ = dog
>         color = string
>         trained = boolean
> 
> Effectively saying that any subsection of 'cats' or 'dogs' has to meet
> the __repeated__ specification. This isn't useful if you want to be able
> to have a 'dog' somewhere else of course.....

I don't want to. :-)


> Using the dotted syntax allows you to avoid magic values in your
> configspec (which is less useful).

But it is enough, and less ugly.


> You could do :
> 
> [ dogs ]
>     [[ fido ]]
> 	__type__ = dogs.dog
>         color = brown
>         trained = yes
>     [[ lassie ]]
> 	__type__ = dogs.dog
>         color = grey
>         trained = no
>     [[ rintintin ]]
> 	__type__ = dogs.dog
>         color = pink
>         trained = no
> 
> I don't like this personally.....

Me neither. I don't want to pollute the config file this way. Types are not
the users' problem.

BTW, __repeated__ is a bit long, what about __many__?

I also need nested repeated sections:

[ cats ]
   [[ __many__ ]]
       color = string
       whiskers = int
       [[[ kittens ]]]
           [[[[ __many__ ]]]]
               age = string
[ dogs ]
   [[ __many__ ]]
       color = string
       trained = boolean
       [[[ puppies ]]]
          [[[[ __many__ ]]]]
              age = string

allowing one to write:

[ cats ]
    [[ felix ]]
        color = black
        whiskers = 5
        [[[ kittens ]]]
            [[[[ mia ]]]]
                age = 1
            [[[[ amy ]]]]
                age = 3
[ dogs ]
    [[ fido ]]
        color = brown
        trained = yes
        [[[ puppies ]]]
            [[[[ fuffy ]]]]
                age = 2
    [[ lassie ]]
        color = grey
        trained = no


Yes, I like this. All those brackets are a little obtrusive, though. What
if we only keep the closing ones?

[ cats ]
    [ felix ]]
        color = black
        whiskers = 5
        [ kittens ]]]
            [ mia ]]]]
                age = 1
            [ amy ]]]]
                age = 3
[ dogs ]
    [ fido ]]
        color = brown
        trained = yes
        [ puppies ]]]
            [ fuffy ]]]]
                age = 2
    [ lassie ]]
        color = grey
        trained = no

Nicer. They're not balanced anymore, but who cares? They just form tokens
here. Look, it's almost as if an invisible rope pulls and stretches them
rightward. :-)

-- 
Nicola Larosa - ni...@te...

When I thought life had some purpose - Then I thought I had some choice
  (I was running blind)
And I made some value judgments - In a self-important voice
  (I was outa line)
But then absurdity came over me - And I longed to lose control
  (into no mind)
Oh all I ever wanted - Was just to come in from the cold
 -- Joni Mitchell, Come in from the cold, Night Ride Home, 1991

Re: [Rest2web-develop] rest2web and ConfigObj

[Nicola L. <ni...@te...>]

>> I've implemented (and checked in) defaults as discussed. I've done one
>> test - more appreciated ;-)
>> 
>> It uses the 'default' keyword in checks, and adds a new attribute
>> 'defaults' on sections.
>>
>> If a keyword is missing but a default is supplied, then the default is
>> used. (The default *is* still passed to the check so that type
>> conversion can be done).

Here, in validate.py, you were using the "pop" method of dicts. Alas, it
was introduced in Python 2.3, so it broke compatibility with Python 2.2 .

Moreover, there was a naked except: clause in ConfigObj.validate that
silenced the exception (*never* use naked except: clauses).

I fixed both problems, maybe not in the cleanest way, see the code. All
tests pass once again on Python 2.2, 2.3 and 2.4 .

-- 
Nicola Larosa - ni...@te...

When I thought life had some purpose - Then I thought I had some choice
  (I was running blind)
And I made some value judgments - In a self-important voice
  (I was outa line)
But then absurdity came over me - And I longed to lose control
  (into no mind)
Oh all I ever wanted - Was just to come in from the cold
 -- Joni Mitchell, Come in from the cold, Night Ride Home, 1991

Re: [Rest2web-develop] rest2web and ConfigObj

[Michael F. <mi...@pc...>]

Nicola Larosa wrote:
>>>I've implemented (and checked in) defaults as discussed. I've done one
>>>test - more appreciated ;-)
>>>
>>>It uses the 'default' keyword in checks, and adds a new attribute
>>>'defaults' on sections.
>>>
>>>If a keyword is missing but a default is supplied, then the default is
>>>used. (The default *is* still passed to the check so that type
>>>conversion can be done).
> 
> 
> Here, in validate.py, you were using the "pop" method of dicts. Alas, it
> was introduced in Python 2.3, so it broke compatibility with Python 2.2 .
> 
> Moreover, there was a naked except: clause in ConfigObj.validate that
> silenced the exception (*never* use naked except: clauses).
> 
> I fixed both problems, maybe not in the cleanest way, see the code. All
> tests pass once again on Python 2.2, 2.3 and 2.4 .
> 

Brilliant by the way..... Nice one.

Regards,

Fuzzy
http://www.voidspace.org.uk/python