configobj-develop — Support and development of ConfigObj

Re: [Configobj-develop] Continuation for multi-line data

[Fuzzyman <fuz...@vo...>]

Abhinandan Jain wrote:

>OK. Thanks anyway. I do not have control over these non-Python
>configuration files - hence the request.
>  
>
You could easily 'pre-process' the config files before passing them to
ConfigObj.

ConfigObj will accept a list of lines as input. So you could read your
file in and split it into lines, then iterate over it and join any that
end in '\' (not forgetting to remove the escape character).

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

>Abhi
>
>  
>
>>>  
>>>      
>>>
>>Hello Abhi,
>>
>>I'm afraid I don't like '\' as continuation syntax, and won't add it 
>>back to ConfigObj.
>>
>>The correct way to create multiline values is with triple quote strings :
>>
>>key = """A multiline value.
>>Really."""
>>
>>All the best,
>>
>>
>>Fuzzyman
>>http://www.voidspace.org.uk/python/index.shtml
>>
>>    
>>
>
>
>-------------------------------------------------------------------------
>Using Tomcat but need to do more? Need to support web services, security?
>Get stuff done quickly with pre-integrated technology to make your job easier
>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>_______________________________________________
>Configobj-develop mailing list
>Con...@li...
>https://lists.sourceforge.net/lists/listinfo/configobj-develop
>
>  
>

Re: [Configobj-develop] Continuation for multi-line data

[Abhinandan J. <ja...@he...>]

OK. Thanks anyway. I do not have control over these non-Python
configuration files - hence the request.

Abhi

> >
> >   
> Hello Abhi,
> 
> I'm afraid I don't like '\' as continuation syntax, and won't add it 
> back to ConfigObj.
> 
> The correct way to create multiline values is with triple quote strings :
> 
> key = """A multiline value.
> Really."""
> 
> All the best,
> 
> 
> Fuzzyman
> http://www.voidspace.org.uk/python/index.shtml
> 

Re: [Configobj-develop] Continuation for multi-line data

[Michael F. <fuz...@vo...>]

Abhinandan Jain wrote:
> Hi
>
> I see in the documentation that the use of '\' as a continuation
> character for multi-line data is no longer supported in ConfigObj
> 4. There are some third party configuration datafiles for which I have
> found this feature very useful when parsing using ConfigObj.
>
> I'd like to put in a request to add back support for this to ConfigObj 4
> as an optional feature. In my case, the data files are fairly simple
> in structure so it is OK with me if enabling this feature means that
> some other features in ConfigObj will not be available when parsing the
> files. 
>
>   
Hello Abhi,

I'm afraid I don't like '\' as continuation syntax, and won't add it 
back to ConfigObj.

The correct way to create multiline values is with triple quote strings :

key = """A multiline value.
Really."""

All the best,


Fuzzyman
http://www.voidspace.org.uk/python/index.shtml


> Thanks
>
> Abhi
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop
>
>   

[Configobj-develop] Continuation for multi-line data

[Abhinandan J. <ja...@he...>]

Hi

I see in the documentation that the use of '\' as a continuation
character for multi-line data is no longer supported in ConfigObj
4. There are some third party configuration datafiles for which I have
found this feature very useful when parsing using ConfigObj.

I'd like to put in a request to add back support for this to ConfigObj 4
as an optional feature. In my case, the data files are fairly simple
in structure so it is OK with me if enabling this feature means that
some other features in ConfigObj will not be available when parsing the
files. 

Thanks

Abhi

Re: [Configobj-develop] String Substitution

[Michael F. <fuz...@vo...>]

Robin Munn wrote:
> On 6/6/06, Fuzzyman <fuz...@vo...> wrote:
>   
>> Rick Kier wrote:
>>
>>     
>>> Hi Robin et all,
>>>
>>> Monday, June 05, 2006, Robin Munn wrote:
>>> RM> There's one other behavior of the interpolation that was surprising to
>>> RM> me: the fact that interpolated variables are *only* allowed to refer
>>> RM> to values in the [DEFAULT] section. That means that I can't do:
>>>
>>> Agree 100% on this.
>>>
>>> We don't use interpolation due to the current behavior
>>>
>>>
>>>       
>> I don't use string substitution at all, so I am willing to listen to the
>> views of users on this.
>>
>> It could theoretically break behaviour for others.
>>
>> Anyone else got opinions ?
>>
>> Fuzzyman
>> http://www.voidspace.org.uk/python/index.shtml
>>     
>
> After giving it some more thought, I'm starting to believe the best
> interpolation behavior is going to be eager-interpolation, pulling
> values from anywhere in the current section, its DEFAULTs, or its
> parents and their DEFAULTs. So the algorithm looks like this:
>
>   

Robin. The only people who have spoken up are in favour of the new 
interpolation, and for interpolation using the current section as well 
as the default section.

That means (unless a violent protest arises) that your changes will go 
into the next version of ConfigObj.

I'm currently working on an update to Movable Python (and a few support 
modules) . This will probably take a couple of weeks, maybe a bit less. 
After that I will work on ConfigObj again and incorporate your changes. 
I will probably turn to you for assistance and clarification at that point.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> When a string interpolation (in either "%(foo)s" or "$foo" syntax) is
> seen, immediately try to interpret it as follows:
> 1. Look for name "foo" in current section. If found, exit loop.
> 2. Look for name "foo" in current section's DEFAULTs. If found, exit loop.
> 3. Move "current section" reference to parent. If no parent, raise
> NotFoundException.
> 4. Repeat from step 1.
>
>
> Advantages of this approach:
>
> 1. No need for special-case "Stop when a $$ is seen" code.
> Special-case code usually makes me uncomfortable unless there's a darn
> good reason for it, and this one smells like a workaround.
>
> 2. The scoping rules work like I would have expected them to work in
> the first place: specifically-given values override default values,
> and parent namespaces are consulted all the way up to the top.
>
> 3. No need for recursive evaluation, or any maximumum-nesting-depth
> parameters. Since whatever $foo expands to has *already* been
> evaluated (or else the "foo" name wouldn't exist), any interpolations
> in the "foo" value have already been expanded. So we just replace
> "$foo" with the value and we're done.
>
>
> Disadvantages of this approach:
>
> 1. How do we round-trip? E.g., if we have "foo = 3" and "bar = $foo",
> after we load the config file, the value of "bar" is 3, and so when we
> write it out we'll get "foo = 3" and "bar = 3". Solution: if
> interpolation happens, keep the value of the original string around
> for use when we write out the config file. Question: where to keep
> that original string?
>
> 2. Values modified after the config file is parsed won't propagate
> their changes "upwards" to anything that includes them. If after
> parsing the file, we set "foo" to 4, "bar" will still be 3. This may
> not be the desired behavior for some people: they might want to change
> the "path_root" value and have that change propagate throughout their
> config structure. Solution: since we kept the original values around,
> when "foo" changes, we can re-evaluate whatever depends on "foo".
> (This would be a perfect place to use PyCells, once that project is
> complete). Or we could simply re-evaluate the entire config structure
> to be safe.
>
> Note: Some people may *not* want the value of "bar" to be changed when
> "foo" is modified. We should make this an option somewhere -- perhaps
> a "propagate_changes=False" keyword to a function call, or a
> "propagate_changes" attribute in the ConfigObj instance. Haven't
> thought about this one deeply yet.
>
>
> Summary: I propose eager evaluation of interpolated values, with
> lexical scoping for finding values. The original, non-interpolated
> strings should be saved for the purposes of configfile round-tripping
> and, optionally, post-evaluation propagation of changes to dependent
> values (e.g., changing the value of "bar = $foo" when "foo" is
> modified).
>
>   

Re: [Configobj-develop] String Substitution

[Robin M. <rob...@gm...>]

On 6/8/06, Michael Foord <fuz...@vo...> wrote:
> Robin Munn wrote:
> > Here's the patch. This one implements a *proper* recursive,
> > lazy-loading interpolation approach, and no longer suffers from the
> > "stop when a $$ is seen" problem. If anything stops interpolation of
> > one key, the rest of the keys in the string will not have their
> > interpolation stopped. Thus, "$foo + $bar + $baz" will follow "$baz"
> > all the way down the interpolation tree even if "$foo" immediately
> > evaluates to "$$99.95" (which becomes "$99.95" and stops interpolation
> > of "$foo").
> >
>
> Thanks for this Robin.
>
> I'm going to have to take some time to understand this and work out what
> is really going on. :-)
>
> It sounds good though, appreciated.
>
> Fuzzyman
> http://www.voidspace.org.uk/python/index.shtml

Think of the interpolation values as nodes of a directed, acyclic
graph (a tree, basically), and the dependencies between them as the
edges of the graph (or the branches on the tree). E.g., if you have
"bar = $foo" then the node "bar" is the parent of the node "foo". The
patch I submitted basically does a depth-first search on that graph
(or a basic root-to-leaves traversal of the tree, if you prefer to
think of it that way) until it reaches the endnodes of the graph (the
leaves of the tree), which have no interpolated values in them. Then,
using those "final" (no more interpolation needed) values, it climbs
back up the tree until it reaches the root again, and returns with the
final value.

I wish I had a whiteboard; I could explain this so much better with
diagrams. :-)

-- 
Robin Munn
Rob...@gm...
GPG key 0xD6497014

Re: [Configobj-develop] String Substitution

[Robin M. <rob...@gm...>]

Here's the patch to change the section search behavior. Instead of
looking only at DEFAULT sections, it will look at the current section,
then its DEFAULT, then move up to the parent section and repeat the
process.

This is a patch against a version of configobj.py that has the 3rd
template interpolation patch (the one in my previous email to this
thread) already applied.

I've also attached the sample .ini file I used for testing, called
"deepsearch.ini", so you can verify for yourself that the code works.

Note that a couple of the documentation changes in this patch
(particularly the ones where InterpolationDepthError was changed to
InterpolationLoopError) should have gone into
3rd-template-interpolation.patch instead. This won't be a problem if
both patches go into the SVN tree, but if you decide to reject the
change-search-behavior.patch and keep the old "only search a few
DEFAULT sections" behavior, then you'll need to cherrypick the
relevant documentation patches out of this patch. Shouldn't be
difficult, since this patch isn't all that large.

I hope this makes it in; I'd really prefer the behavior that this
patch implements.

-- 
Robin Munn
Rob...@gm...
GPG key 0xD6497014

Re: [Configobj-develop] String Substitution

[Michael F. <fuz...@vo...>]

Robin Munn wrote:
> Here's the patch. This one implements a *proper* recursive,
> lazy-loading interpolation approach, and no longer suffers from the
> "stop when a $$ is seen" problem. If anything stops interpolation of
> one key, the rest of the keys in the string will not have their
> interpolation stopped. Thus, "$foo + $bar + $baz" will follow "$baz"
> all the way down the interpolation tree even if "$foo" immediately
> evaluates to "$$99.95" (which becomes "$99.95" and stops interpolation
> of "$foo").
>

Thanks for this Robin.

I'm going to have to take some time to understand this and work out what 
is really going on. :-)

It sounds good though, appreciated.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml
> Infinite-recursion loops will be detected as soon as any value is
> entered for the second time, and an exception will be thrown at that
> point. This has the added advantage of making the MAX_INTERPOL_DEPTH
> value completely unnecessary -- with my patch applied,
> MAX_INTERPOL_DEPTH can still be set, but it has no effect whatsoever.
> The recursion will continue down as deep as necessary as long as there
> are no loops.
>
> I've tested this code against all the scenarios I could come up with,
> and it's worked properly every time.
>
> This version of the patch makes no change to the "which section to
> look at" behavior: it only looks at sections named DEFAULT. Next up: a
> patch that changes this lookup order to the following:
>
> 1) Current section
> 2) Current section's DEFAULT
> 3) current = current.parent; repeat from 1
>
> ------------------------------------------------------------------------
>
> === docs/configobj.txt
> ==================================================================
> --- docs/configobj.txt	(revision 20140)
> +++ docs/configobj.txt	(local)
> @@ -786,13 +786,13 @@
>  * create_empty
>  * file_error
>  
> -interpolate
> -~~~~~~~~~~~
> +interpolation
> +~~~~~~~~~~~~~
>  
>  ConfigObj can perform string interpolation in a *similar* way to
>  ``ConfigParser``. See the interpolation_ section for full details.
>  
> -If ``interpolate`` is set to ``False``, then interpolation is *not* done when
> +If ``interpolation`` is set to ``False``, then interpolation is *not* done when
>  you fetch values.
>  
>  stringify
> @@ -1957,9 +1957,29 @@
>  =============
>  
>  ConfigObj allows string interpolation *similar* to the way ``ConfigParser``
> +or ``string.Template`` work. The value of the ``interpolation`` attribute
> +determines which style of interpolation you want to use. Valid values are
> +"ConfigParser" or "Template" (case-insensitive, so "configparser" and
> +"template" will also work). For backwards compatibility reasons, the value
> +``True`` is also a valid value for the ``interpolation`` attribute, and
> +will select ``ConfigParser``-style interpolation. At some undetermined point
> +in the future, that default *may* change to ``Template``-style interpolation.
>  
> -You specify a value to be substituted by including ``%(name)s`` in the value.
> +For ``ConfigParser``-style interpolation, you specify a value to be
> +substituted by including ``%(name)s`` in the value.
>  
> +For ``Template``-style interpolation, you specify a value to be substituted
> +by including ``${name}`` in the value. Alternately, if 'name' is a valid
> +Python identifier (i.e., is composed of nothing but alphanumeric characters,
> +plus the underscore character), then the braces are optional and the value
> +can be written as ``$name``.
> +
> +Note that ``ConfigParser``-style interpolation and ``Template``-style
> +interpolation are mutually exclusive; you cannot have a configuration file
> +that's a mix of one or the other. Pick one and stick to it. ``Template``-style
> +interpolation is simpler to read and write by hand, and is recommended if
> +you don't have a particular reason to use ``ConfigParser``-style.
> +
>  Interpolation checks first the 'DEFAULT' sub-section of the current section to
>  see if ``name`` is the key to a value. ('name' is case sensitive).
>  
> === pythonutils/configobj.py
> ==================================================================
> --- pythonutils/configobj.py	(revision 20140)
> +++ pythonutils/configobj.py	(local)
> @@ -102,6 +102,7 @@
>  __all__ = (
>      '__version__',
>      'DEFAULT_INDENT_TYPE',
> +    'DEFAULT_INTERPOLATION',
>      'NUM_INDENT_SPACES',
>      'MAX_INTERPOL_DEPTH',
>      'ConfigObjError',
> @@ -112,7 +113,7 @@
>      'ConfigObj',
>      'SimpleVal',
>      'InterpolationError',
> -    'InterpolationDepthError',
> +    'InterpolationLoopError',
>      'MissingInterpolationOption',
>      'RepeatSectionError',
>      'UnreprError',
> @@ -121,6 +122,7 @@
>      'flatten_errors',
>  )
>  
> +DEFAULT_INTERPOLATION = 'configparser'
>  DEFAULT_INDENT_TYPE = ' '
>  NUM_INDENT_SPACES = 4
>  MAX_INTERPOL_DEPTH = 10
> @@ -250,13 +252,13 @@
>  class InterpolationError(ConfigObjError):
>      """Base class for the two interpolation errors."""
>  
> -class InterpolationDepthError(InterpolationError):
> +class InterpolationLoopError(InterpolationError):
>      """Maximum interpolation depth exceeded in string interpolation."""
>  
>      def __init__(self, option):
>          InterpolationError.__init__(
>              self,
> -            'max interpolation depth exceeded in value "%s".' % option)
> +            'interpolation loop detected in value "%s".' % option)
>  
>  class RepeatSectionError(ConfigObjError):
>      """
> @@ -276,11 +278,157 @@
>      """An error parsing in unrepr mode."""
>  
>  
> +class InterpolationEngine(object):
> +    """
> +    A helper class to help perform string interpolation.
> +
> +    This class is an abstract base class; its descendants perform
> +    the actual work.
> +    """
> +
> +    # compiled regexp to use in self.interpolate()
> +    _KEYCRE = re.compile(r"%\(([^)]*)\)s")
> +
> +    def __init__(self, section):
> +        # the Section instance that "owns" this engine
> +        self.section = section
> +
> +    def interpolate(self, key, value):
> +        def recursive_interpolate(key, value, section, backtrail):
> +            """The function that does the actual work.
> +
> +            ``value``: the string we're trying to interpolate.
> +            ``section``: the section in which that string was found
> +            ``backtrail``: a dict to keep track of where we've been,
> +            to detect and prevent infinite recursion loops
> +
> +            This is similar to a depth-first-search algorithm.
> +            """
> +            # Have we been here already?
> +            if backtrail.has_key((key, section.name)):
> +                # Yes - infinite loop detected
> +                raise InterpolationLoopError(key)
> +            # Place a marker on our backtrail so we won't come back here again
> +            backtrail[(key, section.name)] = 1
> +
> +            # Now start the actual work
> +            match = self._KEYCRE.search(value)
> +            while match:
> +                # The actual parsing of the match is implementation-dependent,
> +                # so delegate to our helper function
> +                k, v, s = self._parse_match(match)
> +                if k is None:
> +                    # That's the signal that no further interpolation is needed
> +                    replacement = v
> +                else:
> +                    # Further interpolation may be needed to obtain final value
> +                    replacement = recursive_interpolate(k, v, s, backtrail)
> +                # Replace the matched string with its final value
> +                start, end = match.span()
> +                value = ''.join((value[:start], replacement, value[end:]))
> +                new_search_start = start + len(replacement)
> +                # Pick up the next interpolation key, if any, for next time
> +                # through the while loop
> +                match = self._KEYCRE.search(value, new_search_start)
> +
> +            # Now safe to come back here again; remove marker from backtrail
> +            del backtrail[(key, section.name)]
> +
> +            return value
> +
> +        # Back in interpolate(), all we have to do is kick off the recursive
> +        # function with appropriate starting values
> +        value = recursive_interpolate(key, value, self.section, {})
> +        return value
> +
> +    def _fetch(self, key):
> +        """Helper function to fetch values from owning section.
> +
> +        Returns a 2-tuple: the value, and the section where it was found.
> +        """
> +        # switch off interpolation before we try and fetch anything !
> +        save_interp = self.section.main.interpolation
> +        self.section.main.interpolation = False
> +        # try the 'DEFAULT' member of owning section first
> +        current_section = self.section
> +        val = current_section.get('DEFAULT', {}).get(key)
> +        # try the 'DEFAULT' member of owner's parent section next
> +        if val is None:
> +            current_section = self.section.parent
> +            val = current_section.get('DEFAULT', {}).get(key)
> +        # last, try the 'DEFAULT' member of the main section
> +        if val is None:
> +            current_section = self.section.main
> +            val = current_section.get('DEFAULT', {}).get(key)
> +        # restore interpolation to previous value before returning
> +        self.section.main.interpolation = save_interp
> +        if val is None:
> +            raise MissingInterpolationOption(key)
> +        return val, current_section
> +
> +    def _parse_match(self, match):
> +        """Implementation-dependent helper function.
> +
> +        Will be passed a match object corresponding to the interpolation
> +        key we just found (e.g., "%(foo)s" or "$foo"). Should look up that
> +        key in the appropriate config file section (using the ``_fetch()``
> +        helper function) and return a 3-tuple: (key, value, section)
> +
> +        ``key`` is the name of the key we're looking for
> +        ``value`` is the value found for that key
> +        ``section`` is a reference to the section where it was found
> +
> +        ``key`` and ``section`` should be None if no further
> +        interpolation should be performed on the resulting value
> +        (e.g., if we interpolated "$$" and returned "$").
> +        """
> +        raise NotImplementedError
> +    
> +
> +class ConfigParserInterpolation(InterpolationEngine):
> +    """Behaves like ConfigParser."""
> +    _KEYCRE = re.compile(r"%\(([^)]*)\)s")
> +
> +    def _parse_match(self, match):
> +        key = match.group(1)
> +        value, section = self._fetch(key)
> +        return key, value, section
> +
> +
> +class TemplateInterpolation(InterpolationEngine):
> +    """Behaves like string.Template."""
> +    _delimiter = '$'
> +    _KEYCRE = re.compile(r"""
> +        \$(?:
> +          (?P<escaped>\$)              |   # Two $ signs
> +          (?P<named>[_a-z][_a-z0-9]*)  |   # $name format
> +          {(?P<braced>[^}]*)}              # ${name} format
> +        )
> +        """, re.IGNORECASE | re.VERBOSE)
> +
> +    def _parse_match(self, match):
> +        # Valid name (in or out of braces): fetch value from section
> +        key = match.group('named') or match.group('braced')
> +        if key is not None:
> +            value, section = self._fetch(key)
> +            return key, value, section
> +        # Escaped delimiter (e.g., $$): return single delimiter
> +        if match.group('escaped') is not None:
> +            # Return None for key and section to indicate it's time to stop
> +            return None, self._delimiter, None
> +        # Anything else: ignore completely, just return it unchanged
> +        return None, match.group(), None
> +
> +interpolation_engines = {
> +    'configparser': ConfigParserInterpolation,
> +    'template': TemplateInterpolation,
> +}
> +
>  class Section(dict):
>      """
>      A dictionary-like object that represents a section in a config file.
>      
> -    It does string interpolation if the 'interpolate' attribute
> +    It does string interpolation if the 'interpolation' attribute
>      of the 'main' object is set to True.
>      
>      Interpolation is tried first from the 'DEFAULT' section of this object,
> @@ -293,8 +441,6 @@
>      Iteration follows the order: scalars, then sections.
>      """
>  
> -    _KEYCRE = re.compile(r"%\(([^)]*)\)s|.")
> -
>      def __init__(self, parent, depth, main, indict=None, name=None):
>          """
>          * parent is the section above
> @@ -335,46 +481,33 @@
>          for entry in indict:
>              self[entry] = indict[entry]
>  
> -    def _interpolate(self, value):
> -        """Nicked from ConfigParser."""
> -        depth = MAX_INTERPOL_DEPTH
> -        # loop through this until it's done
> -        while depth:
> -            depth -= 1
> -            if value.find("%(") != -1:
> -                value = self._KEYCRE.sub(self._interpolation_replace, value)
> +    def _interpolate(self, key, value):
> +        try:
> +            # do we already have an interpolation engine?
> +            engine = self._interpolation_engine
> +        except AttributeError:
> +            # not yet: first time running _interpolate(), so pick the engine
> +            name = self.main.interpolation
> +            if name == True:  # note that "if name:" would be incorrect here
> +                # backwards-compatibility: interpolation=True means use default
> +                name = DEFAULT_INTERPOLATION
> +            name = name.lower()  # so that "Template", "template", etc. all work
> +            klass = interpolation_engines.get(name, None)
> +            if klass is None:
> +                # invalid value for self.main.interpolation
> +                self.main.interpolation = False
> +                return value
>              else:
> -                break
> -        else:
> -            raise InterpolationDepthError(value)
> -        return value
> +                # save reference to engine so we don't have to do this again
> +                engine = self._interpolation_engine = klass(self)
> +        # let the engine do the actual work
> +        return engine.interpolate(key, value)
>  
> -    def _interpolation_replace(self, match):
> -        """ """
> -        s = match.group(1)
> -        if s is None:
> -            return match.group()
> -        else:
> -            # switch off interpolation before we try and fetch anything !
> -            self.main.interpolation = False
> -            # try the 'DEFAULT' member of *this section* first
> -            val = self.get('DEFAULT', {}).get(s)
> -            # try the 'DEFAULT' member of the *parent section* next
> -            if val is None:
> -                val = self.parent.get('DEFAULT', {}).get(s)
> -            # last, try the 'DEFAULT' member of the *main section*
> -            if val is None:
> -                val = self.main.get('DEFAULT', {}).get(s)
> -            self.main.interpolation = True
> -            if val is None:
> -                raise MissingInterpolationOption(s)
> -            return val
> -
>      def __getitem__(self, key):
>          """Fetch the item and do string interpolation."""
>          val = dict.__getitem__(self, key)
>          if self.main.interpolation and isinstance(val, StringTypes):
> -            return self._interpolate(val)
> +            return self._interpolate(key, val)
>          return val
>  
>      def __setitem__(self, key, value, unrepr=False):
> @@ -471,7 +604,7 @@
>              del self.inline_comments[key]
>              self.sections.remove(key)
>          if self.main.interpolation and isinstance(val, StringTypes):
> -            return self._interpolate(val)
> +            return self._interpolate(key, val)
>          return val
>  
>      def popitem(self):
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop
>   

Re: [Configobj-develop] String Substitution

[Robin M. <rob...@gm...>]

Here's the patch. This one implements a *proper* recursive,
lazy-loading interpolation approach, and no longer suffers from the
"stop when a $$ is seen" problem. If anything stops interpolation of
one key, the rest of the keys in the string will not have their
interpolation stopped. Thus, "$foo + $bar + $baz" will follow "$baz"
all the way down the interpolation tree even if "$foo" immediately
evaluates to "$$99.95" (which becomes "$99.95" and stops interpolation
of "$foo").

Infinite-recursion loops will be detected as soon as any value is
entered for the second time, and an exception will be thrown at that
point. This has the added advantage of making the MAX_INTERPOL_DEPTH
value completely unnecessary -- with my patch applied,
MAX_INTERPOL_DEPTH can still be set, but it has no effect whatsoever.
The recursion will continue down as deep as necessary as long as there
are no loops.

I've tested this code against all the scenarios I could come up with,
and it's worked properly every time.

This version of the patch makes no change to the "which section to
look at" behavior: it only looks at sections named DEFAULT. Next up: a
patch that changes this lookup order to the following:

1) Current section
2) Current section's DEFAULT
3) current = current.parent; repeat from 1

-- 
Robin Munn
Rob...@gm...
GPG key 0xD6497014

Re: [Configobj-develop] String Substitution

[Robin M. <rob...@gm...>]

Following up on my own post:

On 6/7/06, Robin Munn <rob...@gm...> wrote:
> Summary: I propose eager evaluation of interpolated values, with
> lexical scoping for finding values. The original, non-interpolated
> strings should be saved for the purposes of configfile round-tripping
> and, optionally, post-evaluation propagation of changes to dependent
> values (e.g., changing the value of "bar = $foo" when "foo" is
> modified).

I started to try to implement the above in code, and ran into enough
problems that it made me realize that this is a bad idea, and that the
author of the above proposal is clearly an idiot. ;-)

There's nothing wrong with lazy-loading. The *proper* way to do it, in
fact, is to lazy-load the values we find, *recursively*, in a classic
depth-first algorithm. (Keeping track of our backtrail, to catch any
infinite loops). Thus:

foo = 123
bar = $foo
baz = $$foo
quux = $bar + $baz

Finding the final value of "quux":
1) Interpolate $bar
1a) Interpolate $foo
1a1) Find value "123"
1b) Final value of bar is "123"
2) Value of quux now "123 + $baz". Interpolate $baz
2a) Interpolate $$foo
2b) Final value of baz is "$foo"
3) Final value of quux is "123 + $foo".

The original (non-interpolated) values are left in the Section dict at
all times, so that every time they're loaded, they will be
re-interpolated anew. This allows for "foo" to be changed, and the
values of "bar" and "quux" to automatically be re-computed next time
they're accessed. (But not "baz", since it doesn't depend on the value
of "foo").

This will be a lot easier to implement in code than my previous
proposal, since it requires fewer changes to what's already there.
I'll make two patches: one with the current which-section-to-look-at
behavior (only looking at sections named DEFAULT), and another with my
preferred look-at-all-parent-sections behavior.

-- 
Robin Munn
Rob...@gm...
GPG key 0xD6497014

[Configobj-develop] ConfigObj Stuff

[<fuz...@vo...>]

{ran_emo} `ConfigObj 4.3.2 <http://www.voidspace.org.uk/python/configobj.html>`_ had over two hundred downloads in the first forty-eight hours. That's nice. {sm;:-p}

Also it looks like **ConfigObj 4.3.1** got turned into a `debian package <http://groups.google.com/group/linux.debian.changes.devel/browse_thread/thread/dbf3448d173485fc/40b01451b0719e44>`_ only hours before I released 4.3.2. {sm;:lol:}

More importantly, there are now eggs up on the `cheeseshop <http://cheeseshop.python.org/pypi/ConfigObj/4.3.2>`_, so `Turbogears <http://www.turbogears.org>`_ users (and anyone else) should be able to install ConfigObj using `setuptools <http://peak.telecommunity.com/DevCenter/setuptools>`_. These eggs also contain `validate <http://www.voidspace.org.uk/python/validate.html>`_.

The current release *should* address the issues in `this Turbogears ticket <http://trac.turbogears.org/turbogears/ticket/863>`_.

There are two proposals to improve `string substitution <http://www.voidspace.org.uk/python/configobj.html#interpolation>`_ [#]_ in ConfigObj.

The first is to allow `PEP 292 <http://www.python.org/dev/peps/pep-0292/>`_ string substitution.

Currently ConfigObj uses `ConfigParser <http://docs.python.org/lib/module-ConfigParser.html>`_ style interpolation :

    | ``foodir = %(something)``
    | ``foodir = %(dir)s/whatever``

**PEP 292** (along with lots of other languages) uses :

    | ``foodir = $something``
    | ``foodir = ${something}/whatever``

This is much better, but for backwards compatibility the ``ConfigParser`` style substitution would need to be the default. The `interpolation option <http://www.voidspace.org.uk/python/configobj.html#interpolate>`_ would then need to have several options :

* ``True`` - ConfigParser style
* ``False`` - off
* ``ConfigParser`` - guess {sm;:-)}
* ``Template`` - PEP 292 style

This is more complicated than it was. {sm;:???:}

Everyone who has commented on this has said that it is a good idea, so this will probably make its way into the next version of ConfigObj. Many thanks to Robin Munn for the suggestion, code and documentation. It does leave an issue of recursive interpolation and ``$$`` which is used in PEP 292 substitution to escape the ``$`` sign. Maybe I should just get rid of recursive interpolation ?

The second change is *slightly* more controversial. Currently ConfigObj does interpolation from the **DEFAULT** section of the current section, parent section or root section.

A couple of users have suggested that using the current section first would be better ? This *could* break backwards incompatibility, but as I don't use string substitution at all it doesn't affect me. Anyone else got any thoughts ?

.. [#] Also called interpolation.

Re: [Configobj-develop] String Substitution

[Robin M. <rob...@gm...>]

On 6/6/06, Fuzzyman <fuz...@vo...> wrote:
> Rick Kier wrote:
>
> >Hi Robin et all,
> >
> >Monday, June 05, 2006, Robin Munn wrote:
> >RM> There's one other behavior of the interpolation that was surprising to
> >RM> me: the fact that interpolated variables are *only* allowed to refer
> >RM> to values in the [DEFAULT] section. That means that I can't do:
> >
> >Agree 100% on this.
> >
> >We don't use interpolation due to the current behavior
> >
> >
> I don't use string substitution at all, so I am willing to listen to the
> views of users on this.
>
> It could theoretically break behaviour for others.
>
> Anyone else got opinions ?
>
> Fuzzyman
> http://www.voidspace.org.uk/python/index.shtml

After giving it some more thought, I'm starting to believe the best
interpolation behavior is going to be eager-interpolation, pulling
values from anywhere in the current section, its DEFAULTs, or its
parents and their DEFAULTs. So the algorithm looks like this:

When a string interpolation (in either "%(foo)s" or "$foo" syntax) is
seen, immediately try to interpret it as follows:
1. Look for name "foo" in current section. If found, exit loop.
2. Look for name "foo" in current section's DEFAULTs. If found, exit loop.
3. Move "current section" reference to parent. If no parent, raise
NotFoundException.
4. Repeat from step 1.


Advantages of this approach:

1. No need for special-case "Stop when a $$ is seen" code.
Special-case code usually makes me uncomfortable unless there's a darn
good reason for it, and this one smells like a workaround.

2. The scoping rules work like I would have expected them to work in
the first place: specifically-given values override default values,
and parent namespaces are consulted all the way up to the top.

3. No need for recursive evaluation, or any maximumum-nesting-depth
parameters. Since whatever $foo expands to has *already* been
evaluated (or else the "foo" name wouldn't exist), any interpolations
in the "foo" value have already been expanded. So we just replace
"$foo" with the value and we're done.


Disadvantages of this approach:

1. How do we round-trip? E.g., if we have "foo = 3" and "bar = $foo",
after we load the config file, the value of "bar" is 3, and so when we
write it out we'll get "foo = 3" and "bar = 3". Solution: if
interpolation happens, keep the value of the original string around
for use when we write out the config file. Question: where to keep
that original string?

2. Values modified after the config file is parsed won't propagate
their changes "upwards" to anything that includes them. If after
parsing the file, we set "foo" to 4, "bar" will still be 3. This may
not be the desired behavior for some people: they might want to change
the "path_root" value and have that change propagate throughout their
config structure. Solution: since we kept the original values around,
when "foo" changes, we can re-evaluate whatever depends on "foo".
(This would be a perfect place to use PyCells, once that project is
complete). Or we could simply re-evaluate the entire config structure
to be safe.

Note: Some people may *not* want the value of "bar" to be changed when
"foo" is modified. We should make this an option somewhere -- perhaps
a "propagate_changes=False" keyword to a function call, or a
"propagate_changes" attribute in the ConfigObj instance. Haven't
thought about this one deeply yet.


Summary: I propose eager evaluation of interpolated values, with
lexical scoping for finding values. The original, non-interpolated
strings should be saved for the purposes of configfile round-tripping
and, optionally, post-evaluation propagation of changes to dependent
values (e.g., changing the value of "bar = $foo" when "foo" is
modified).

-- 
Robin Munn
Rob...@gm...
GPG key 0xD6497014

Re: [Configobj-develop] String Substitution

[Fuzzyman <fuz...@vo...>]

Rick Kier wrote:

>Hi Robin et all,
>
>Monday, June 05, 2006, you wrote:
>
>RM> On 5/23/06, Fuzzyman <fuz...@vo...> wrote:
>  
>
>>>Hmmm... on the other hand I'm not convinced recursive interpolation is
>>>used by anyone. I don't mind it being dropped for the string templating
>>>interpolation - *if* that makes it any easier.
>>>      
>>>
>
>RM> Seeing your blog post (where you mention that PEP-292-style substition
>RM> will be going into the next ConfigObj release) got me thinking about
>RM> recursive interpolation again. Right now the way it works is as
>RM> follows:
>
>RM> first = "one"
>RM> second = "$first plus one"
>RM> third = "$second plus one"
>
>RM> "third" is evaluated, yielding "$second plus one" -- which is then
>RM> further evaluated to yield "one plus one plus one". This is "lazy"
>RM> evaluation -- each string is only evaluated once its value is
>RM> requested.
>
>RM> What if this was changed to use "eager" evaluation instead, where each
>RM> string is evaluated as it goes past?
>
>RM> first = "one"
>RM> second = "$first plus one" -- immediately evaluated to yield "one plus one"
>RM> third = "$second plus one" -- immediately evaluated to yield "one plus
>RM> one plus one" (in one single step, since the value of $second is now
>RM> "one plus one" which needs no further recursive evaluation)
>
>RM> The advantage of lazy loading is that no particular order is enforced
>RM> on values. We could have written it in the order:
>
>RM> third = "$second plus one"
>RM> second = "$first plus one"
>RM> first = "one"
>
>RM> And it would have worked. That ordering would cause an error under
>RM> eager-evaluation, because "$second" is as yet undefined when "third"
>RM> is being created.
>
>
>RM> There's one other behavior of the interpolation that was surprising to
>RM> me: the fact that interpolated variables are *only* allowed to refer
>RM> to values in the [DEFAULT] section. That means that I can't do:
>
>Agree 100% on this.
>
>We don't use interpolation due to the current behavior
>  
>
I don't use string substitution at all, so I am willing to listen to the
views of users on this.

It could theoretically break behaviour for others.

Anyone else got opinions ?

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

>Cheers,
>
>TeamSTARS
>  Dick Gordon and Rick Kier
>  
>RM> [my_program]
>RM> path = /home/rmunn/foo
>RM> configfile = ${path}/bar.txt
>
>RM> Instead, I have to do:
>
>RM> [DEFAULT]
>RM> path = /home/rmunn/foo
>
>RM> [my_program]
>RM> configfile = ${path}/bar.txt
>
>RM> This, IMHO, breaks the Principle of Least Surprise. This may be how
>RM> ConfigParser behaves, but is this behavior correct? I would suggest
>RM> that it isn't; that people writing config files are much more likely
>RM> to naturally write the first form than the second.
>
>RM> This may be two separate issues, though, so perhaps I should break the
>RM> second one out into its own thread.
>
>
>
>
>
>_______________________________________________
>Configobj-develop mailing list
>Con...@li...
>https://lists.sourceforge.net/lists/listinfo/configobj-develop
>
>  
>

Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Rick K. <ri...@mc...>]

Hi Robin et all,

Monday, June 05, 2006, you wrote:

RM> On 5/23/06, Fuzzyman <fuz...@vo...> wrote:
>> Hmmm... on the other hand I'm not convinced recursive interpolation is
>> used by anyone. I don't mind it being dropped for the string templating
>> interpolation - *if* that makes it any easier.

RM> Seeing your blog post (where you mention that PEP-292-style substition
RM> will be going into the next ConfigObj release) got me thinking about
RM> recursive interpolation again. Right now the way it works is as
RM> follows:

RM> first =3D "one"
RM> second =3D "$first plus one"
RM> third =3D "$second plus one"

RM> "third" is evaluated, yielding "$second plus one" -- which is then
RM> further evaluated to yield "one plus one plus one". This is "lazy"
RM> evaluation -- each string is only evaluated once its value is
RM> requested.

RM> What if this was changed to use "eager" evaluation instead, where each
RM> string is evaluated as it goes past?

RM> first =3D "one"
RM> second =3D "$first plus one" -- immediately evaluated to yield "one plu=
s one"
RM> third =3D "$second plus one" -- immediately evaluated to yield "one plus
RM> one plus one" (in one single step, since the value of $second is now
RM> "one plus one" which needs no further recursive evaluation)

RM> The advantage of lazy loading is that no particular order is enforced
RM> on values. We could have written it in the order:

RM> third =3D "$second plus one"
RM> second =3D "$first plus one"
RM> first =3D "one"

RM> And it would have worked. That ordering would cause an error under
RM> eager-evaluation, because "$second" is as yet undefined when "third"
RM> is being created.


RM> There's one other behavior of the interpolation that was surprising to
RM> me: the fact that interpolated variables are *only* allowed to refer
RM> to values in the [DEFAULT] section. That means that I can't do:

Agree 100% on this.

We don't use interpolation due to the current behavior

Cheers,

TeamSTARS
  Dick Gordon and Rick Kier
 =20
RM> [my_program]
RM> path =3D /home/rmunn/foo
RM> configfile =3D ${path}/bar.txt

RM> Instead, I have to do:

RM> [DEFAULT]
RM> path =3D /home/rmunn/foo

RM> [my_program]
RM> configfile =3D ${path}/bar.txt

RM> This, IMHO, breaks the Principle of Least Surprise. This may be how
RM> ConfigParser behaves, but is this behavior correct? I would suggest
RM> that it isn't; that people writing config files are much more likely
RM> to naturally write the first form than the second.

RM> This may be two separate issues, though, so perhaps I should break the
RM> second one out into its own thread.

[Configobj-develop] ConfigObj 4.3.2

[<fuz...@vo...>]

{ran_emo} `ConfigObj 4.3.2 <http://www.voidspace.org.uk/python/configobj.html>`_ has just been released.

You can download it from `configobj-4.3.2.zip <http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=configobj-4.3.2.zip>`_.

This is a bugfix and minor feature enhancement release. There is a bugfix in `unrepr mode <http://www.voidspace.org.uk/python/configobj.html#unrepr-mode>`_, and exception handling has been improved.

The full changelog is :

    Changed error handling, if parsing finds a single error then that error will be re-raised. That error will still have an ``errors`` and a ``config`` attribute. That means the error will be more comprehensible.

    Fixed bug where '\n' terminated files could be truncated.

    Bugfix in ``unrepr`` mode, it couldn't handle '#' in values. (Thanks to Philippe Normand for the report.)

    As a consequence of this fix, ConfigObj doesn't now keep inline comments in ``unrepr`` mode. This is because the parser in the `compiler package <http://docs.python.org/lib/compiler.html>`_ doesn't keep comments. {sm;:-)}

    Error messages are now more useful. They tell you the number of parsing errors and the line number of the first error. (In the case of multiple errors.)

    Line numbers in exceptions now start at 1, not 0.

    Errors in ``unrepr`` mode are now handled the same way as in the normal mode. The errors stored will be an ``UnreprError``.

There is also a proposal to support `PEP 292 <http://www.python.org/dev/peps/pep-0292/>`_ string substitution (which is much better). This will be the target of the next release of **ConfigObj**.

Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Robin M. <rob...@gm...>]

On 5/23/06, Fuzzyman <fuz...@vo...> wrote:
> Hmmm... on the other hand I'm not convinced recursive interpolation is
> used by anyone. I don't mind it being dropped for the string templating
> interpolation - *if* that makes it any easier.

Seeing your blog post (where you mention that PEP-292-style substition
will be going into the next ConfigObj release) got me thinking about
recursive interpolation again. Right now the way it works is as
follows:

first = "one"
second = "$first plus one"
third = "$second plus one"

"third" is evaluated, yielding "$second plus one" -- which is then
further evaluated to yield "one plus one plus one". This is "lazy"
evaluation -- each string is only evaluated once its value is
requested.

What if this was changed to use "eager" evaluation instead, where each
string is evaluated as it goes past?

first = "one"
second = "$first plus one" -- immediately evaluated to yield "one plus one"
third = "$second plus one" -- immediately evaluated to yield "one plus
one plus one" (in one single step, since the value of $second is now
"one plus one" which needs no further recursive evaluation)

The advantage of lazy loading is that no particular order is enforced
on values. We could have written it in the order:

third = "$second plus one"
second = "$first plus one"
first = "one"

And it would have worked. That ordering would cause an error under
eager-evaluation, because "$second" is as yet undefined when "third"
is being created.


There's one other behavior of the interpolation that was surprising to
me: the fact that interpolated variables are *only* allowed to refer
to values in the [DEFAULT] section. That means that I can't do:

[my_program]
path = /home/rmunn/foo
configfile = ${path}/bar.txt

Instead, I have to do:

[DEFAULT]
path = /home/rmunn/foo

[my_program]
configfile = ${path}/bar.txt

This, IMHO, breaks the Principle of Least Surprise. This may be how
ConfigParser behaves, but is this behavior correct? I would suggest
that it isn't; that people writing config files are much more likely
to naturally write the first form than the second.

This may be two separate issues, though, so perhaps I should break the
second one out into its own thread.

-- 
Robin Munn
Rob...@gm...
GPG key 0xD6497014

RE: [Configobj-develop] Spacing around default divider ' = '

[George F. <Geo...@sa...>]

Yikes...my bad,I found the section I need to update on my local copy =
ConfigObj..
Thanks
-george

-----Original Message-----
From: con...@li... =
[mailto:con...@li...] On Behalf Of =
Michael Foord
Sent: Wednesday, May 24, 2006 5:17 PM
To: con...@li...
Subject: Re: [Configobj-develop] Spacing around default divider ' =3D '

George Flaherty wrote:
> Is there a parameter/argument I can override such that there is no =
space before and after the default divider. I thought in previous =
versions there was...but I can't remember.
>
> When I write back my modifications, I see all my properties have a =
space before and after the '  =3D ' ?
>  =20
It was possible in ConfigObj 3. It's not possible in ConfigObj 4, and I =
don't think it is something we will support. We already have too many =
options. :-)

On the other hand, it should be *very* easy to patch...

Fuzzyman
http://www.voidspace.org.uk/python/shareware.shtml

> Thanks
> -george
>
>
>
>
> -------------------------------------------------------
> All the advantages of Linux Managed Hosting--Without the Cost and =
Risk!
> Fully trained technicians. The highest number of Red Hat=20
> certifications in the hosting industry. Fanatical Support. Click to=20
> learn more
> http://sel.as-us.falkag.net/sel?cmd=3Dk&kid=107521&bid$8729&dat=121642
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop
>
>  =20



-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications =
in the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D107521&bid=3D248729&dat=3D=
121642
_______________________________________________
Configobj-develop mailing list
Con...@li...
https://lists.sourceforge.net/lists/listinfo/configobj-develop

Re: [Configobj-develop] Spacing around default divider ' = '

[Michael F. <fuz...@vo...>]

George Flaherty wrote:
> Is there a parameter/argument I can override such that there is no space before and after the default divider. I thought in previous versions there was...but I can't remember.
>
> When I write back my modifications, I see all my properties have a space before and after the '  = ' ?
>   
It was possible in ConfigObj 3. It's not possible in ConfigObj 4, and I 
don't think it is something we will support. We already have too many 
options. :-)

On the other hand, it should be *very* easy to patch...

Fuzzyman
http://www.voidspace.org.uk/python/shareware.shtml

> Thanks
> -george
>
>
>
>
> -------------------------------------------------------
> All the advantages of Linux Managed Hosting--Without the Cost and Risk!
> Fully trained technicians. The highest number of Red Hat certifications in
> the hosting industry. Fanatical Support. Click to learn more
> http://sel.as-us.falkag.net/sel?cmd=k&kid7521&bid$8729&dat1642
> _______________________________________________
> Configobj-develop mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/configobj-develop
>
>   

[Configobj-develop] Spacing around default divider ' = '

[George F. <Geo...@sa...>]

Is there a parameter/argument I can override such that there is no space =
before and after the default divider. I thought in previous versions =
there was...but I can't remember.

When I write back my modifications, I see all my properties have a space =
before and after the '  =3D ' ?

Thanks
-george

Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Fuzzyman <fuz...@vo...>]

Hello Robin,

Thanks for your comments/work etc.

I'll probably push out a 4.3.2 release soon with the minor bugfixes I've
already done and then work on a 4.4.0 release. We can then decide how to
integrate the new templating into that.

The bazaar folks are concerned about ConfigObj performance and aim to
create some patches to address that, hopefully they can be integrated
into the same release.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Robin Munn wrote:

> On 5/23/06, Fuzzyman <fuz...@vo...> wrote:
>
>> Robin Munn wrote:
>>
>> > On 5/23/06, Robin Munn <rob...@gm...> wrote:
>> >
>> >> > Instead of reimplementing parts of string.Template, I'd copy it
>> >> wholesale
>> >> > from the stdlib string module to an additional module, and do a
>> >> conditional
>> >> > import, like this:
>> >> >
>> >> > try:
>> >> >     from string import Template
>> >> > except ImportError:
>> >> >     from configobj.string import Template
>> >> >
>> >> > That way, when ConfigObj drops support for the older Python
>> >> versions, the
>> >> > additional module and the conditional import will be removed, and
>> >> no code
>> >> > duplication will remain.
>> >>
>> >> That was my first approach, but it didn't actually work. The problem
>> >> was that string.Template expects you to pass it a single mapping
>> >> (whether it's a dict or some other dict-like object, it doesn't
>> >> matter), and doesn't have an approach for saying "If this mapping
>> >> fails, try this other mapping". I suppose I could have caught the
>> >> resulting KeyError and called string.Template.substitute() again with
>> >> the second mapping, but I'm not sure it would have worked, and it
>> >> would have looked quite a bit uglier than what I ended up with.
>> >>
>> >> OTOH, you have a very good point about using already tested and
>> >> deployed code. So I'll reimplement the TemplateEngine class using
>> >> string.Template, using the "call substitute() a second and third time
>> >> if you get KeyError" approach, and see if it works out better. I'll
>> >> post my results as a second patch if it works out. (Or even if, IMO,
>> >> it doesn't).
>> >
>> >
>> > Attempting this again, I'm reminded why I abandoned this approach when
>> > I tried it yesterday. It turns out that because of the way
>> > string.Template is written, I can't safely replicate ConfigObj's
>> > current behavior. Here's why:
>> >
>> > 1) If ConfigObj doesn't find the value in any of the sections it
>> > searches (the three DEFAULT subsections), it raises a
>> > MissingInterpolationError. string.Template can do the same thing,
>> > since its substitute() function will raise KeyError if the identifier
>> > given isn't found in the mapping. (E.g., you tell string.Template to
>> > interpolate "$foo" but don't have a key "foo" in the dict you hand
>> > it). To chain the DEFAULT sections, I could just chain three calls to
>> > string.Template.substitute(), only raising MissingInterpolationError
>> > if the last one fails to find any values. So far so good.
>> >
>> > But: 2) ConfigObj also promises recursive interpolation, up to ten
>> > levels deep. And here I run into a problem. Say I want to put the
>> > value "$100" in my configuration file. I write it as "money = $$100",
>> > since string.Template-style interpolation promises that a doubled
>> > delimiter will be turned into a single delimiter in the output and
>> > otherwise left untouched. Hence, "$$100" should become "$100". But
>> > then the recursive interpolation kicks in, and the string "$100" is
>> > sent around for another try -- where it fails, because "100" is not a
>> > valid Python identifier. So I can't use substitute(); I need to use
>> > safe_substitute(), which doesn't raise exceptions when it encounters
>> > an invalid substitution, but simply leaves it alone. Using
>> > safe_substitute() instead of substitute() will allow me to leave the
>> > "$100" string alone. But wait -- if I'm using safe_substitute(), then
>> > I'm also not raising an exception when I try to interpolate "$foo"
>> > without a key "foo" in the values dictionary. Oops.
>> >
>> > I can't satisfy both 1) and 2) above by using the string.Template
>> > class. So I need to write my own implementation, that can throw an
>> > error when it sees "$foo" but will leave "$$100" alone.
>> >
>> > Actually, as I was working through this example, I just realized that
>> > my patch still has a bug in it. It can deal with the template "$$100"
>> > and produce "$100", and then leave that value alone -- but only
>> > because "100" isn't a valid Python identifier and thus doesn't match
>> > the template regexp anymore. But what if you really, really want to
>> > put the word "$name" in one of the values of your config file? Doing
>> > "$$name" won't work, because that interprets to "$name" and then a
>> > MissingInterpolationError will be raised when TemplateEngine can't
>> > find the key "name" in any of the sections it searches.
>> >
>> > What this patch really needs is a way to say "stop the interpolation
>> > recursion now, we're through". The only approach I've come up with so
>> > far is to say that if you encounter a $$ in your input, then stop
>> > interpolating, because you've just turned that $$ into a $. And on the
>> > next cycle you're going to start interpolating that value, when the
>> > user has clearly communicated (by his escaping the delimiter) his
>> > intent for that particular value *not* to be interpolated.
>> >
>> > That simply can't be done using string.Template, not even with
>> > monkeypatching. What you would need is to be able to reach down inside
>> > the string.Template.substitute() function's *helper* function and add
>> > a line of code to say "If you find the 'escaped' group in this match
>> > object, then don't just return the delimiter, but *also* set a flag to
>> > let me know." That can't be done without resorting to bytecode hacks.
>> > (Or perhaps, in Python 2.5, AST manipulation). And that's even worse
>> > than rewriting string.Template myself.
>> >
>> > At the end of the day, there's no getting around it -- string.Template
>> > is not designed for recursive interpolation. If we want to allow
>> > recursive interpolation and get it right, we need to write the code
>> > ourselves. Fortunately, it comes out pretty small.
>> >
>> Hmmm... on the other hand I'm not convinced recursive interpolation is
>> used by anyone. I don't mind it being dropped for the string templating
>> interpolation - *if* that makes it any easier.
>>
>> Fuzzyman
>> http://www.voidspace.org.uk/python/index.shtml
>
>
> Here's a revised version of the template-interpolation patch that
> stops recursing the minute it encounters a doubled-up delimiter. It
> added 14 lines of new code: 6 lines dealing with the "stop_recursion"
> flag, and 8 lines of comments explaining why the flag might get set.
>
> I've got a pretty complete "foo.ini" file that exercises most of the
> features in ConfigObj's interpolation by now. I should turn it into a
> set of sensible test cases (basically, read in a config file and make
> sure the interpolated options get their correct values and don't throw
> exceptions). But I'm not sure my regular workload will stand any more
> inattention today, so I'll have to put it off for now. Therefore, in
> addition to the revised patch, I'm attaching the "foo.ini" file I've
> been using for testing. If someone else wants to turn it into test
> cases before I get around to it, be my guest.
>
> The one thing that I haven't included in my documentation patch, that
> might possibly be surprising behavior to somebody who has an obscure
> corner case, is the fact that recursion stops when it hits a $$. So if
> someone's done something like:
>
> first = foo
> second = $$bar
> third = "$first and $second"
> fourth = "$third and $second and $first"
> fifth = $fourth
>
> When "$fourth" is interpolated, the result will be "$third and $second
> and $first". That will yield "$first and $second and $$bar and foo".
> Which gives "foo and $$bar and $bar and foo" -- and because a "$$"
> just went past, interpolation stops right there.
>
> That might surprise someone, but I'm having a real hard time coming up
> with reasons why anyone would want to do something twisted like this.
> :-) So the "stop when you hit a $$" behavior should be fine for most
> purposes.
>
>------------------------------------------------------------------------
>
>=== docs/configobj.txt
>==================================================================
>--- docs/configobj.txt	(revision 19655)
>+++ docs/configobj.txt	(local)
>@@ -787,13 +787,13 @@
> * create_empty
> * file_error
> 
>-interpolate
>-~~~~~~~~~~~
>+interpolation
>+~~~~~~~~~~~~~
> 
> ConfigObj can perform string interpolation in a *similar* way to
> ``ConfigParser``. See the interpolation_ section for full details.
> 
>-If ``interpolate`` is set to ``False``, then interpolation is *not* done when
>+If ``interpolation`` is set to ``False``, then interpolation is *not* done when
> you fetch values.
> 
> stringify
>@@ -1943,9 +1943,29 @@
> =============
> 
> ConfigObj allows string interpolation *similar* to the way ``ConfigParser``
>+or ``string.Template`` work. The value of the ``interpolation`` attribute
>+determines which style of interpolation you want to use. Valid values are
>+"ConfigParser" or "Template" (case-insensitive, so "configparser" and
>+"template" will also work). For backwards compatibility reasons, the value
>+``True`` is also a valid value for the ``interpolation`` attribute, and
>+will select ``ConfigParser``-style interpolation. At some undetermined point
>+in the future, that default *may* change to ``Template``-style interpolation.
> 
>-You specify a value to be substituted by including ``%(name)s`` in the value.
>+For ``ConfigParser``-style interpolation, you specify a value to be
>+substituted by including ``%(name)s`` in the value.
> 
>+For ``Template``-style interpolation, you specify a value to be substituted
>+by including ``${name}`` in the value. Alternately, if 'name' is a valid
>+Python identifier (i.e., is composed of nothing but alphanumeric characters,
>+plus the underscore character), then the braces are optional and the value
>+can be written as ``$name``.
>+
>+Note that ``ConfigParser``-style interpolation and ``Template``-style
>+interpolation are mutually exclusive; you cannot have a configuration file
>+that's a mix of one or the other. Pick one and stick to it. ``Template``-style
>+interpolation is simpler to read and write by hand, and is recommended if
>+you don't have a particular reason to use ``ConfigParser``-style.
>+
> Interpolation checks first the 'DEFAULT' sub-section of the current section to
> see if ``name`` is the key to a value. ('name' is case sensitive).
> 
>=== pythonutils/configobj.py
>==================================================================
>--- pythonutils/configobj.py	(revision 19655)
>+++ pythonutils/configobj.py	(local)
>@@ -102,6 +102,7 @@
> __all__ = (
>     '__version__',
>     'DEFAULT_INDENT_TYPE',
>+    'DEFAULT_INTERPOLATION',
>     'NUM_INDENT_SPACES',
>     'MAX_INTERPOL_DEPTH',
>     'ConfigObjError',
>@@ -121,6 +122,7 @@
>     'flatten_errors',
> )
> 
>+DEFAULT_INTERPOLATION = 'configparser'
> DEFAULT_INDENT_TYPE = ' '
> NUM_INDENT_SPACES = 4
> MAX_INTERPOL_DEPTH = 10
>@@ -276,11 +278,114 @@
>     """An error parsing in unrepr mode."""
> 
> 
>+class InterpolationEngine(object):
>+    """
>+    A helper class to help perform string interpolation.
>+
>+    This class is an abstract base class; its descendants perform
>+    the actual work.
>+    """
>+
>+    # compiled regexp to use in self.interpolate()
>+    _KEYCRE = re.compile(r"%\(([^)]*)\)s")
>+
>+    def __init__(self, section):
>+        # the Section instance that "owns" this engine
>+        self.section = section
>+        # flag to tell the engine to stop looping through the recursion
>+        # (can't use an exception for this, because some implementations
>+        # such as TemplateEngine need to set this flag and still be able
>+        # to return a value)
>+        self.stop_recursion = False
>+
>+    def interpolate(self, value):
>+        depth = MAX_INTERPOL_DEPTH
>+        # loop through this until it's done
>+        self.stop_recursion = False
>+        while depth:
>+            depth -= 1
>+            if self._KEYCRE.search(value):
>+                value = self._KEYCRE.sub(self._interpolation_replace, value)
>+            else:
>+                break
>+            # self._interpolation_replace might have set this flag
>+            if self.stop_recursion:
>+                break
>+        else:
>+            raise InterpolationDepthError(value)
>+        self.stop_recursion = False
>+        return value
>+
>+    def _fetch(self, name):
>+        """Helper function to fetch values from owning section."""
>+        # switch off interpolation before we try and fetch anything !
>+        save_interp = self.section.main.interpolation
>+        self.section.main.interpolation = False
>+        # try the 'DEFAULT' member of owning section first
>+        val = self.section.get('DEFAULT', {}).get(name)
>+        # try the 'DEFAULT' member of owner's parent section next
>+        if val is None:
>+            val = self.section.parent.get('DEFAULT', {}).get(name)
>+        # last, try the 'DEFAULT' member of the main section
>+        if val is None:
>+            val = self.section.main.get('DEFAULT', {}).get(name)
>+        # restore interpolation to previous value before returning
>+        self.section.main.interpolation = save_interp
>+        if val is None:
>+            raise MissingInterpolationOption(name)
>+        return val
>+
>+    def _interpolation_replace(self, match):
>+        """Implementation-dependent helper function."""
>+        raise NotImplementedError
>+    
>+
>+class ConfigParserInterpolation(InterpolationEngine):
>+    """Behaves like ConfigParser."""
>+    _KEYCRE = re.compile(r"%\(([^)]*)\)s")
>+
>+    def _interpolation_replace(self, match):
>+        s = match.group(1)
>+        return self._fetch(s)
>+
>+
>+class TemplateInterpolation(InterpolationEngine):
>+    """Behaves like string.Template."""
>+    _delimiter = '$'
>+    _KEYCRE = re.compile(r"""
>+        \$(?:
>+          (?P<escaped>\$)              |   # Two $ signs
>+          (?P<named>[_a-z][_a-z0-9]*)  |   # $name format
>+          {(?P<braced>[^}]*)}              # ${name} format
>+        )
>+        """, re.IGNORECASE | re.VERBOSE)
>+
>+    def _interpolation_replace(self, match):
>+        # Valid name (in or out of braces): fetch value from section
>+        s = match.group('named') or match.group('braced')
>+        if s is not None:
>+            return self._fetch(s)
>+        # Escaped delimiter (e.g., $$): return single delimiter
>+        if match.group('escaped') is not None:
>+            # Stop the recursive interpolation now, because the user intended
>+            # this dollar sign to end up in the final result. If we go through
>+            # one more level of recursion, we'll end up interpolating it.
>+            self.stop_recursion = True
>+            return self._delimiter
>+        # Anything else: ignore completely, just return it unchanged
>+        raise AssertionError, "Shouldn't get here -- invalid Template interpolation value"
>+        return match.group()
>+
>+interpolation_engines = {
>+    'configparser': ConfigParserInterpolation,
>+    'template': TemplateInterpolation,
>+}
>+
> class Section(dict):
>     """
>     A dictionary-like object that represents a section in a config file.
>     
>-    It does string interpolation if the 'interpolate' attribute
>+    It does string interpolation if the 'interpolation' attribute
>     of the 'main' object is set to True.
>     
>     Interpolation is tried first from the 'DEFAULT' section of this object,
>@@ -293,8 +398,6 @@
>     Iteration follows the order: scalars, then sections.
>     """
> 
>-    _KEYCRE = re.compile(r"%\(([^)]*)\)s|.")
>-
>     def __init__(self, parent, depth, main, indict=None, name=None):
>         """
>         * parent is the section above
>@@ -336,40 +439,27 @@
>             self[entry] = indict[entry]
> 
>     def _interpolate(self, value):
>-        """Nicked from ConfigParser."""
>-        depth = MAX_INTERPOL_DEPTH
>-        # loop through this until it's done
>-        while depth:
>-            depth -= 1
>-            if value.find("%(") != -1:
>-                value = self._KEYCRE.sub(self._interpolation_replace, value)
>+        try:
>+            # do we already have an interpolation engine?
>+            engine = self._interpolation_engine
>+        except AttributeError:
>+            # not yet: first time running _interpolate(), so pick the engine
>+            name = self.main.interpolation
>+            if name == True:  # note that "if name:" would be incorrect here
>+                # backwards-compatibility: interpolation=True means use default
>+                name = DEFAULT_INTERPOLATION
>+            name = name.lower()  # so that "Template", "template", etc. all work
>+            klass = interpolation_engines.get(name, None)
>+            if klass is None:
>+                # invalid value for self.main.interpolation
>+                self.main.interpolation = False
>+                return value
>             else:
>-                break
>-        else:
>-            raise InterpolationDepthError(value)
>-        return value
>+                # save reference to engine so we don't have to do this again
>+                engine = self._interpolation_engine = klass(self)
>+        # let the engine do the actual work
>+        return engine.interpolate(value)
> 
>-    def _interpolation_replace(self, match):
>-        """ """
>-        s = match.group(1)
>-        if s is None:
>-            return match.group()
>-        else:
>-            # switch off interpolation before we try and fetch anything !
>-            self.main.interpolation = False
>-            # try the 'DEFAULT' member of *this section* first
>-            val = self.get('DEFAULT', {}).get(s)
>-            # try the 'DEFAULT' member of the *parent section* next
>-            if val is None:
>-                val = self.parent.get('DEFAULT', {}).get(s)
>-            # last, try the 'DEFAULT' member of the *main section*
>-            if val is None:
>-                val = self.main.get('DEFAULT', {}).get(s)
>-            self.main.interpolation = True
>-            if val is None:
>-                raise MissingInterpolationOption(s)
>-            return val
>-
>     def __getitem__(self, key):
>         """Fetch the item and do string interpolation."""
>         val = dict.__getitem__(self, key)
>
>
>
>
>
>  
>

Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Robin M. <rob...@gm...>]

T24gNS8yMy8wNiwgRnV6enltYW4gPGZ1enp5bWFuQHZvaWRzcGFjZS5vcmcudWs+IHdyb3RlOgo+
IFJvYmluIE11bm4gd3JvdGU6Cj4KPiA+IE9uIDUvMjMvMDYsIFJvYmluIE11bm4gPHJvYmluLm11
bm5AZ21haWwuY29tPiB3cm90ZToKPiA+Cj4gPj4gPiBJbnN0ZWFkIG9mIHJlaW1wbGVtZW50aW5n
IHBhcnRzIG9mIHN0cmluZy5UZW1wbGF0ZSwgSSdkIGNvcHkgaXQKPiA+PiB3aG9sZXNhbGUKPiA+
PiA+IGZyb20gdGhlIHN0ZGxpYiBzdHJpbmcgbW9kdWxlIHRvIGFuIGFkZGl0aW9uYWwgbW9kdWxl
LCBhbmQgZG8gYQo+ID4+IGNvbmRpdGlvbmFsCj4gPj4gPiBpbXBvcnQsIGxpa2UgdGhpczoKPiA+
PiA+Cj4gPj4gPiB0cnk6Cj4gPj4gPiAgICAgZnJvbSBzdHJpbmcgaW1wb3J0IFRlbXBsYXRlCj4g
Pj4gPiBleGNlcHQgSW1wb3J0RXJyb3I6Cj4gPj4gPiAgICAgZnJvbSBjb25maWdvYmouc3RyaW5n
IGltcG9ydCBUZW1wbGF0ZQo+ID4+ID4KPiA+PiA+IFRoYXQgd2F5LCB3aGVuIENvbmZpZ09iaiBk
cm9wcyBzdXBwb3J0IGZvciB0aGUgb2xkZXIgUHl0aG9uCj4gPj4gdmVyc2lvbnMsIHRoZQo+ID4+
ID4gYWRkaXRpb25hbCBtb2R1bGUgYW5kIHRoZSBjb25kaXRpb25hbCBpbXBvcnQgd2lsbCBiZSBy
ZW1vdmVkLCBhbmQKPiA+PiBubyBjb2RlCj4gPj4gPiBkdXBsaWNhdGlvbiB3aWxsIHJlbWFpbi4K
PiA+Pgo+ID4+IFRoYXQgd2FzIG15IGZpcnN0IGFwcHJvYWNoLCBidXQgaXQgZGlkbid0IGFjdHVh
bGx5IHdvcmsuIFRoZSBwcm9ibGVtCj4gPj4gd2FzIHRoYXQgc3RyaW5nLlRlbXBsYXRlIGV4cGVj
dHMgeW91IHRvIHBhc3MgaXQgYSBzaW5nbGUgbWFwcGluZwo+ID4+ICh3aGV0aGVyIGl0J3MgYSBk
aWN0IG9yIHNvbWUgb3RoZXIgZGljdC1saWtlIG9iamVjdCwgaXQgZG9lc24ndAo+ID4+IG1hdHRl
ciksIGFuZCBkb2Vzbid0IGhhdmUgYW4gYXBwcm9hY2ggZm9yIHNheWluZyAiSWYgdGhpcyBtYXBw
aW5nCj4gPj4gZmFpbHMsIHRyeSB0aGlzIG90aGVyIG1hcHBpbmciLiBJIHN1cHBvc2UgSSBjb3Vs
ZCBoYXZlIGNhdWdodCB0aGUKPiA+PiByZXN1bHRpbmcgS2V5RXJyb3IgYW5kIGNhbGxlZCBzdHJp
bmcuVGVtcGxhdGUuc3Vic3RpdHV0ZSgpIGFnYWluIHdpdGgKPiA+PiB0aGUgc2Vjb25kIG1hcHBp
bmcsIGJ1dCBJJ20gbm90IHN1cmUgaXQgd291bGQgaGF2ZSB3b3JrZWQsIGFuZCBpdAo+ID4+IHdv
dWxkIGhhdmUgbG9va2VkIHF1aXRlIGEgYml0IHVnbGllciB0aGFuIHdoYXQgSSBlbmRlZCB1cCB3
aXRoLgo+ID4+Cj4gPj4gT1RPSCwgeW91IGhhdmUgYSB2ZXJ5IGdvb2QgcG9pbnQgYWJvdXQgdXNp
bmcgYWxyZWFkeSB0ZXN0ZWQgYW5kCj4gPj4gZGVwbG95ZWQgY29kZS4gU28gSSdsbCByZWltcGxl
bWVudCB0aGUgVGVtcGxhdGVFbmdpbmUgY2xhc3MgdXNpbmcKPiA+PiBzdHJpbmcuVGVtcGxhdGUs
IHVzaW5nIHRoZSAiY2FsbCBzdWJzdGl0dXRlKCkgYSBzZWNvbmQgYW5kIHRoaXJkIHRpbWUKPiA+
PiBpZiB5b3UgZ2V0IEtleUVycm9yIiBhcHByb2FjaCwgYW5kIHNlZSBpZiBpdCB3b3JrcyBvdXQg
YmV0dGVyLiBJJ2xsCj4gPj4gcG9zdCBteSByZXN1bHRzIGFzIGEgc2Vjb25kIHBhdGNoIGlmIGl0
IHdvcmtzIG91dC4gKE9yIGV2ZW4gaWYsIElNTywKPiA+PiBpdCBkb2Vzbid0KS4KPiA+Cj4gPgo+
ID4gQXR0ZW1wdGluZyB0aGlzIGFnYWluLCBJJ20gcmVtaW5kZWQgd2h5IEkgYWJhbmRvbmVkIHRo
aXMgYXBwcm9hY2ggd2hlbgo+ID4gSSB0cmllZCBpdCB5ZXN0ZXJkYXkuIEl0IHR1cm5zIG91dCB0
aGF0IGJlY2F1c2Ugb2YgdGhlIHdheQo+ID4gc3RyaW5nLlRlbXBsYXRlIGlzIHdyaXR0ZW4sIEkg
Y2FuJ3Qgc2FmZWx5IHJlcGxpY2F0ZSBDb25maWdPYmoncwo+ID4gY3VycmVudCBiZWhhdmlvci4g
SGVyZSdzIHdoeToKPiA+Cj4gPiAxKSBJZiBDb25maWdPYmogZG9lc24ndCBmaW5kIHRoZSB2YWx1
ZSBpbiBhbnkgb2YgdGhlIHNlY3Rpb25zIGl0Cj4gPiBzZWFyY2hlcyAodGhlIHRocmVlIERFRkFV
TFQgc3Vic2VjdGlvbnMpLCBpdCByYWlzZXMgYQo+ID4gTWlzc2luZ0ludGVycG9sYXRpb25FcnJv
ci4gc3RyaW5nLlRlbXBsYXRlIGNhbiBkbyB0aGUgc2FtZSB0aGluZywKPiA+IHNpbmNlIGl0cyBz
dWJzdGl0dXRlKCkgZnVuY3Rpb24gd2lsbCByYWlzZSBLZXlFcnJvciBpZiB0aGUgaWRlbnRpZmll
cgo+ID4gZ2l2ZW4gaXNuJ3QgZm91bmQgaW4gdGhlIG1hcHBpbmcuIChFLmcuLCB5b3UgdGVsbCBz
dHJpbmcuVGVtcGxhdGUgdG8KPiA+IGludGVycG9sYXRlICIkZm9vIiBidXQgZG9uJ3QgaGF2ZSBh
IGtleSAiZm9vIiBpbiB0aGUgZGljdCB5b3UgaGFuZAo+ID4gaXQpLiBUbyBjaGFpbiB0aGUgREVG
QVVMVCBzZWN0aW9ucywgSSBjb3VsZCBqdXN0IGNoYWluIHRocmVlIGNhbGxzIHRvCj4gPiBzdHJp
bmcuVGVtcGxhdGUuc3Vic3RpdHV0ZSgpLCBvbmx5IHJhaXNpbmcgTWlzc2luZ0ludGVycG9sYXRp
b25FcnJvcgo+ID4gaWYgdGhlIGxhc3Qgb25lIGZhaWxzIHRvIGZpbmQgYW55IHZhbHVlcy4gU28g
ZmFyIHNvIGdvb2QuCj4gPgo+ID4gQnV0OiAyKSBDb25maWdPYmogYWxzbyBwcm9taXNlcyByZWN1
cnNpdmUgaW50ZXJwb2xhdGlvbiwgdXAgdG8gdGVuCj4gPiBsZXZlbHMgZGVlcC4gQW5kIGhlcmUg
SSBydW4gaW50byBhIHByb2JsZW0uIFNheSBJIHdhbnQgdG8gcHV0IHRoZQo+ID4gdmFsdWUgIiQx
MDAiIGluIG15IGNvbmZpZ3VyYXRpb24gZmlsZS4gSSB3cml0ZSBpdCBhcyAibW9uZXkgPSAkJDEw
MCIsCj4gPiBzaW5jZSBzdHJpbmcuVGVtcGxhdGUtc3R5bGUgaW50ZXJwb2xhdGlvbiBwcm9taXNl
cyB0aGF0IGEgZG91YmxlZAo+ID4gZGVsaW1pdGVyIHdpbGwgYmUgdHVybmVkIGludG8gYSBzaW5n
bGUgZGVsaW1pdGVyIGluIHRoZSBvdXRwdXQgYW5kCj4gPiBvdGhlcndpc2UgbGVmdCB1bnRvdWNo
ZWQuIEhlbmNlLCAiJCQxMDAiIHNob3VsZCBiZWNvbWUgIiQxMDAiLiBCdXQKPiA+IHRoZW4gdGhl
IHJlY3Vyc2l2ZSBpbnRlcnBvbGF0aW9uIGtpY2tzIGluLCBhbmQgdGhlIHN0cmluZyAiJDEwMCIg
aXMKPiA+IHNlbnQgYXJvdW5kIGZvciBhbm90aGVyIHRyeSAtLSB3aGVyZSBpdCBmYWlscywgYmVj
YXVzZSAiMTAwIiBpcyBub3QgYQo+ID4gdmFsaWQgUHl0aG9uIGlkZW50aWZpZXIuIFNvIEkgY2Fu
J3QgdXNlIHN1YnN0aXR1dGUoKTsgSSBuZWVkIHRvIHVzZQo+ID4gc2FmZV9zdWJzdGl0dXRlKCks
IHdoaWNoIGRvZXNuJ3QgcmFpc2UgZXhjZXB0aW9ucyB3aGVuIGl0IGVuY291bnRlcnMKPiA+IGFu
IGludmFsaWQgc3Vic3RpdHV0aW9uLCBidXQgc2ltcGx5IGxlYXZlcyBpdCBhbG9uZS4gVXNpbmcK
PiA+IHNhZmVfc3Vic3RpdHV0ZSgpIGluc3RlYWQgb2Ygc3Vic3RpdHV0ZSgpIHdpbGwgYWxsb3cg
bWUgdG8gbGVhdmUgdGhlCj4gPiAiJDEwMCIgc3RyaW5nIGFsb25lLiBCdXQgd2FpdCAtLSBpZiBJ
J20gdXNpbmcgc2FmZV9zdWJzdGl0dXRlKCksIHRoZW4KPiA+IEknbSBhbHNvIG5vdCByYWlzaW5n
IGFuIGV4Y2VwdGlvbiB3aGVuIEkgdHJ5IHRvIGludGVycG9sYXRlICIkZm9vIgo+ID4gd2l0aG91
dCBhIGtleSAiZm9vIiBpbiB0aGUgdmFsdWVzIGRpY3Rpb25hcnkuIE9vcHMuCj4gPgo+ID4gSSBj
YW4ndCBzYXRpc2Z5IGJvdGggMSkgYW5kIDIpIGFib3ZlIGJ5IHVzaW5nIHRoZSBzdHJpbmcuVGVt
cGxhdGUKPiA+IGNsYXNzLiBTbyBJIG5lZWQgdG8gd3JpdGUgbXkgb3duIGltcGxlbWVudGF0aW9u
LCB0aGF0IGNhbiB0aHJvdyBhbgo+ID4gZXJyb3Igd2hlbiBpdCBzZWVzICIkZm9vIiBidXQgd2ls
bCBsZWF2ZSAiJCQxMDAiIGFsb25lLgo+ID4KPiA+IEFjdHVhbGx5LCBhcyBJIHdhcyB3b3JraW5n
IHRocm91Z2ggdGhpcyBleGFtcGxlLCBJIGp1c3QgcmVhbGl6ZWQgdGhhdAo+ID4gbXkgcGF0Y2gg
c3RpbGwgaGFzIGEgYnVnIGluIGl0LiBJdCBjYW4gZGVhbCB3aXRoIHRoZSB0ZW1wbGF0ZSAiJCQx
MDAiCj4gPiBhbmQgcHJvZHVjZSAiJDEwMCIsIGFuZCB0aGVuIGxlYXZlIHRoYXQgdmFsdWUgYWxv
bmUgLS0gYnV0IG9ubHkKPiA+IGJlY2F1c2UgIjEwMCIgaXNuJ3QgYSB2YWxpZCBQeXRob24gaWRl
bnRpZmllciBhbmQgdGh1cyBkb2Vzbid0IG1hdGNoCj4gPiB0aGUgdGVtcGxhdGUgcmVnZXhwIGFu
eW1vcmUuIEJ1dCB3aGF0IGlmIHlvdSByZWFsbHksIHJlYWxseSB3YW50IHRvCj4gPiBwdXQgdGhl
IHdvcmQgIiRuYW1lIiBpbiBvbmUgb2YgdGhlIHZhbHVlcyBvZiB5b3VyIGNvbmZpZyBmaWxlPyBE
b2luZwo+ID4gIiQkbmFtZSIgd29uJ3Qgd29yaywgYmVjYXVzZSB0aGF0IGludGVycHJldHMgdG8g
IiRuYW1lIiBhbmQgdGhlbiBhCj4gPiBNaXNzaW5nSW50ZXJwb2xhdGlvbkVycm9yIHdpbGwgYmUg
cmFpc2VkIHdoZW4gVGVtcGxhdGVFbmdpbmUgY2FuJ3QKPiA+IGZpbmQgdGhlIGtleSAibmFtZSIg
aW4gYW55IG9mIHRoZSBzZWN0aW9ucyBpdCBzZWFyY2hlcy4KPiA+Cj4gPiBXaGF0IHRoaXMgcGF0
Y2ggcmVhbGx5IG5lZWRzIGlzIGEgd2F5IHRvIHNheSAic3RvcCB0aGUgaW50ZXJwb2xhdGlvbgo+
ID4gcmVjdXJzaW9uIG5vdywgd2UncmUgdGhyb3VnaCIuIFRoZSBvbmx5IGFwcHJvYWNoIEkndmUg
Y29tZSB1cCB3aXRoIHNvCj4gPiBmYXIgaXMgdG8gc2F5IHRoYXQgaWYgeW91IGVuY291bnRlciBh
ICQkIGluIHlvdXIgaW5wdXQsIHRoZW4gc3RvcAo+ID4gaW50ZXJwb2xhdGluZywgYmVjYXVzZSB5
b3UndmUganVzdCB0dXJuZWQgdGhhdCAkJCBpbnRvIGEgJC4gQW5kIG9uIHRoZQo+ID4gbmV4dCBj
eWNsZSB5b3UncmUgZ29pbmcgdG8gc3RhcnQgaW50ZXJwb2xhdGluZyB0aGF0IHZhbHVlLCB3aGVu
IHRoZQo+ID4gdXNlciBoYXMgY2xlYXJseSBjb21tdW5pY2F0ZWQgKGJ5IGhpcyBlc2NhcGluZyB0
aGUgZGVsaW1pdGVyKSBoaXMKPiA+IGludGVudCBmb3IgdGhhdCBwYXJ0aWN1bGFyIHZhbHVlICpu
b3QqIHRvIGJlIGludGVycG9sYXRlZC4KPiA+Cj4gPiBUaGF0IHNpbXBseSBjYW4ndCBiZSBkb25l
IHVzaW5nIHN0cmluZy5UZW1wbGF0ZSwgbm90IGV2ZW4gd2l0aAo+ID4gbW9ua2V5cGF0Y2hpbmcu
IFdoYXQgeW91IHdvdWxkIG5lZWQgaXMgdG8gYmUgYWJsZSB0byByZWFjaCBkb3duIGluc2lkZQo+
ID4gdGhlIHN0cmluZy5UZW1wbGF0ZS5zdWJzdGl0dXRlKCkgZnVuY3Rpb24ncyAqaGVscGVyKiBm
dW5jdGlvbiBhbmQgYWRkCj4gPiBhIGxpbmUgb2YgY29kZSB0byBzYXkgIklmIHlvdSBmaW5kIHRo
ZSAnZXNjYXBlZCcgZ3JvdXAgaW4gdGhpcyBtYXRjaAo+ID4gb2JqZWN0LCB0aGVuIGRvbid0IGp1
c3QgcmV0dXJuIHRoZSBkZWxpbWl0ZXIsIGJ1dCAqYWxzbyogc2V0IGEgZmxhZyB0bwo+ID4gbGV0
IG1lIGtub3cuIiBUaGF0IGNhbid0IGJlIGRvbmUgd2l0aG91dCByZXNvcnRpbmcgdG8gYnl0ZWNv
ZGUgaGFja3MuCj4gPiAoT3IgcGVyaGFwcywgaW4gUHl0aG9uIDIuNSwgQVNUIG1hbmlwdWxhdGlv
bikuIEFuZCB0aGF0J3MgZXZlbiB3b3JzZQo+ID4gdGhhbiByZXdyaXRpbmcgc3RyaW5nLlRlbXBs
YXRlIG15c2VsZi4KPiA+Cj4gPiBBdCB0aGUgZW5kIG9mIHRoZSBkYXksIHRoZXJlJ3Mgbm8gZ2V0
dGluZyBhcm91bmQgaXQgLS0gc3RyaW5nLlRlbXBsYXRlCj4gPiBpcyBub3QgZGVzaWduZWQgZm9y
IHJlY3Vyc2l2ZSBpbnRlcnBvbGF0aW9uLiBJZiB3ZSB3YW50IHRvIGFsbG93Cj4gPiByZWN1cnNp
dmUgaW50ZXJwb2xhdGlvbiBhbmQgZ2V0IGl0IHJpZ2h0LCB3ZSBuZWVkIHRvIHdyaXRlIHRoZSBj
b2RlCj4gPiBvdXJzZWx2ZXMuIEZvcnR1bmF0ZWx5LCBpdCBjb21lcyBvdXQgcHJldHR5IHNtYWxs
Lgo+ID4KPiBIbW1tLi4uIG9uIHRoZSBvdGhlciBoYW5kIEknbSBub3QgY29udmluY2VkIHJlY3Vy
c2l2ZSBpbnRlcnBvbGF0aW9uIGlzCj4gdXNlZCBieSBhbnlvbmUuIEkgZG9uJ3QgbWluZCBpdCBi
ZWluZyBkcm9wcGVkIGZvciB0aGUgc3RyaW5nIHRlbXBsYXRpbmcKPiBpbnRlcnBvbGF0aW9uIC0g
KmlmKiB0aGF0IG1ha2VzIGl0IGFueSBlYXNpZXIuCj4KPiBGdXp6eW1hbgo+IGh0dHA6Ly93d3cu
dm9pZHNwYWNlLm9yZy51ay9weXRob24vaW5kZXguc2h0bWwKCkhlcmUncyBhIHJldmlzZWQgdmVy
c2lvbiBvZiB0aGUgdGVtcGxhdGUtaW50ZXJwb2xhdGlvbiBwYXRjaCB0aGF0CnN0b3BzIHJlY3Vy
c2luZyB0aGUgbWludXRlIGl0IGVuY291bnRlcnMgYSBkb3VibGVkLXVwIGRlbGltaXRlci4gSXQK
YWRkZWQgMTQgbGluZXMgb2YgbmV3IGNvZGU6IDYgbGluZXMgZGVhbGluZyB3aXRoIHRoZSAic3Rv
cF9yZWN1cnNpb24iCmZsYWcsIGFuZCA4IGxpbmVzIG9mIGNvbW1lbnRzIGV4cGxhaW5pbmcgd2h5
IHRoZSBmbGFnIG1pZ2h0IGdldCBzZXQuCgpJJ3ZlIGdvdCBhIHByZXR0eSBjb21wbGV0ZSAiZm9v
LmluaSIgZmlsZSB0aGF0IGV4ZXJjaXNlcyBtb3N0IG9mIHRoZQpmZWF0dXJlcyBpbiBDb25maWdP
YmoncyBpbnRlcnBvbGF0aW9uIGJ5IG5vdy4gSSBzaG91bGQgdHVybiBpdCBpbnRvIGEKc2V0IG9m
IHNlbnNpYmxlIHRlc3QgY2FzZXMgKGJhc2ljYWxseSwgcmVhZCBpbiBhIGNvbmZpZyBmaWxlIGFu
ZCBtYWtlCnN1cmUgdGhlIGludGVycG9sYXRlZCBvcHRpb25zIGdldCB0aGVpciBjb3JyZWN0IHZh
bHVlcyBhbmQgZG9uJ3QgdGhyb3cKZXhjZXB0aW9ucykuIEJ1dCBJJ20gbm90IHN1cmUgbXkgcmVn
dWxhciB3b3JrbG9hZCB3aWxsIHN0YW5kIGFueSBtb3JlCmluYXR0ZW50aW9uIHRvZGF5LCBzbyBJ
J2xsIGhhdmUgdG8gcHV0IGl0IG9mZiBmb3Igbm93LiBUaGVyZWZvcmUsIGluCmFkZGl0aW9uIHRv
IHRoZSByZXZpc2VkIHBhdGNoLCBJJ20gYXR0YWNoaW5nIHRoZSAiZm9vLmluaSIgZmlsZSBJJ3Zl
CmJlZW4gdXNpbmcgZm9yIHRlc3RpbmcuIElmIHNvbWVvbmUgZWxzZSB3YW50cyB0byB0dXJuIGl0
IGludG8gdGVzdApjYXNlcyBiZWZvcmUgSSBnZXQgYXJvdW5kIHRvIGl0LCBiZSBteSBndWVzdC4K
ClRoZSBvbmUgdGhpbmcgdGhhdCBJIGhhdmVuJ3QgaW5jbHVkZWQgaW4gbXkgZG9jdW1lbnRhdGlv
biBwYXRjaCwgdGhhdAptaWdodCBwb3NzaWJseSBiZSBzdXJwcmlzaW5nIGJlaGF2aW9yIHRvIHNv
bWVib2R5IHdobyBoYXMgYW4gb2JzY3VyZQpjb3JuZXIgY2FzZSwgaXMgdGhlIGZhY3QgdGhhdCBy
ZWN1cnNpb24gc3RvcHMgd2hlbiBpdCBoaXRzIGEgJCQuIFNvIGlmCnNvbWVvbmUncyBkb25lIHNv
bWV0aGluZyBsaWtlOgoKZmlyc3QgPSBmb28Kc2Vjb25kID0gJCRiYXIKdGhpcmQgPSAiJGZpcnN0
IGFuZCAkc2Vjb25kIgpmb3VydGggPSAiJHRoaXJkIGFuZCAkc2Vjb25kIGFuZCAkZmlyc3QiCmZp
ZnRoID0gJGZvdXJ0aAoKV2hlbiAiJGZvdXJ0aCIgaXMgaW50ZXJwb2xhdGVkLCB0aGUgcmVzdWx0
IHdpbGwgYmUgIiR0aGlyZCBhbmQgJHNlY29uZAphbmQgJGZpcnN0Ii4gVGhhdCB3aWxsIHlpZWxk
ICIkZmlyc3QgYW5kICRzZWNvbmQgYW5kICQkYmFyIGFuZCBmb28iLgpXaGljaCBnaXZlcyAiZm9v
IGFuZCAkJGJhciBhbmQgJGJhciBhbmQgZm9vIiAtLSBhbmQgYmVjYXVzZSBhICIkJCIKanVzdCB3
ZW50IHBhc3QsIGludGVycG9sYXRpb24gc3RvcHMgcmlnaHQgdGhlcmUuCgpUaGF0IG1pZ2h0IHN1
cnByaXNlIHNvbWVvbmUsIGJ1dCBJJ20gaGF2aW5nIGEgcmVhbCBoYXJkIHRpbWUgY29taW5nIHVw
CndpdGggcmVhc29ucyB3aHkgYW55b25lIHdvdWxkIHdhbnQgdG8gZG8gc29tZXRoaW5nIHR3aXN0
ZWQgbGlrZSB0aGlzLgo6LSkgU28gdGhlICJzdG9wIHdoZW4geW91IGhpdCBhICQkIiBiZWhhdmlv
ciBzaG91bGQgYmUgZmluZSBmb3IgbW9zdApwdXJwb3Nlcy4KCi0tIApSb2JpbiBNdW5uClJvYmlu
Lk11bm5AZ21haWwuY29tCkdQRyBrZXkgMHhENjQ5NzAxNAo=

Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Fuzzyman <fuz...@vo...>]

Robin Munn wrote:

> On 5/23/06, Robin Munn <rob...@gm...> wrote:
>
>> > Instead of reimplementing parts of string.Template, I'd copy it
>> wholesale
>> > from the stdlib string module to an additional module, and do a
>> conditional
>> > import, like this:
>> >
>> > try:
>> >     from string import Template
>> > except ImportError:
>> >     from configobj.string import Template
>> >
>> > That way, when ConfigObj drops support for the older Python
>> versions, the
>> > additional module and the conditional import will be removed, and
>> no code
>> > duplication will remain.
>>
>> That was my first approach, but it didn't actually work. The problem
>> was that string.Template expects you to pass it a single mapping
>> (whether it's a dict or some other dict-like object, it doesn't
>> matter), and doesn't have an approach for saying "If this mapping
>> fails, try this other mapping". I suppose I could have caught the
>> resulting KeyError and called string.Template.substitute() again with
>> the second mapping, but I'm not sure it would have worked, and it
>> would have looked quite a bit uglier than what I ended up with.
>>
>> OTOH, you have a very good point about using already tested and
>> deployed code. So I'll reimplement the TemplateEngine class using
>> string.Template, using the "call substitute() a second and third time
>> if you get KeyError" approach, and see if it works out better. I'll
>> post my results as a second patch if it works out. (Or even if, IMO,
>> it doesn't).
>
>
> Attempting this again, I'm reminded why I abandoned this approach when
> I tried it yesterday. It turns out that because of the way
> string.Template is written, I can't safely replicate ConfigObj's
> current behavior. Here's why:
>
> 1) If ConfigObj doesn't find the value in any of the sections it
> searches (the three DEFAULT subsections), it raises a
> MissingInterpolationError. string.Template can do the same thing,
> since its substitute() function will raise KeyError if the identifier
> given isn't found in the mapping. (E.g., you tell string.Template to
> interpolate "$foo" but don't have a key "foo" in the dict you hand
> it). To chain the DEFAULT sections, I could just chain three calls to
> string.Template.substitute(), only raising MissingInterpolationError
> if the last one fails to find any values. So far so good.
>
> But: 2) ConfigObj also promises recursive interpolation, up to ten
> levels deep. And here I run into a problem. Say I want to put the
> value "$100" in my configuration file. I write it as "money = $$100",
> since string.Template-style interpolation promises that a doubled
> delimiter will be turned into a single delimiter in the output and
> otherwise left untouched. Hence, "$$100" should become "$100". But
> then the recursive interpolation kicks in, and the string "$100" is
> sent around for another try -- where it fails, because "100" is not a
> valid Python identifier. So I can't use substitute(); I need to use
> safe_substitute(), which doesn't raise exceptions when it encounters
> an invalid substitution, but simply leaves it alone. Using
> safe_substitute() instead of substitute() will allow me to leave the
> "$100" string alone. But wait -- if I'm using safe_substitute(), then
> I'm also not raising an exception when I try to interpolate "$foo"
> without a key "foo" in the values dictionary. Oops.
>
> I can't satisfy both 1) and 2) above by using the string.Template
> class. So I need to write my own implementation, that can throw an
> error when it sees "$foo" but will leave "$$100" alone.
>
> Actually, as I was working through this example, I just realized that
> my patch still has a bug in it. It can deal with the template "$$100"
> and produce "$100", and then leave that value alone -- but only
> because "100" isn't a valid Python identifier and thus doesn't match
> the template regexp anymore. But what if you really, really want to
> put the word "$name" in one of the values of your config file? Doing
> "$$name" won't work, because that interprets to "$name" and then a
> MissingInterpolationError will be raised when TemplateEngine can't
> find the key "name" in any of the sections it searches.
>
> What this patch really needs is a way to say "stop the interpolation
> recursion now, we're through". The only approach I've come up with so
> far is to say that if you encounter a $$ in your input, then stop
> interpolating, because you've just turned that $$ into a $. And on the
> next cycle you're going to start interpolating that value, when the
> user has clearly communicated (by his escaping the delimiter) his
> intent for that particular value *not* to be interpolated.
>
> That simply can't be done using string.Template, not even with
> monkeypatching. What you would need is to be able to reach down inside
> the string.Template.substitute() function's *helper* function and add
> a line of code to say "If you find the 'escaped' group in this match
> object, then don't just return the delimiter, but *also* set a flag to
> let me know." That can't be done without resorting to bytecode hacks.
> (Or perhaps, in Python 2.5, AST manipulation). And that's even worse
> than rewriting string.Template myself.
>
> At the end of the day, there's no getting around it -- string.Template
> is not designed for recursive interpolation. If we want to allow
> recursive interpolation and get it right, we need to write the code
> ourselves. Fortunately, it comes out pretty small.
>
Hmmm... on the other hand I'm not convinced recursive interpolation is
used by anyone. I don't mind it being dropped for the string templating
interpolation - *if* that makes it any easier.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Robin M. <rob...@gm...>]

T24gNS8yMy8wNiwgUm9iaW4gTXVubiA8cm9iaW4ubXVubkBnbWFpbC5jb20+IHdyb3RlOgo+ID4g
SW5zdGVhZCBvZiByZWltcGxlbWVudGluZyBwYXJ0cyBvZiBzdHJpbmcuVGVtcGxhdGUsIEknZCBj
b3B5IGl0IHdob2xlc2FsZQo+ID4gZnJvbSB0aGUgc3RkbGliIHN0cmluZyBtb2R1bGUgdG8gYW4g
YWRkaXRpb25hbCBtb2R1bGUsIGFuZCBkbyBhIGNvbmRpdGlvbmFsCj4gPiBpbXBvcnQsIGxpa2Ug
dGhpczoKPiA+Cj4gPiB0cnk6Cj4gPiAgICAgZnJvbSBzdHJpbmcgaW1wb3J0IFRlbXBsYXRlCj4g
PiBleGNlcHQgSW1wb3J0RXJyb3I6Cj4gPiAgICAgZnJvbSBjb25maWdvYmouc3RyaW5nIGltcG9y
dCBUZW1wbGF0ZQo+ID4KPiA+IFRoYXQgd2F5LCB3aGVuIENvbmZpZ09iaiBkcm9wcyBzdXBwb3J0
IGZvciB0aGUgb2xkZXIgUHl0aG9uIHZlcnNpb25zLCB0aGUKPiA+IGFkZGl0aW9uYWwgbW9kdWxl
IGFuZCB0aGUgY29uZGl0aW9uYWwgaW1wb3J0IHdpbGwgYmUgcmVtb3ZlZCwgYW5kIG5vIGNvZGUK
PiA+IGR1cGxpY2F0aW9uIHdpbGwgcmVtYWluLgo+Cj4gVGhhdCB3YXMgbXkgZmlyc3QgYXBwcm9h
Y2gsIGJ1dCBpdCBkaWRuJ3QgYWN0dWFsbHkgd29yay4gVGhlIHByb2JsZW0KPiB3YXMgdGhhdCBz
dHJpbmcuVGVtcGxhdGUgZXhwZWN0cyB5b3UgdG8gcGFzcyBpdCBhIHNpbmdsZSBtYXBwaW5nCj4g
KHdoZXRoZXIgaXQncyBhIGRpY3Qgb3Igc29tZSBvdGhlciBkaWN0LWxpa2Ugb2JqZWN0LCBpdCBk
b2Vzbid0Cj4gbWF0dGVyKSwgYW5kIGRvZXNuJ3QgaGF2ZSBhbiBhcHByb2FjaCBmb3Igc2F5aW5n
ICJJZiB0aGlzIG1hcHBpbmcKPiBmYWlscywgdHJ5IHRoaXMgb3RoZXIgbWFwcGluZyIuIEkgc3Vw
cG9zZSBJIGNvdWxkIGhhdmUgY2F1Z2h0IHRoZQo+IHJlc3VsdGluZyBLZXlFcnJvciBhbmQgY2Fs
bGVkIHN0cmluZy5UZW1wbGF0ZS5zdWJzdGl0dXRlKCkgYWdhaW4gd2l0aAo+IHRoZSBzZWNvbmQg
bWFwcGluZywgYnV0IEknbSBub3Qgc3VyZSBpdCB3b3VsZCBoYXZlIHdvcmtlZCwgYW5kIGl0Cj4g
d291bGQgaGF2ZSBsb29rZWQgcXVpdGUgYSBiaXQgdWdsaWVyIHRoYW4gd2hhdCBJIGVuZGVkIHVw
IHdpdGguCj4KPiBPVE9ILCB5b3UgaGF2ZSBhIHZlcnkgZ29vZCBwb2ludCBhYm91dCB1c2luZyBh
bHJlYWR5IHRlc3RlZCBhbmQKPiBkZXBsb3llZCBjb2RlLiBTbyBJJ2xsIHJlaW1wbGVtZW50IHRo
ZSBUZW1wbGF0ZUVuZ2luZSBjbGFzcyB1c2luZwo+IHN0cmluZy5UZW1wbGF0ZSwgdXNpbmcgdGhl
ICJjYWxsIHN1YnN0aXR1dGUoKSBhIHNlY29uZCBhbmQgdGhpcmQgdGltZQo+IGlmIHlvdSBnZXQg
S2V5RXJyb3IiIGFwcHJvYWNoLCBhbmQgc2VlIGlmIGl0IHdvcmtzIG91dCBiZXR0ZXIuIEknbGwK
PiBwb3N0IG15IHJlc3VsdHMgYXMgYSBzZWNvbmQgcGF0Y2ggaWYgaXQgd29ya3Mgb3V0LiAoT3Ig
ZXZlbiBpZiwgSU1PLAo+IGl0IGRvZXNuJ3QpLgoKQXR0ZW1wdGluZyB0aGlzIGFnYWluLCBJJ20g
cmVtaW5kZWQgd2h5IEkgYWJhbmRvbmVkIHRoaXMgYXBwcm9hY2ggd2hlbgpJIHRyaWVkIGl0IHll
c3RlcmRheS4gSXQgdHVybnMgb3V0IHRoYXQgYmVjYXVzZSBvZiB0aGUgd2F5CnN0cmluZy5UZW1w
bGF0ZSBpcyB3cml0dGVuLCBJIGNhbid0IHNhZmVseSByZXBsaWNhdGUgQ29uZmlnT2JqJ3MKY3Vy
cmVudCBiZWhhdmlvci4gSGVyZSdzIHdoeToKCjEpIElmIENvbmZpZ09iaiBkb2Vzbid0IGZpbmQg
dGhlIHZhbHVlIGluIGFueSBvZiB0aGUgc2VjdGlvbnMgaXQKc2VhcmNoZXMgKHRoZSB0aHJlZSBE
RUZBVUxUIHN1YnNlY3Rpb25zKSwgaXQgcmFpc2VzIGEKTWlzc2luZ0ludGVycG9sYXRpb25FcnJv
ci4gc3RyaW5nLlRlbXBsYXRlIGNhbiBkbyB0aGUgc2FtZSB0aGluZywKc2luY2UgaXRzIHN1YnN0
aXR1dGUoKSBmdW5jdGlvbiB3aWxsIHJhaXNlIEtleUVycm9yIGlmIHRoZSBpZGVudGlmaWVyCmdp
dmVuIGlzbid0IGZvdW5kIGluIHRoZSBtYXBwaW5nLiAoRS5nLiwgeW91IHRlbGwgc3RyaW5nLlRl
bXBsYXRlIHRvCmludGVycG9sYXRlICIkZm9vIiBidXQgZG9uJ3QgaGF2ZSBhIGtleSAiZm9vIiBp
biB0aGUgZGljdCB5b3UgaGFuZAppdCkuIFRvIGNoYWluIHRoZSBERUZBVUxUIHNlY3Rpb25zLCBJ
IGNvdWxkIGp1c3QgY2hhaW4gdGhyZWUgY2FsbHMgdG8Kc3RyaW5nLlRlbXBsYXRlLnN1YnN0aXR1
dGUoKSwgb25seSByYWlzaW5nIE1pc3NpbmdJbnRlcnBvbGF0aW9uRXJyb3IKaWYgdGhlIGxhc3Qg
b25lIGZhaWxzIHRvIGZpbmQgYW55IHZhbHVlcy4gU28gZmFyIHNvIGdvb2QuCgpCdXQ6IDIpIENv
bmZpZ09iaiBhbHNvIHByb21pc2VzIHJlY3Vyc2l2ZSBpbnRlcnBvbGF0aW9uLCB1cCB0byB0ZW4K
bGV2ZWxzIGRlZXAuIEFuZCBoZXJlIEkgcnVuIGludG8gYSBwcm9ibGVtLiBTYXkgSSB3YW50IHRv
IHB1dCB0aGUKdmFsdWUgIiQxMDAiIGluIG15IGNvbmZpZ3VyYXRpb24gZmlsZS4gSSB3cml0ZSBp
dCBhcyAibW9uZXkgPSAkJDEwMCIsCnNpbmNlIHN0cmluZy5UZW1wbGF0ZS1zdHlsZSBpbnRlcnBv
bGF0aW9uIHByb21pc2VzIHRoYXQgYSBkb3VibGVkCmRlbGltaXRlciB3aWxsIGJlIHR1cm5lZCBp
bnRvIGEgc2luZ2xlIGRlbGltaXRlciBpbiB0aGUgb3V0cHV0IGFuZApvdGhlcndpc2UgbGVmdCB1
bnRvdWNoZWQuIEhlbmNlLCAiJCQxMDAiIHNob3VsZCBiZWNvbWUgIiQxMDAiLiBCdXQKdGhlbiB0
aGUgcmVjdXJzaXZlIGludGVycG9sYXRpb24ga2lja3MgaW4sIGFuZCB0aGUgc3RyaW5nICIkMTAw
IiBpcwpzZW50IGFyb3VuZCBmb3IgYW5vdGhlciB0cnkgLS0gd2hlcmUgaXQgZmFpbHMsIGJlY2F1
c2UgIjEwMCIgaXMgbm90IGEKdmFsaWQgUHl0aG9uIGlkZW50aWZpZXIuIFNvIEkgY2FuJ3QgdXNl
IHN1YnN0aXR1dGUoKTsgSSBuZWVkIHRvIHVzZQpzYWZlX3N1YnN0aXR1dGUoKSwgd2hpY2ggZG9l
c24ndCByYWlzZSBleGNlcHRpb25zIHdoZW4gaXQgZW5jb3VudGVycwphbiBpbnZhbGlkIHN1YnN0
aXR1dGlvbiwgYnV0IHNpbXBseSBsZWF2ZXMgaXQgYWxvbmUuIFVzaW5nCnNhZmVfc3Vic3RpdHV0
ZSgpIGluc3RlYWQgb2Ygc3Vic3RpdHV0ZSgpIHdpbGwgYWxsb3cgbWUgdG8gbGVhdmUgdGhlCiIk
MTAwIiBzdHJpbmcgYWxvbmUuIEJ1dCB3YWl0IC0tIGlmIEknbSB1c2luZyBzYWZlX3N1YnN0aXR1
dGUoKSwgdGhlbgpJJ20gYWxzbyBub3QgcmFpc2luZyBhbiBleGNlcHRpb24gd2hlbiBJIHRyeSB0
byBpbnRlcnBvbGF0ZSAiJGZvbyIKd2l0aG91dCBhIGtleSAiZm9vIiBpbiB0aGUgdmFsdWVzIGRp
Y3Rpb25hcnkuIE9vcHMuCgpJIGNhbid0IHNhdGlzZnkgYm90aCAxKSBhbmQgMikgYWJvdmUgYnkg
dXNpbmcgdGhlIHN0cmluZy5UZW1wbGF0ZQpjbGFzcy4gU28gSSBuZWVkIHRvIHdyaXRlIG15IG93
biBpbXBsZW1lbnRhdGlvbiwgdGhhdCBjYW4gdGhyb3cgYW4KZXJyb3Igd2hlbiBpdCBzZWVzICIk
Zm9vIiBidXQgd2lsbCBsZWF2ZSAiJCQxMDAiIGFsb25lLgoKQWN0dWFsbHksIGFzIEkgd2FzIHdv
cmtpbmcgdGhyb3VnaCB0aGlzIGV4YW1wbGUsIEkganVzdCByZWFsaXplZCB0aGF0Cm15IHBhdGNo
IHN0aWxsIGhhcyBhIGJ1ZyBpbiBpdC4gSXQgY2FuIGRlYWwgd2l0aCB0aGUgdGVtcGxhdGUgIiQk
MTAwIgphbmQgcHJvZHVjZSAiJDEwMCIsIGFuZCB0aGVuIGxlYXZlIHRoYXQgdmFsdWUgYWxvbmUg
LS0gYnV0IG9ubHkKYmVjYXVzZSAiMTAwIiBpc24ndCBhIHZhbGlkIFB5dGhvbiBpZGVudGlmaWVy
IGFuZCB0aHVzIGRvZXNuJ3QgbWF0Y2gKdGhlIHRlbXBsYXRlIHJlZ2V4cCBhbnltb3JlLiBCdXQg
d2hhdCBpZiB5b3UgcmVhbGx5LCByZWFsbHkgd2FudCB0bwpwdXQgdGhlIHdvcmQgIiRuYW1lIiBp
biBvbmUgb2YgdGhlIHZhbHVlcyBvZiB5b3VyIGNvbmZpZyBmaWxlPyBEb2luZwoiJCRuYW1lIiB3
b24ndCB3b3JrLCBiZWNhdXNlIHRoYXQgaW50ZXJwcmV0cyB0byAiJG5hbWUiIGFuZCB0aGVuIGEK
TWlzc2luZ0ludGVycG9sYXRpb25FcnJvciB3aWxsIGJlIHJhaXNlZCB3aGVuIFRlbXBsYXRlRW5n
aW5lIGNhbid0CmZpbmQgdGhlIGtleSAibmFtZSIgaW4gYW55IG9mIHRoZSBzZWN0aW9ucyBpdCBz
ZWFyY2hlcy4KCldoYXQgdGhpcyBwYXRjaCByZWFsbHkgbmVlZHMgaXMgYSB3YXkgdG8gc2F5ICJz
dG9wIHRoZSBpbnRlcnBvbGF0aW9uCnJlY3Vyc2lvbiBub3csIHdlJ3JlIHRocm91Z2giLiBUaGUg
b25seSBhcHByb2FjaCBJJ3ZlIGNvbWUgdXAgd2l0aCBzbwpmYXIgaXMgdG8gc2F5IHRoYXQgaWYg
eW91IGVuY291bnRlciBhICQkIGluIHlvdXIgaW5wdXQsIHRoZW4gc3RvcAppbnRlcnBvbGF0aW5n
LCBiZWNhdXNlIHlvdSd2ZSBqdXN0IHR1cm5lZCB0aGF0ICQkIGludG8gYSAkLiBBbmQgb24gdGhl
Cm5leHQgY3ljbGUgeW91J3JlIGdvaW5nIHRvIHN0YXJ0IGludGVycG9sYXRpbmcgdGhhdCB2YWx1
ZSwgd2hlbiB0aGUKdXNlciBoYXMgY2xlYXJseSBjb21tdW5pY2F0ZWQgKGJ5IGhpcyBlc2NhcGlu
ZyB0aGUgZGVsaW1pdGVyKSBoaXMKaW50ZW50IGZvciB0aGF0IHBhcnRpY3VsYXIgdmFsdWUgKm5v
dCogdG8gYmUgaW50ZXJwb2xhdGVkLgoKVGhhdCBzaW1wbHkgY2FuJ3QgYmUgZG9uZSB1c2luZyBz
dHJpbmcuVGVtcGxhdGUsIG5vdCBldmVuIHdpdGgKbW9ua2V5cGF0Y2hpbmcuIFdoYXQgeW91IHdv
dWxkIG5lZWQgaXMgdG8gYmUgYWJsZSB0byByZWFjaCBkb3duIGluc2lkZQp0aGUgc3RyaW5nLlRl
bXBsYXRlLnN1YnN0aXR1dGUoKSBmdW5jdGlvbidzICpoZWxwZXIqIGZ1bmN0aW9uIGFuZCBhZGQK
YSBsaW5lIG9mIGNvZGUgdG8gc2F5ICJJZiB5b3UgZmluZCB0aGUgJ2VzY2FwZWQnIGdyb3VwIGlu
IHRoaXMgbWF0Y2gKb2JqZWN0LCB0aGVuIGRvbid0IGp1c3QgcmV0dXJuIHRoZSBkZWxpbWl0ZXIs
IGJ1dCAqYWxzbyogc2V0IGEgZmxhZyB0bwpsZXQgbWUga25vdy4iIFRoYXQgY2FuJ3QgYmUgZG9u
ZSB3aXRob3V0IHJlc29ydGluZyB0byBieXRlY29kZSBoYWNrcy4KKE9yIHBlcmhhcHMsIGluIFB5
dGhvbiAyLjUsIEFTVCBtYW5pcHVsYXRpb24pLiBBbmQgdGhhdCdzIGV2ZW4gd29yc2UKdGhhbiBy
ZXdyaXRpbmcgc3RyaW5nLlRlbXBsYXRlIG15c2VsZi4KCkF0IHRoZSBlbmQgb2YgdGhlIGRheSwg
dGhlcmUncyBubyBnZXR0aW5nIGFyb3VuZCBpdCAtLSBzdHJpbmcuVGVtcGxhdGUKaXMgbm90IGRl
c2lnbmVkIGZvciByZWN1cnNpdmUgaW50ZXJwb2xhdGlvbi4gSWYgd2Ugd2FudCB0byBhbGxvdwpy
ZWN1cnNpdmUgaW50ZXJwb2xhdGlvbiBhbmQgZ2V0IGl0IHJpZ2h0LCB3ZSBuZWVkIHRvIHdyaXRl
IHRoZSBjb2RlCm91cnNlbHZlcy4gRm9ydHVuYXRlbHksIGl0IGNvbWVzIG91dCBwcmV0dHkgc21h
bGwuCgotLSAKUm9iaW4gTXVubgpSb2Jpbi5NdW5uQGdtYWlsLmNvbQpHUEcga2V5IDB4RDY0OTcw
MTQK

Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Robin M. <rob...@gm...>]

T24gNS8yMy8wNiwgTmljb2xhIExhcm9zYSA8bmljb0B0ZWtuaWNvLm5ldD4gd3JvdGU6Cj4gPiBU
aGUgcmVhbCBxdWVzdGlvbiBpcyB3aGV0aGVyIHRoZSBsYXJnZXIgY29kZSBiYXNlIGFuZC9vciBz
dGVlcGVyCj4gPiBsZWFybmluZyBjdXJ2ZSBhcmUgd29ydGggdGhlIGJlbmVmaXRzIHRoZSBuZXcg
ZmVhdHVyZSB3b3VsZCBicmluZy4gSW4KPiA+IHRoaXMgY2FzZSwgSSB0aGluayB0aGV5IGFyZSwg
YXMgSSdsbCBleHBsYWluLgo+Cj4gSSBhZ3JlZS4gVGhlIG5ldyBzeW50YXggaXMgc2ltcGxlciwg
c3VwcG9ydGVkIGJ5IFB5dGhvbiwgYW5kIHdpZGVseSB1c2VkCj4gZWxzZXdoZXJlOyBpdCBzZWVt
cyB3b3J0aCB0aGUgYWRkaXRpb25hbCBidXJkZW4uCj4KPgo+ID4gTXkgcGF0Y2ggYWRkcyA4NSBh
ZGRpdGlvbmFsIGxpbmVzIG9mIGNvZGUsIGluY2x1ZGluZyBjb21tZW50cy4gKFRoZSA3NQo+ID4g
bGluZXMgb2YgY29kZSBmaWd1cmUgSSBxdW90ZWQgYWJvdmUgd2FzIGJlZm9yZSBJIGNvbW1lbnRl
ZCB0aGUgY29kZSB0bwo+ID4gbXkgc2F0aXNmYWN0aW9uKS4gTW9zdCBvZiBDb25maWdPYmoncyAy
MDAwIGxpbmVzIG9mIGNvZGUgZGlkbid0IG5lZWQgdG8KPiA+IGJlIHRvdWNoZWQsIHNvIHRoZSBu
ZXcgZmVhdHVyZSBzaG91bGQgYmUgZWFzeSB0byB1bmRlcnN0YW5kIGFuZCB0aGUKPiA+IGNvZGVi
YXNlJ3MgbWFudGFpbmFiaWxpdHkgc2hvdWxkbid0IHN1ZmZlci4gSSd2ZSBhdHRhY2hlZCB0aGUg
cGF0Y2gsIHNvCj4gPiB5b3UgY2FuIHJldmlldyBpdCBmb3IgeW91cnNlbGYgYW5kIGRlY2lkZSBo
b3cgZWFzeSBvciBkaWZmaWN1bHQgaXQgd291bGQKPiA+IGJlIHRvIG1haW50YWluLiAoSSd2ZSB0
ZXN0ZWQgaXQgYW5kLCBJIGJlbGlldmUsIGdvdHRlbiBhbGwgdGhlIGJ1Z3MKPiA+IHdvcmtlZCBv
dXQpLgo+Cj4gTWluaW1pemluZyB0aGUgcGF0Y2ggbGVuZ3RoIGlzIGdvb2QsIGJ1dCBpdCdzIGV2
ZW4gYmV0dGVyIHRvIHJldXNlIGFscmVhZHkKPiB0ZXN0ZWQgYW5kIGRlcGxveWVkIGNvZGU6IHRo
ZXJlZm9yZSBJJ2QgZ28gZm9yIGEgZGlmZmVyZW50IGltcGxlbWVudGF0aW9uCj4gc3RyYXRlZ3ku
Cj4KPiBJbnN0ZWFkIG9mIHJlaW1wbGVtZW50aW5nIHBhcnRzIG9mIHN0cmluZy5UZW1wbGF0ZSwg
SSdkIGNvcHkgaXQgd2hvbGVzYWxlCj4gZnJvbSB0aGUgc3RkbGliIHN0cmluZyBtb2R1bGUgdG8g
YW4gYWRkaXRpb25hbCBtb2R1bGUsIGFuZCBkbyBhIGNvbmRpdGlvbmFsCj4gaW1wb3J0LCBsaWtl
IHRoaXM6Cj4KPiB0cnk6Cj4gICAgIGZyb20gc3RyaW5nIGltcG9ydCBUZW1wbGF0ZQo+IGV4Y2Vw
dCBJbXBvcnRFcnJvcjoKPiAgICAgZnJvbSBjb25maWdvYmouc3RyaW5nIGltcG9ydCBUZW1wbGF0
ZQo+Cj4gVGhhdCB3YXksIHdoZW4gQ29uZmlnT2JqIGRyb3BzIHN1cHBvcnQgZm9yIHRoZSBvbGRl
ciBQeXRob24gdmVyc2lvbnMsIHRoZQo+IGFkZGl0aW9uYWwgbW9kdWxlIGFuZCB0aGUgY29uZGl0
aW9uYWwgaW1wb3J0IHdpbGwgYmUgcmVtb3ZlZCwgYW5kIG5vIGNvZGUKPiBkdXBsaWNhdGlvbiB3
aWxsIHJlbWFpbi4KClRoYXQgd2FzIG15IGZpcnN0IGFwcHJvYWNoLCBidXQgaXQgZGlkbid0IGFj
dHVhbGx5IHdvcmsuIFRoZSBwcm9ibGVtCndhcyB0aGF0IHN0cmluZy5UZW1wbGF0ZSBleHBlY3Rz
IHlvdSB0byBwYXNzIGl0IGEgc2luZ2xlIG1hcHBpbmcKKHdoZXRoZXIgaXQncyBhIGRpY3Qgb3Ig
c29tZSBvdGhlciBkaWN0LWxpa2Ugb2JqZWN0LCBpdCBkb2Vzbid0Cm1hdHRlciksIGFuZCBkb2Vz
bid0IGhhdmUgYW4gYXBwcm9hY2ggZm9yIHNheWluZyAiSWYgdGhpcyBtYXBwaW5nCmZhaWxzLCB0
cnkgdGhpcyBvdGhlciBtYXBwaW5nIi4gSSBzdXBwb3NlIEkgY291bGQgaGF2ZSBjYXVnaHQgdGhl
CnJlc3VsdGluZyBLZXlFcnJvciBhbmQgY2FsbGVkIHN0cmluZy5UZW1wbGF0ZS5zdWJzdGl0dXRl
KCkgYWdhaW4gd2l0aAp0aGUgc2Vjb25kIG1hcHBpbmcsIGJ1dCBJJ20gbm90IHN1cmUgaXQgd291
bGQgaGF2ZSB3b3JrZWQsIGFuZCBpdAp3b3VsZCBoYXZlIGxvb2tlZCBxdWl0ZSBhIGJpdCB1Z2xp
ZXIgdGhhbiB3aGF0IEkgZW5kZWQgdXAgd2l0aC4KCk9UT0gsIHlvdSBoYXZlIGEgdmVyeSBnb29k
IHBvaW50IGFib3V0IHVzaW5nIGFscmVhZHkgdGVzdGVkIGFuZApkZXBsb3llZCBjb2RlLiBTbyBJ
J2xsIHJlaW1wbGVtZW50IHRoZSBUZW1wbGF0ZUVuZ2luZSBjbGFzcyB1c2luZwpzdHJpbmcuVGVt
cGxhdGUsIHVzaW5nIHRoZSAiY2FsbCBzdWJzdGl0dXRlKCkgYSBzZWNvbmQgYW5kIHRoaXJkIHRp
bWUKaWYgeW91IGdldCBLZXlFcnJvciIgYXBwcm9hY2gsIGFuZCBzZWUgaWYgaXQgd29ya3Mgb3V0
IGJldHRlci4gSSdsbApwb3N0IG15IHJlc3VsdHMgYXMgYSBzZWNvbmQgcGF0Y2ggaWYgaXQgd29y
a3Mgb3V0LiAoT3IgZXZlbiBpZiwgSU1PLAppdCBkb2Vzbid0KS4KClAuUy4gQWxvbmcgdGhlIHdh
eSwgSSBkaXNjb3ZlcmVkIHRoYXQgdGhlIHdheSBpbnRlcnBvbGF0aW9uIHNlbGVjdHMKdGhlIHZh
bHVlcyB0byBpbnNlcnQgKG9ubHkgbG9va2luZyBhdCBERUZBVUxUIHNlY3Rpb25zKSBkb2Vzbid0
IG1hdGNoCndoYXQgSSB3YXMgZXhwZWN0aW5nLiBXaGF0IEkgZXhwZWN0ZWQgd2FzIHRoYXQgaXQg
d291bGQgZmlyc3QgbG9vayBpbgp0aGUgKmN1cnJlbnQqIHNlY3Rpb24sIHRoZW4gdGhlIGN1cnJl
bnQgc2VjdGlvbidzIERFRkFVTFQsIHRoZW4gdGhlCmN1cnJlbnQgc2VjdGlvbidzIHBhcmVudCwg
dGhlbiB0aGUgcGFyZW50J3MgREVGQVVMVCwgYW5kIHNvIG9uIHVwIHRoZQp0cmVlLiBIb3dldmVy
LCB0aGF0J3MgYSBjb21wbGV0ZWx5IHNlcGFyYXRlIGRpc2N1c3Npb24gZnJvbSB3aGV0aGVyCnN0
cmluZy5UZW1wbGF0ZS1zdHlsZSBpbnRlcnBvbGF0aW9uIHNob3VsZCBnbyBpbiwgYW5kIHNvIEkn
bGwgc2F2ZQp0aGF0IG9uZSBmb3IgYSBkaWZmZXJlbnQgdGhyZWFkLgoKLS0gClJvYmluIE11bm4K
Um9iaW4uTXVubkBnbWFpbC5jb20KR1BHIGtleSAweEQ2NDk3MDE0Cg==

Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Nicola L. <ni...@te...>]

> The real question is whether the larger code base and/or steeper
> learning curve are worth the benefits the new feature would bring. In
> this case, I think they are, as I'll explain.

I agree. The new syntax is simpler, supported by Python, and widely used
elsewhere; it seems worth the additional burden.


> My patch adds 85 additional lines of code, including comments. (The 75
> lines of code figure I quoted above was before I commented the code to
> my satisfaction). Most of ConfigObj's 2000 lines of code didn't need to 
> be touched, so the new feature should be easy to understand and the 
> codebase's mantainability shouldn't suffer. I've attached the patch, so
> you can review it for yourself and decide how easy or difficult it would
> be to maintain. (I've tested it and, I believe, gotten all the bugs
> worked out).

Minimizing the patch length is good, but it's even better to reuse already
tested and deployed code: therefore I'd go for a different implementation
strategy.

Instead of reimplementing parts of string.Template, I'd copy it wholesale
from the stdlib string module to an additional module, and do a conditional
import, like this:

try:
    from string import Template
except ImportError:
    from configobj.string import Template

That way, when ConfigObj drops support for the older Python versions, the
additional module and the conditional import will be removed, and no code
duplication will remain.


-- 
Nicola Larosa - http://www.tekNico.net/

As for the music factories--a.k.a. the major record companies--what they
want is power. They will never accept P2P sharing as long as it remains
a way to escape from their power. For their abuses against the people,
they deserve to be abolished, and that should be everyone's goal.
 -- Richard Stallman, February 2006