Re: [Configobj-develop] Simpler interpolation using PEP 292 $templates

[Fuzzyman <fuz...@vo...>]

Robin Munn wrote:

> On 5/23/06, Robin Munn <rob...@gm...> wrote:
>
>> > Instead of reimplementing parts of string.Template, I'd copy it
>> wholesale
>> > from the stdlib string module to an additional module, and do a
>> conditional
>> > import, like this:
>> >
>> > try:
>> >     from string import Template
>> > except ImportError:
>> >     from configobj.string import Template
>> >
>> > That way, when ConfigObj drops support for the older Python
>> versions, the
>> > additional module and the conditional import will be removed, and
>> no code
>> > duplication will remain.
>>
>> That was my first approach, but it didn't actually work. The problem
>> was that string.Template expects you to pass it a single mapping
>> (whether it's a dict or some other dict-like object, it doesn't
>> matter), and doesn't have an approach for saying "If this mapping
>> fails, try this other mapping". I suppose I could have caught the
>> resulting KeyError and called string.Template.substitute() again with
>> the second mapping, but I'm not sure it would have worked, and it
>> would have looked quite a bit uglier than what I ended up with.
>>
>> OTOH, you have a very good point about using already tested and
>> deployed code. So I'll reimplement the TemplateEngine class using
>> string.Template, using the "call substitute() a second and third time
>> if you get KeyError" approach, and see if it works out better. I'll
>> post my results as a second patch if it works out. (Or even if, IMO,
>> it doesn't).
>
>
> Attempting this again, I'm reminded why I abandoned this approach when
> I tried it yesterday. It turns out that because of the way
> string.Template is written, I can't safely replicate ConfigObj's
> current behavior. Here's why:
>
> 1) If ConfigObj doesn't find the value in any of the sections it
> searches (the three DEFAULT subsections), it raises a
> MissingInterpolationError. string.Template can do the same thing,
> since its substitute() function will raise KeyError if the identifier
> given isn't found in the mapping. (E.g., you tell string.Template to
> interpolate "$foo" but don't have a key "foo" in the dict you hand
> it). To chain the DEFAULT sections, I could just chain three calls to
> string.Template.substitute(), only raising MissingInterpolationError
> if the last one fails to find any values. So far so good.
>
> But: 2) ConfigObj also promises recursive interpolation, up to ten
> levels deep. And here I run into a problem. Say I want to put the
> value "$100" in my configuration file. I write it as "money = $$100",
> since string.Template-style interpolation promises that a doubled
> delimiter will be turned into a single delimiter in the output and
> otherwise left untouched. Hence, "$$100" should become "$100". But
> then the recursive interpolation kicks in, and the string "$100" is
> sent around for another try -- where it fails, because "100" is not a
> valid Python identifier. So I can't use substitute(); I need to use
> safe_substitute(), which doesn't raise exceptions when it encounters
> an invalid substitution, but simply leaves it alone. Using
> safe_substitute() instead of substitute() will allow me to leave the
> "$100" string alone. But wait -- if I'm using safe_substitute(), then
> I'm also not raising an exception when I try to interpolate "$foo"
> without a key "foo" in the values dictionary. Oops.
>
> I can't satisfy both 1) and 2) above by using the string.Template
> class. So I need to write my own implementation, that can throw an
> error when it sees "$foo" but will leave "$$100" alone.
>
> Actually, as I was working through this example, I just realized that
> my patch still has a bug in it. It can deal with the template "$$100"
> and produce "$100", and then leave that value alone -- but only
> because "100" isn't a valid Python identifier and thus doesn't match
> the template regexp anymore. But what if you really, really want to
> put the word "$name" in one of the values of your config file? Doing
> "$$name" won't work, because that interprets to "$name" and then a
> MissingInterpolationError will be raised when TemplateEngine can't
> find the key "name" in any of the sections it searches.
>
> What this patch really needs is a way to say "stop the interpolation
> recursion now, we're through". The only approach I've come up with so
> far is to say that if you encounter a $$ in your input, then stop
> interpolating, because you've just turned that $$ into a $. And on the
> next cycle you're going to start interpolating that value, when the
> user has clearly communicated (by his escaping the delimiter) his
> intent for that particular value *not* to be interpolated.
>
> That simply can't be done using string.Template, not even with
> monkeypatching. What you would need is to be able to reach down inside
> the string.Template.substitute() function's *helper* function and add
> a line of code to say "If you find the 'escaped' group in this match
> object, then don't just return the delimiter, but *also* set a flag to
> let me know." That can't be done without resorting to bytecode hacks.
> (Or perhaps, in Python 2.5, AST manipulation). And that's even worse
> than rewriting string.Template myself.
>
> At the end of the day, there's no getting around it -- string.Template
> is not designed for recursive interpolation. If we want to allow
> recursive interpolation and get it right, we need to write the code
> ourselves. Fortunately, it comes out pretty small.
>
Hmmm... on the other hand I'm not convinced recursive interpolation is
used by anyone. I don't mind it being dropped for the string templating
interpolation - *if* that makes it any easier.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml