|
From: Fuzzyman <fuz...@vo...> - 2006-05-23 11:11:34
|
Robin Munn wrote: > On 5/23/06, Robin Munn <rob...@gm...> wrote: > >> > Instead of reimplementing parts of string.Template, I'd copy it >> wholesale >> > from the stdlib string module to an additional module, and do a >> conditional >> > import, like this: >> > >> > try: >> > from string import Template >> > except ImportError: >> > from configobj.string import Template >> > >> > That way, when ConfigObj drops support for the older Python >> versions, the >> > additional module and the conditional import will be removed, and >> no code >> > duplication will remain. >> >> That was my first approach, but it didn't actually work. The problem >> was that string.Template expects you to pass it a single mapping >> (whether it's a dict or some other dict-like object, it doesn't >> matter), and doesn't have an approach for saying "If this mapping >> fails, try this other mapping". I suppose I could have caught the >> resulting KeyError and called string.Template.substitute() again with >> the second mapping, but I'm not sure it would have worked, and it >> would have looked quite a bit uglier than what I ended up with. >> >> OTOH, you have a very good point about using already tested and >> deployed code. So I'll reimplement the TemplateEngine class using >> string.Template, using the "call substitute() a second and third time >> if you get KeyError" approach, and see if it works out better. I'll >> post my results as a second patch if it works out. (Or even if, IMO, >> it doesn't). > > > Attempting this again, I'm reminded why I abandoned this approach when > I tried it yesterday. It turns out that because of the way > string.Template is written, I can't safely replicate ConfigObj's > current behavior. Here's why: > > 1) If ConfigObj doesn't find the value in any of the sections it > searches (the three DEFAULT subsections), it raises a > MissingInterpolationError. string.Template can do the same thing, > since its substitute() function will raise KeyError if the identifier > given isn't found in the mapping. (E.g., you tell string.Template to > interpolate "$foo" but don't have a key "foo" in the dict you hand > it). To chain the DEFAULT sections, I could just chain three calls to > string.Template.substitute(), only raising MissingInterpolationError > if the last one fails to find any values. So far so good. > > But: 2) ConfigObj also promises recursive interpolation, up to ten > levels deep. And here I run into a problem. Say I want to put the > value "$100" in my configuration file. I write it as "money = $$100", > since string.Template-style interpolation promises that a doubled > delimiter will be turned into a single delimiter in the output and > otherwise left untouched. Hence, "$$100" should become "$100". But > then the recursive interpolation kicks in, and the string "$100" is > sent around for another try -- where it fails, because "100" is not a > valid Python identifier. So I can't use substitute(); I need to use > safe_substitute(), which doesn't raise exceptions when it encounters > an invalid substitution, but simply leaves it alone. Using > safe_substitute() instead of substitute() will allow me to leave the > "$100" string alone. But wait -- if I'm using safe_substitute(), then > I'm also not raising an exception when I try to interpolate "$foo" > without a key "foo" in the values dictionary. Oops. > > I can't satisfy both 1) and 2) above by using the string.Template > class. So I need to write my own implementation, that can throw an > error when it sees "$foo" but will leave "$$100" alone. > > Actually, as I was working through this example, I just realized that > my patch still has a bug in it. It can deal with the template "$$100" > and produce "$100", and then leave that value alone -- but only > because "100" isn't a valid Python identifier and thus doesn't match > the template regexp anymore. But what if you really, really want to > put the word "$name" in one of the values of your config file? Doing > "$$name" won't work, because that interprets to "$name" and then a > MissingInterpolationError will be raised when TemplateEngine can't > find the key "name" in any of the sections it searches. > > What this patch really needs is a way to say "stop the interpolation > recursion now, we're through". The only approach I've come up with so > far is to say that if you encounter a $$ in your input, then stop > interpolating, because you've just turned that $$ into a $. And on the > next cycle you're going to start interpolating that value, when the > user has clearly communicated (by his escaping the delimiter) his > intent for that particular value *not* to be interpolated. > > That simply can't be done using string.Template, not even with > monkeypatching. What you would need is to be able to reach down inside > the string.Template.substitute() function's *helper* function and add > a line of code to say "If you find the 'escaped' group in this match > object, then don't just return the delimiter, but *also* set a flag to > let me know." That can't be done without resorting to bytecode hacks. > (Or perhaps, in Python 2.5, AST manipulation). And that's even worse > than rewriting string.Template myself. > > At the end of the day, there's no getting around it -- string.Template > is not designed for recursive interpolation. If we want to allow > recursive interpolation and get it right, we need to write the code > ourselves. Fortunately, it comes out pretty small. > Hmmm... on the other hand I'm not convinced recursive interpolation is used by anyone. I don't mind it being dropped for the string templating interpolation - *if* that makes it any easier. Fuzzyman http://www.voidspace.org.uk/python/index.shtml |