From: Steve H. <sh...@zi...> - 2002-08-16 18:55:12
|
I have put a new PyYaml tarball at the following location: http://mountainwebtools.com/PyYaml/PyYaml_16Aug2002.tgz (about 42K) (I don't have access to yaml.org at the moment, so the tarball on the download page there is now a version behind.) The new PyYaml adds features for serializing objects. The distribution also includes a file called TestClasses.py, which shows various use cases for dumping and loading objects. Here is some working code for your perusal: import yaml from test import assertEquals from here import flushLeft from math import sqrt """ DUMPING OBJECTS: > YAML has several ways of persisting objects. One view of an object is that it's nothing more than it's dictionary. By default, YAML emits self.__dict__ from objects. """ class SimpleObject: def __init__(self): self.x = 100 self.y = 20 def xyz(self): return self.x + self.y + self.z def testDumpingDictionary(): obj = SimpleObject() obj.z = 3 output = yaml.dump(obj) expected = simpleObjectYaml() assertEquals(output, expected) def simpleObjectYaml(): return flushLeft(""" --- !!__main__.SimpleObject x: 100 y: 20 z: 3 """) testDumpingDictionary() """ LOADING OBJECTS: > Of course, we would expect to be able to load that YAML right back into Python, and we can. You get back an object of the intended class, with all of the normal methods. Be aware, though, that if your class creates methods on the fly, or if it does other trickery, then you might get unexpected results. """ def testLoadingDictionary(): obj = yaml.load(simpleObjectYaml()).next() assertEquals(obj.xyz(), 123) testLoadingDictionary() """ CUSTOM DUMPING: > Some times you want more control over how you dump objects in YAML. You might not want to dump all members of the object, for example. Also, you may not want to export the module name. """ class Triangle: def __init__(self,x,y): self.x = x self.y = y self._hypotneuse = sqrt(x*x + y*y) def to_yaml(self): # hide the hypotneuse attribute, it's private; # also use inches instead of feet view = { 'x_inches': self.x * 12, 'y_inches': self.y * 12 } return (view, '!!triangle_in_inches') def testDumpTriangle(): triangle = Triangle(3,4) assertEquals(yaml.dump(triangle), triangleYaml()) def triangleYaml(): return flushLeft(""" --- !!triangle_in_inches x_inches: 36 y_inches: 48 """) testDumpTriangle() """ CUSTOM LOADING: > You have control over the YAML load process too. Normally YAML resolves private types for you automatically, but you can override its behavior. """ class MyResolver: def resolveType(self, data, typestring): if typestring == '!!triangle_in_inches': x = data['x_inches'] / 12 y = data['y_inches'] / 12 return Triangle(x,y) else: raise 'Private type %s not supported' % typestring def testCustomLoad(): obj = yaml.load(triangleYaml(),MyResolver()).next() assertEquals(obj._hypotneuse, 5.0) testCustomLoad() """ USING FROM_YAML(): > Another way to customize the YAML loading process is to supply a from_yaml() method. For example, you might want the loading of an object to have some kind of a side effect. Or, you may need to calculate some value that is not part of the attribute. """ class TestConfig: lastTester = None def from_yaml(self, args): self.tester = args['tester'] self.hitcount = args['hitcount'] + 1 TestConfig.lastTester = self.tester return self def testFromYaml(): yamlData = flushLeft(""" --- !!__main__.TestConfig tester: showell hitcount: 42 """) obj = yaml.load(yamlData).next() assertEquals(obj.hitcount, 43) assertEquals(TestConfig.lastTester, 'showell') testFromYaml() === Comments are welcome. -- Steve |
From: Dave K. <dku...@cu...> - 2002-08-16 20:55:37
|
Below is Python code to convert YAML to XML and to convert XML to YAML. This code makes the assumption that an XML element is represented in YAML by a dictionary (mapping object) that contains the key "name" and optional keys "attributes", "text", and "children". You may be muttering: "You idiot. The whole $@#% reason for YAML is to enable us to avoid using XML." Well, yes. But, then, there will be times when YAML users will need to interchange YAML and XML, for example when they need to receive XML from or send XML to external systems. I think one thing interesting about this code is not that it is impressive, but rather how un-impressive it is. Converting between YAML and XML turns out to be very easy. And that is, it seems to me, a strong point of YAML. I'm sure that clean-up is needed and that corner cases would have to be handled. But, as a first demo, it seems to work well enough. Thanks to Steve Howell for PyYaml. - Dave Dave Kuhlman dku...@re... http://www.rexx.com/~dkuhlman ============================================================= #!/usr/bin/env python import sys, string, re, types import yaml from xml.dom import minidom from xml.dom import Node # # Convert a YAML file to XML and write it to stdout. # def convertYaml2Xml(inFileName): inobj = yaml.loadFile(inFileName) out = [] level = 0 convertYaml2XmlAux(inobj, level, out) outStr = "".join(out) sys.stdout.write(outStr) def convertYaml2XmlAux(inobj, level, out): for obj in inobj: if type(obj) == types.DictType and \ obj.has_key('name'): addLevel(level, out) out.append('<%s' % obj['name']) if obj.has_key('attributes'): for key in obj['attributes']: out.append(' %s="%s"' % (key, obj['attributes'][key])) if not (obj.has_key('children') or obj.has_key('text')): out.append('/>\n') else: if obj.has_key('children'): out.append('>\n') else: out.append('>') if obj.has_key('text'): out.append(obj['text']) if obj.has_key('children'): convertYaml2XmlAux(obj['children'], level + 1, out) addLevel(level, out) out.append('</%s>\n' % obj['name']) # # Convert an XML document (file) to YAML and write it to stdout. # def convertXml2Yaml(inFileName): doc = minidom.parse(inFileName) out = [] out.append('---\n') level = 0 root = doc.childNodes[0] convertXml2YamlAux(root, level, out) content = "".join(out) sys.stdout.write(content) def convertXml2YamlAux(obj, level, out): addLevel(level, out) out.append('-\n') level += 1 addLevel(level, out) out.append('name: %s\n' % obj.nodeName) # Dump the attributes. attrs = obj.attributes if attrs.length > 0: addLevel(level, out) out.append('attributes:\n') level += 1 for idx in range(attrs.length): attr = attrs.item(idx) addLevel(level, out) out.append('%s: %s\n' % (attr.name, attr.value)) level -= 1 # Dump the text. text = [] for child in obj.childNodes: if child.nodeType == Node.TEXT_NODE and \ not isAllWhiteSpace(child.nodeValue): text.append(child.nodeValue) if text: textStr = "".join(text) addLevel(level, out) out.append('text: %s\n' % textStr) # Dump the children. found = 0 for child in obj.childNodes: if child.nodeType == Node.ELEMENT_NODE: found = 1 break if found: addLevel(level, out) out.append('children:\n') level += 1 for child in obj.childNodes: if child.nodeType == Node.ELEMENT_NODE: convertXml2YamlAux(child, level, out) # # Utility functions. # def addLevel(level, out): for idx in range(level): out.append(' ') NonWhiteSpacePattern = re.compile('\S') def isAllWhiteSpace(text): if NonWhiteSpacePattern.search(text): return 0 return 1 USAGE_TEXT = """ Convert a file from YAML to XML or XML to YAML and write it to stdout. Usage: python convertyaml.py <option> <in_file> Options: -y2x Convert YAML file to XML document. -x2y Convert XML document to YAML. """ def usage(): print USAGE_TEXT sys.exit(-1) def main(): args = sys.argv[1:] if len(args) != 2: usage() option = args[0] inFileName = args[1] if option == '-y2x': convertYaml2Xml(inFileName) elif option == '-x2y': convertXml2Yaml(inFileName) else: usage() if __name__ == '__main__': main() #import pdb #pdb.run('main()') |
From: Steve H. <sh...@zi...> - 2002-08-16 21:27:23
|
Dave Kuhlman wrote: > You may be muttering: "You idiot. The whole $@#% reason for YAML > is to enable us to avoid using XML." [...] Not at all! A few us have been down the same path in Perl-land. It's great when you take advantage of the best of both worlds. > I think one thing interesting about this code is not that it is > impressive, but rather how un-impressive it is. Converting between > YAML and XML turns out to be very easy. And that is, it seems to > me, a strong point of YAML. > Cool. Map-like structures in XML transform nicely to YAML, and vice versa, but arrays pose a trickier problem. We definitely have wrestled with XML like this: <books> <book>Programming Ruby</book> <book>Python Cookbook</book> <books> How would this map to YAML? There are a couple solutions, and they have different tradeoffs. > I'm sure that clean-up is needed and that corner cases would have > to be handled. But, as a first demo, it seems to work well enough. > Let me know if you'd like me to put this in the PyYaml distribution. I think a lot of folks would find it useful. I would make one minor request, though--any chance that you could have it call yaml.dump() instead of manually formatting the YAML? I suggest this for two reasons: 1) If you have an intermediate data structure, you have more flexibility. For example, you might go from XML to YAML, but you might cache data in pickle. We did a similar thing in Perl, using Storable as our cache. 2) If you use yaml.dump(), you'll get all the upcoming yaml.dump() features for free, plus you can help me debug it. ;) Cheers, Steve |
From: Dave K. <dku...@cu...> - 2002-08-16 22:51:35
|
On Fri, Aug 16, 2002 at 05:26:45PM -0400, Steve Howell wrote: > Dave Kuhlman wrote: > > You may be muttering: "You idiot. The whole $@#% reason for YAML > > is to enable us to avoid using XML." [...] > > Not at all! A few us have been down the same path in Perl-land. It's great > when you take advantage of the best of both worlds. > > > I think one thing interesting about this code is not that it is > > impressive, but rather how un-impressive it is. Converting between > > YAML and XML turns out to be very easy. And that is, it seems to > > me, a strong point of YAML. > > > > Cool. Map-like structures in XML transform nicely to YAML, and > vice versa, but arrays pose a trickier problem. We definitely > have wrestled with XML like this: > > <books> > <book>Programming Ruby</book> > <book>Python Cookbook</book> > <books> > > How would this map to YAML? There are a couple solutions, and they have > different tradeoffs. Here is what my code generates: #================================ --- - name: books children: - name: book text: Programming Ruby - name: book text: Python Cookbook #================================ Not trying to say that this is optimal. Certainly, there are other representations that may have benefits for particular uses. > > > I'm sure that clean-up is needed and that corner cases would have > > to be handled. But, as a first demo, it seems to work well enough. > > > > Let me know if you'd like me to put this in the PyYaml > distribution. I think a lot of folks would find it useful. I > would make one minor request, though--any chance that you could > have it call yaml.dump() instead of manually formatting the YAML? > I suggest this for two reasons: If you think it might be of value to some PyYaml users, you are of course welcome to include it. I'll do some clean-up, if you have suggestions. > > 1) If you have an intermediate data structure, you have more flexibility. For > example, you might go from XML to YAML, but you might cache data in pickle. We > did a similar thing in Perl, using Storable as our cache. > > 2) If you use yaml.dump(), you'll get all the upcoming yaml.dump() features for > free, plus you can help me debug it. ;) Sounds like a very good suggestion to me. Letting PyYaml do the formatting is definitely a good way to go. I'll look into doing another version. - Dave -- Dave Kuhlman dku...@re... http://www.rexx.com/~dkuhlman |
From: Dave K. <dku...@cu...> - 2002-08-17 23:37:58
Attachments:
convertyaml.tar.gz
|
Steve - A new version of the Python code that converts YAML to XML and back is attached. I took your advice, built Python data structures, and let yaml.dump() do the formatting. I'd like to flatter myself and say that I'd have thought of that if I had enough time. (But how much time?) Since you asked about different YAML representations and formats for XML, I've also included two implementations: one that formats an XML element as a dictionary/mapping and the other that formats an element as a sequence. Basically, one is using keywords (in a dictionary) to represent structures and the other is using position (within a list). The beauty of Python (and your suggestion to shift the formatting out to yaml.dump) is that these routines were concise and clear enough so that implementing the second representation (sequences instead of dictionaries) was simple and took very little time. Once again, if you feel that this conversion code would make a useful demo in PyYaml, you are welcome to include it. Let me know if there is anything more that I can do to help. I'm always looking for ideas. - Dave -- Dave Kuhlman dku...@re... http://www.rexx.com/~dkuhlman |
From: Steve H. <sh...@zi...> - 2002-08-18 17:07:00
|
From: "Dave Kuhlman" <dku...@cu...> > [...] > Since you asked about different YAML representations and formats > for XML, I've also included two implementations: one that formats > an XML element as a dictionary/mapping and the other that formats > an element as a sequence. Basically, one is using keywords (in a > dictionary) to represent structures and the other is using position > (within a list). > [...] > Once again, if you feel that this conversion code would make a > useful demo in PyYaml, you are welcome to include it. > > Let me know if there is anything more that I can do to help. I'm > always looking for ideas. > Dave, I will definitely include your code in the next release of PyYaml, as well as the README and sample xml file. Some things that you could do to help: 1) If you want to actively maintain the YAML/XML conversion code, could you get a perforce account from Neil and make the changes directly. You are also welcome to submit patches to me. 2) Non-Python users will appreciate what you've done with the two different mappings. Is there any chance that you could write this up on the Wiki? Maybe show the person.xml example first, and then show how it maps to the two different YAML representations. Perhaps you could add a link to the page under the Ideas section here: http://wiki.yaml.org/yamlwiki/FrontPage 3) If you're feeling more ambitious, we need somebody to start tackling the mappings between XML schemas and YAML schemas. The work you've done with generateDS.py could probably be leveraged nicely for this, but at this stage, we don't necessarily need code; we could start by discussing formats, and propose some example mappings. I don't mean to overwhelm you with suggestions, but there's a lot of work to do, and you did ask. :) Thanks for contributing the conversion code. I think people will find it quite useful. Cheers, Steve |