From: Kirill S. <xi...@ga...> - 2006-05-03 18:07:28
|
I've got a bug report for PyYAML: http://pyyaml.org/ticket/10 In short, the submitter has the YAML code: Chip: Name: "CHIPNAME" RegSize: 4 << : !include file_name: ymlInclude2.yml and wants the content of the file 'ymlInclude2.yml' to be inserted into the 'Chip' mapping. He defined a constructor for the '!include' tag to read the content of the file specified by 'file_name'. Unfortunately, it doesn't work. According to the merge key definition (http://yaml.org/type/merge.html), the above document is transformed to Chip: Name: "CHIPNAME" RegSize: 4 file_name: ymlInclude2.yml which is not what the submitter wants. Still I think it's a valid use case for the merge key. Perhaps its definition can be modified to allow this example. What do you think? -- xi |
From: Oren Ben-K. <or...@be...> - 2006-05-03 18:35:06
|
On Wednesday 03 May 2006 11:07, Kirill Simonov wrote: > In short, the submitter has the YAML code: > > Chip: > Name: "CHIPNAME" > RegSize: 4 > << : !include > file_name: ymlInclude2.yml > > Unfortunately, it doesn't work. According to the merge key definition > (http://yaml.org/type/merge.html), the above document is transformed > to > > Chip: > Name: "CHIPNAME" > RegSize: 4 > file_name: ymlInclude2.yml > > which is not what the submitter wants. Is this the fault of the !include expansion? I'd expect that !include be converted to a mapping before the value node was passed to << for inclusion... in which case it would just work... or am I missing something? Have fun, Oren Ben-Kiki |
From: Kirill S. <xi...@ga...> - 2006-05-03 18:44:41
|
On Wed, May 03, 2006 at 11:34:59AM -0700, Oren Ben-Kiki wrote: > On Wednesday 03 May 2006 11:07, Kirill Simonov wrote: > > In short, the submitter has the YAML code: > > > > Chip: > > Name: "CHIPNAME" > > RegSize: 4 > > << : !include > > file_name: ymlInclude2.yml > > > > Unfortunately, it doesn't work. According to the merge key definition > > (http://yaml.org/type/merge.html), the above document is transformed > > to > > > > Chip: > > Name: "CHIPNAME" > > RegSize: 4 > > file_name: ymlInclude2.yml > > > > which is not what the submitter wants. > > Is this the fault of the !include expansion? I'd expect that !include be > converted to a mapping before the value node was passed to << for > inclusion... in which case it would just work... or am I missing > something? In this case, the tag '!include' is ignored. From the spec: If the value associated with the key is a single mapping node, each of its key/value pairs is inserted into the current mapping, unless the key already exists in it. PyYAML does what the spec requires. It takes the pair file_name: yamlInclude2.yml and inserts it into the 'Chip' mapping node. The tag '!include' is not even considered. -- xi |
From: Clark C. E. <cc...@cl...> - 2006-05-03 18:48:43
|
On Wed, May 03, 2006 at 09:44:33PM +0300, Kirill Simonov wrote: | On Wed, May 03, 2006 at 11:34:59AM -0700, Oren Ben-Kiki wrote: | > Is this the fault of the !include expansion? I'd expect that !include be | > converted to a mapping before the value node was passed to << for | > inclusion... in which case it would just work... or am I missing | > something? | | In this case, the tag '!include' is ignored. From the spec: | | If the value associated with the key is a single mapping node, each of | its key/value pairs is inserted into the current mapping, unless the key | already exists in it. | | PyYAML does what the spec requires. It takes the pair | file_name: yamlInclude2.yml | and inserts it into the 'Chip' mapping node. The tag '!include' is not | even considered. Perhaps we need to provide an evaluation order for type tags, going leafs first, and values before keys. In this case, the !include would be evaluated before the !!merge tag is activated. At this time, the whole process of how this works is underspecified. Best, Clark |
From: Kirill S. <xi...@ga...> - 2006-05-03 18:59:52
|
> Perhaps we need to provide an evaluation order for type tags, going > leafs first, and values before keys. In this case, the !include would > be evaluated before the !!merge tag is activated. At this time, the > whole process of how this works is underspecified. The problem is that the process of merging is described in terms of "nodes". Therefore it is required by the spec that the value of the merge key is not converted to a native object. -- xi |
From: Oren Ben-K. <or...@be...> - 2006-05-03 19:40:15
|
On Wednesday 03 May 2006 12:00, Kirill Simonov wrote: > The problem is that the process of merging is described in terms of > "nodes". Therefore it is required by the spec that the value of the > merge key is not converted to a native object. The spec does say "nodes", but this line of reasoning leads to the conclusion that !include and !merge are impossible in the first place - they are both "nodes" that mysteriously vanish/mutate at some point! Loading is inevitably an incremental process. The spec implies a "bottom up" approach already, e.g. tag resolution of a node may depend on the total of the node's content, but not on the content of sibling nodes. Now, in principle, each node is converted to a single native object (or "cave drawing" as Ingy likes to say). However, reflecting on this for a second it is obvious that when a collection node is loaded to some "cave drawing", there need not be a 1-1 mapping from some specific part of it to a particular node contained in the collection. If we had placed such a restriction, we would no longer allow for _any_ sort of "cave drawing" to be used. An example of this is our !set notation. All the null value nodes "disappear" when we load it into a new Ruby "Set" object. Taken together, I think it is inevitable that loading keys such as !include and !merge is done as a bottom up process that uses the node structure (and tag) as _input_, but has no further restrictions on its behavior. It is good we have !include and !merge in the spec to drive home this point. After all, if this were not the case, !merge and !include wouldn't be possible in the first place. In fact, !merge and !include demonstrate another important concept. In general we expect that loading YAML nodes to "cave drawings" be a reversible process (aka "round tripping"). However, the spec makes no such hard requirement. It is certainly _desirable_ in many cases to build a round-trip system, but it is also desirable in other cases to build a load-only (or dump-only) system. The spec intentionally does not preclude such systems. A system that supports !include and !merge is _typically_ "load only". Dumping the content back would _typically_ simply dump the included/merged keys, which - again - is intentionally OK under the spec, as long as it is understood by all parties that the result is "not the same" as the original YAML document. Have fun, Oren Ben-Kiki |
From: Kirill S. <xi...@ga...> - 2006-05-04 12:37:55
|
On Wednesday 03 May 2006 22:40, Oren Ben-Kiki wrote: > On Wednesday 03 May 2006 12:00, Kirill Simonov wrote: > > The problem is that the process of merging is described in terms of > > "nodes". Therefore it is required by the spec that the value of the > > merge key is not converted to a native object. > > The spec does say "nodes", but this line of reasoning leads to the > conclusion that !include and !merge are impossible in the first place - > they are both "nodes" that mysteriously vanish/mutate at some point! Hmm... I always vaguely felt that implementing !!merge keys is impossible, and now you confirmed my concern. :) Seems that the definition of !!merge keys at http://yaml.org/type/merge.html is not complete enough to make loading unambiguous. As I said, the spec suggests to simply ignore the tag of the value node, which effectively disallow !include tag. Another my concern is what to do if !!merge keys are nested. For instance, consider the document: foo: 1 <<: bar: 2 <<: baz: 3 How should this document be loaded? I can invent even more interesting examples: --- &A <<: *A Now could you tell me what the yaml processor should do with this document? -- Kirill |
From: Oren Ben-K. <or...@be...> - 2006-05-04 15:36:41
|
On Thursday 04 May 2006 05:38, Kirill Simonov wrote: > Hmm... I always vaguely felt that implementing !!merge keys is > impossible, and now you confirmed my concern. :) lol > Seems that the definition of !!merge keys at > http://yaml.org/type/merge.html is not complete enough to make > loading unambiguous. Obviously - you raise excellent points that should go into the spec. > As I said, the spec suggests to simply ignore the tag of the value > node, which effectively disallow !include tag. Where does it say that? I mean, sure you can ignore them if all you want is to manipulate the nodes. A YAML editor would be perfectly within its rights to load all mapping nodes as hashes, sequences as arrays, and scalars as strings, "ignoring" all tags. I'd expect it to _remember_ the tag though, at least if there is any intention on writing the stuff back to disk at any time :-) > Another my concern is what to do if !!merge keys are nested. For > instance, consider the document: > > foo: 1 > <<: > bar: 2 > <<: > baz: 3 > > How should this document be loaded? Since loading content is bottom up, it ends up: foo: 1 bar: 2 baz: 3 While this is atrocious, I don't see that we can ban it. > I can invent even more interesting examples: > > --- &A > <<: *A > > Now could you tell me what the yaml processor should do with this > document? This could either be an error or - if one is charitable - an empty mapping. I'd make that an error - this is abominable and is easy to ban. BTW, a similar thing can happen with a loop of !!include directives, and I'd make it an error there as well. These points should definitely go into the spec. Thanks for debugging this for us! Have fun, Oren Ben-Kiki |