From: Oren Ben-K. <or...@ri...> - 2002-09-09 15:56:53
|
Clark C. Evans [mailto:cc...@cl...] wrote: > Ok. Today is "face the facts" day for Clark... Clark, don't post such proposals unless you are willing to live with them... I'm very uncomfortable with this direction. BTW, it is orthogonal to the #IMPLICIT/DWIM issue (expressed as point (D) in your post). I find it difficult to swallow that I can't just load an untyped mapping node to a Perl hash table. For me this is a show-stopper. I think Brian will feel the same. I really don't see the problem with PHP people prefixing each key:value pair with a '- ' if they really need the order for that particular dictionary (which, most of the time, they won't). You talk about the "shift on this list". I'm not certain you aren't over-stating the case. Of course maybe I'm wrong... Quick poll: How many people feel it is worth giving up on loading YAML maps directly into Perl/Python/Ruby/etc. hash tables so that PHP and others will be able to rely on key order _always_ being preserved? (Very seriously, half-scared) Have fun, Oren Ben-Kiki |
From: Oren Ben-K. <or...@ri...> - 2002-09-09 16:14:21
|
Tom Sawyer [mailto:tra...@tr...] wrote: > > How many people feel it is worth giving up on loading YAML maps > > directly into Perl/Python/Ruby/etc. hash tables so that PHP > > and others > > will be able to rely on key order _always_ being preserved? > > no no, i think it should be a sperate construct. One vote the right way :-) > let not loss > the concept of a functioanl map! let simply gain an ordered > dictionary. but oren's notation dosen't quite work acutally: > > - one: A > - two: B > - three: C > > woudn't this be seen as a sequence each with a mapping of a > sinlge node? like this in ruby: > > [ {'one'=>'A'}, {'two'=>'B'}, {'three'=>'C'} ] Or Perl. And since Perl has no built-in "ordered map" construct, that's a perfectly valid way to represent this. Provide some additional methods on top of it that simulate a map (or something - Brian is the Perl guru) and you have your PHP data structure emulation in Perl. If you want some other internal representation, feel free. That's why loaders exist - to map arbitrary YAML nodes to arbitrary native data structures. Brian's favorite phrase was "convert this to a cave painting". > that's not the same as an ordered hash. or am i wrong about > this. thoughts? It is an ordere hash if you _declare_ it to be (e.g., using "!dict"). > just to thro this out there. here's another alternative: > > one-: A > two-: B > three-: C No, there's no need for new syntax. Have fun, Oren Ben-Kiki |
From: Steve H. <sh...@zi...> - 2002-09-09 16:56:09
|
----- Original Message ----- From: "Oren Ben-Kiki" <or...@ri...> > > It is an ordered hash if you _declare_ it to be (e.g., using "!dict"). > "!dict" makes the most sense for fine-grained control. For the invoice use case, though, you might just want to set ordering semantics in the application itself, thereby keeping the YAML pretty. Cheers, Steve |
From: Oren Ben-K. <or...@ri...> - 2002-09-09 16:38:21
|
ir...@ms... [mailto:ir...@ms...] wrote: > I don't want to destroy the ability to load [] structures and > cousins directly into lists, and {} structures directly into > hashes. That *is* much more important to me than preserving > key order or being able to use YAML with duplicate keys. Good. > I don't see why it has to be an either-or issue though. If the > current processing model does not allow it (e.g., drops > essential information in an intermediate stage), can't a side > contingency be added to the spec or the processing model to > accomodate it, an alternate pathway if the user specifically > requests it? Nope. Because what will happen is that I'll take your document and round-trip it via someone's YAML system, that doesn't provide this path (because it is not part of the model), and your key order goes out the window. And if you require _all_ YAML systems to provide this alternate path, then you have, in effect, required it in the model. > If we switch to a list of pairs for both lists and mappings > in the graph layer (if that's what Clark means by a "named > list"), with [] lists having implicit numeric keys, is it > impossible to store > *somewhere* the fact that this named list came from a [], and > that named list came from a {}, so that they can later be > converted to a list or hash respectively? Nope, for two reasons; first, a Java Hashtable (say) has no place to store this info, and second, even if you did store it, the order of the keys is lost anyway and you can't reconstruct it on output. > Re banishing key order to a schema. Forget about it; it isn't relevant. You want order? Use a sequence (using the shorthand notation). You want ordered pairs? Use a sequence of pairs. It is clean syntactically, it is explicit so the reader/author _knows_ the order is significant, and it works. This should be enough, no? Have fun, Oren Ben-Kiki |
From: Oren Ben-K. <or...@ri...> - 2002-09-09 16:40:21
|
Tom Sawyer [mailto:tra...@tr...] wrote: > so i guess i'm looking to go the other way, rather then just > having scalars and collections, i'm looking at more: > > scalars > sequences > functional-maps > ordered-functional-maps > logical-maps > ordered-logical-maps What is a logical map and how is it different from a functional map? > also, clark, the concept of using schemas for ALL of what > we've been talking about might not be a bad notion. Forget about it. As I wrote to mike: you want order, use a sequence. That's all. The syntax is clean, the semantics is explicit, what more can one want? Have fun, Oren Ben-Kiki |
From: Tom S. <tra...@tr...> - 2002-09-09 16:54:26
|
On Mon, 2002-09-09 at 10:41, Oren Ben-Kiki wrote: > Tom Sawyer [mailto:tra...@tr...] wrote: > > so i guess i'm looking to go the other way, rather then just > > having scalars and collections, i'm looking at more: > > > > scalars > > sequences > > functional-maps > > ordered-functional-maps > > logical-maps > > ordered-logical-maps > > What is a logical map and how is it different from a functional map? a logical map allows duplicate keys. think prolog where you can define identical rules that have different results, thus a variable can be bound to more than one symbol. > > also, clark, the concept of using schemas for ALL of what > > we've been talking about might not be a bad notion. > > Forget about it. As I wrote to mike: you want order, use a sequence. That's > all. The syntax is clean, the semantics is explicit, what more can one want? hmmm... so your saying - one: A - two: B - three: C is suitable to ordered hashes, and in the native rep. can be delt with as a single construct by php, while it takes four constructs in ruby (an array and three single node hashes) and this won't cause any problems? if so then that work fine for me. now, how to fit in the duplicate keys? wait...there's the difficulty. this is perfectly valid: - one: A - one: B - one: C and that can't be handled by a php mapping object (whatever they call them). so duplicate keys can work here, but it is the order-functional-maps that need a little help. ideas? -- tom sawyer, aka transami tra...@tr... |
From: Clark C. E. <cc...@cl...> - 2002-09-10 01:02:26
|
On Mon, Sep 09, 2002 at 11:04:58AM -0600, Tom Sawyer wrote: | > Forget about it. As I wrote to mike: you want order, use a sequence. That's | > all. The syntax is clean, the semantics is explicit, what more can one want? | | hmmm... so your saying | | - one: A | - two: B | - three: C | | is suitable to ordered hashes, and in the native rep. can be delt with | as a single construct by php, while it takes four constructs in ruby (an | array and three single node hashes) and this won't cause any problems? It will work just fine. | now, how to fit in the duplicate keys? wait...there's the difficulty. | this is perfectly valid: | | - one: A | - one: B | - one: C | | and that can't be handled by a php mapping object (whatever they call | them). so duplicate keys can work here, but it is the | order-functional-maps that need a little help. Well the alterntive is to move to a model with a named-map, so you have the same problem of duplicate keys anyway. Thinking that you can move to a model with an ordered-map while excluding duplicates may work for Steve; but it won't work for another person who is just like Steve and things that duplicates are the neatest feature since sliced bread... and the parser handles it... so why not use duplicates in the loader? Best, Clark |
From: Oren Ben-K. <or...@ri...> - 2002-09-09 17:01:18
|
Tom Sawyer [mailto:tra...@tr...] wrote: > a logical map allows duplicate keys. think prolog where you > can define identical rules that have different results, thus > a variable can be bound to more than one symbol. Ah. How does one distinguish between the different entries? Is it by their order? > hmmm... so your saying > > - one: A > - two: B > - three: C > > is suitable to ordered hashes, and in the native rep. can be > delt with as a single construct by php, while it takes four > constructs in ruby (an array and three single node hashes) > and this won't cause any problems? Well, in Perl anyway; I think in ruby there's something they can use (which is why they came up with !ruby/pairs which is basically !dict). But yes, that's the idea. > now, how to fit in the duplicate keys? wait...there's the > difficulty. this is perfectly valid: > > - one: A > - one: B > - one: C > > and that can't be handled by a php mapping object (whatever > they call them). so duplicate keys can work here, but it is > the order-functional-maps that need a little help. > > ideas? It is perfectly valid for !multimap (that's the name by which I know "logical maps"). And it isn't a valid !dict (I suppose). So what? Every data type may place additional restriction on its input beyond those that are specified by YAML. For example, a "domain" field in a sendmail configuration file may not contain the character '#'. Does this mean it shouldn't be a string? The price of a product in a store can't be negative; does this mean it shouldn't be a float? I think this is what Clark meant by saying it is "in the schema". A less scary way of saying it is that the semantics of the document place restrictions on the possible values of the node (and this holds for all node kinds - scalars, maps, and sequences). Have fun, Oren Ben-Kiki |
From: Tom S. <tra...@tr...> - 2002-09-09 18:33:12
|
On Mon, 2002-09-09 at 11:02, Oren Ben-Kiki wrote: > > now, how to fit in the duplicate keys? wait...there's the > > difficulty. this is perfectly valid: > > > > - one: A > > - two: B > > - three: C > > just had an ugly thought. if say php represented the above as it's native array type, it would be problematic for them to add duplicate keys. ouch. so this won't cut it. so even for php this has to be four objects three single node arrays in an array. (by the way it looks like php just calls its combined list-map thing an array) well, i can't say i mind this too much. it does give a way to do ordered pairs, even if not represented natively. ummm...but it would make it difficult to serialize a php array, you would need a special type. 6 of one, 1/2 dozen of another. there's got to be a right way for all of this.... -- tom sawyer, aka transami tra...@tr... |
From: Tom S. <tra...@tr...> - 2002-09-09 16:09:00
|
On Mon, 2002-09-09 at 09:58, Oren Ben-Kiki wrote: > How many people feel it is worth giving up on loading YAML maps directly > into Perl/Python/Ruby/etc. hash tables so that PHP and others will be able > to rely on key order _always_ being preserved? no no, i think it should be a sperate construct. let not loss the concept of a functioanl map! let simply gain an ordered dictionary. but oren's notation dosen't quite work acutally: - one: A - two: B - three: C woudn't this be seen as a sequence each with a mapping of a sinlge node? like this in ruby: [ {'one'=>'A'}, {'two'=>'B'}, {'three'=>'C'} ] that's not the same as an ordered hash. or am i wrong about this. thoughts? just to thro this out there. here's another alternative: one-: A two-: B three-: C -- tom sawyer, aka transami tra...@tr... |
From: <ir...@ms...> - 2002-09-09 16:25:00
|
On Mon, Sep 09, 2002 at 06:58:12PM +0300, Oren Ben-Kiki wrote: > How many people feel it is worth giving up on loading YAML maps directly > into Perl/Python/Ruby/etc. hash tables so that PHP and others will be able > to rely on key order _always_ being preserved? I don't want to destroy the ability to load [] structures and cousins directly into lists, and {} structures directly into hashes. That *is* much more important to me than preserving key order or being able to use YAML with duplicate keys. I don't see why it has to be an either-or issue though. If the current processing model does not allow it (e.g., drops essential information in an intermediate stage), can't a side contingency be added to the spec or the processing model to accomodate it, an alternate pathway if the user specifically requests it? If we switch to a list of pairs for both lists and mappings in the graph layer (if that's what Clark means by a "named list"), with [] lists having implicit numeric keys, is it impossible to store *somewhere* the fact that this named list came from a [], and that named list came from a {}, so that they can later be converted to a list or hash respectively? Re banishing key order to a schema. "Schema" is another of those big scary words. In fact, much scarier than the others. We don't have a schema infrastructure yet and won't for several weeks/months. *Maybe* we can come up with a schema syntax that doesn't require reading a 25-page book to do something simple. (I'm being optimistic; I was going to say a "100-page book".) But after having been burned so much with XML schemas, I'll take a "I believe it when I see it" attitude. All of this makes me reluctant to banish anything to the future schema infrastructure except things that are in the basic nature of schemas, namely, "This key is required, and the value must be this type, or one of a choice of values." -- -Mike (Iron) Orr, ir...@ms... (if mail problems: ms...@oz...) http://iron.cx/ English * Esperanto * Russkiy * Deutsch * Espan~ol |
From: Steve H. <sh...@zi...> - 2002-09-09 16:49:21
|
Oren wrote: > How many people feel it is worth giving up on loading YAML maps directly > into Perl/Python/Ruby/etc. hash tables so that PHP and others will be able > to rely on key order _always_ being preserved? > That's going too far. Preserving key ordering should be an allowable extension, but unordered maps are still the primary use case. Cheers, Steve |
From: Clark C. E. <cc...@cl...> - 2002-09-10 02:53:14
|
According to our website, YAML is first and for most a "native object" serialization language esp for languages like Perl and Python. We should stick hard to that goal. However, that said; we can't stop users of the "side-effects" that our serialization format and parser create: ordered keys and duplicate keys. Thus, we have either two options; duck and ignore that they are doing it; or perhaps try to give them _some_ accomidation. As an added "bonus" we become much more XML compatibilish; and this could help adoption in those circles. Now, we can either create another model for this; or we could phrase the "serial" model in such a way that it allows both of these effects to exists. Quite clearly key order comes for free with the serial model. And since node equality can't be examined without the type resolution at this layer we could even allow duplicate keys. This has the added advantage of clarifying that parsers _must not_ try to eliminate duplicate keys. In this way, tools written for the serial model must worry about duplicate keys and maintain key order. Then, at our more abstract level, our "graph" model we simply reject those instances of the serial model which have duplicate keys and we clearly state that node order is not preserved. So. These thingys can co-exist (without creating a new model) just through current clarifications of the existing model. At this point, an Application has two choices. It can use the YAML Parser only, and take its information from the Serial model; or it can use the YAML Loader and restrict itself to maps without duplicates and where key-order is forbidden... implementations may even want to scramble the keys just to make it crystal clear. I would be happy with this compromise -- but it means following the policy Neil laid out: 1. If an Application wants to use key order as information and/or accept duplicate keys, then it must use the Parser interface directly (which provides output in the serial model). 2. If an Application agrees with the generic model, where key order is not signficant and duplicate keys are not allowed, then it may use the Loader interface (which provides the graph model) To implementers this means the following rule to help keep things straight: Your Loader interface should not in any way be using key order or allowing for duplicate nodes in the stuff it generates. Ideally, this means the loader should only pass the application or object type constructors a fully-created random-access hashtable (ideally with pre-scrambled keys). And not provide any loading "hooks" which break this process (and hence enable the user to accidently violate the model). Thus, people have it both ways. If they want to treat YAML as Syntax, they can use the Parser interface. If they want to use YAML as a Serialization Language (with its model) then they use the Loader interface. There will be reasons to use both interfaces... I know it sounds draconian... but I think this is a good compromise position that allows for both models but still keeps YAML's vision as a serialization language for modern scripting languages pure. Best, Clark |
From: <ir...@ms...> - 2002-09-10 04:10:26
|
It sounds like it's time to start building some reference implementations. I thought there was only one way an application could get data from a .yml file, through the loader, which gives you a top-level mappings-as-dictionaries. That's why I wanted an option to the loader that would yield mappings in an alternate format, because of dictionary limitations in several languages. But if the truth is that there can be a "parser" accessible directly by the user, let's go ahead and implement one so that users can see what their .yml files show up as under it, and can start planning their applications accordingly. Right now, PyYAML has a load() function but no parse(), AFAICT. As for Tom's proposal that user-level tools at all level give the application a native structure, I thought that was the plan all along. On Tue, Sep 10, 2002 at 02:55:03AM +0000, Clark C. Evans wrote: > I know it sounds draconian... but I think this is a good > compromise position that allows for both models but still > keeps YAML's vision as a serialization language for > modern scripting languages pure. No, it sounds much less draconaian than your previous proposals. -- -Mike (Iron) Orr, ir...@ms... (if mail problems: ms...@oz...) http://iron.cx/ English * Esperanto * Russkiy * Deutsch * Espan~ol |
From: Steve H. <sh...@zi...> - 2002-09-10 05:26:58
|
----- Original Message ----- From: "Mike Orr" <ir...@ms...> > It sounds like it's time to start building some reference > implementations. I thought there was only one way an application could > get data from a .yml file, through the loader, which gives you a > top-level mappings-as-dictionaries. That's why I wanted an option > to the loader that would yield mappings in an alternate format, > because of dictionary limitations in several languages. But if the > truth is that there can be a "parser" accessible directly by the > user, let's go ahead and implement one so that users can see what > their .yml files show up as under it, and can start planning their > applications accordingly. Right now, PyYAML has a load() function > but no parse(), AFAICT. > Correct. PyYaml just has a load() for now, but it is being rewritten to have an exposed parser layer. The loader will be built on top of that. The loader will basically call into the parser, getting events like StartMap, StartSequence, etc., and then build up a tree-like Python data structure. At first the loader will only populate two concrete data structures--standard Python lists and standard Python maps--to make up this tree. If a few more Mike Orrs twist my arm, though, then I will make a loader that deals with abstract data structures. Mike will need to be smart enough to realize that his ReallyCoolOrderedDictionaryClass that he just wrote three hours ago in Python will not be supported by Clark's ML language generic-model-driven, YAML-compliant, directive-enabled ypath tool until six to twelve months later. I have faith in his common sense (Mike's). Mike's also free to write his own loader that deals directly with the parser. Where's Why The Lucky Stiff through all this discussion? Methinks he's writing the YAML mailing list bot. Cheers, Steve |
From: Clark C. E. <cc...@cl...> - 2002-09-10 14:49:52
|
On Mon, Sep 09, 2002 at 09:10:24PM -0700, Mike Orr wrote: | It sounds like it's time to start building some reference | implementations. This is "libyaml". I'm going to shift focus back to working on this when Neil is ready for me to join the fun. There is a Python binding for libyaml. _why said that he'd also join. Also over the next two-three weeks I'm going to work on a new wording for the information model based on the discussion here and the clarifications which have emerged. It will reflect the logic in the compromise position. All of this will be less-than-part-time as I have much work to do for my day job. Once we have libyaml working better and the information model explained in a more clear manner (thank you for all of your feedback) it'll be time for me to focus on YPATH and a SCHEMA. I figure it will take about 6 months to come up with an initial stable pass. I'm going to approach schema from the basis of RELAXNG (the co-work of James Clark and Murata Makoto) both of these fellas are brilliant people and have probably worked out much of the schema issues we have to worry about. There will be some work, however, as the information model they use is XML. So bridge will have to be spanned here. It'd be great if anyone here wants to drive the YPATH or SCHEMA effort. It is alot of work. Best, Clark |
From: <ir...@ms...> - 2002-09-10 16:32:32
|
On Tue, Sep 10, 2002 at 02:51:42PM +0000, Clark C. Evans wrote: > I'm going to approach schema from the basis of RELAXNG > (the co-work of James Clark and Murata Makoto) both of > these fellas are brilliant people and have probably > worked out much of the schema issues we have to worry > about. There will be some work, however, as the > information model they use is XML. So bridge will > have to be spanned here. When the schema is expressed as an external document, it'd be great if that document itself is YAML. When the schema is expressed as a data structure in the application, or as arguments/attributes to the loader/dumper function, this obviously won't apply, but we'll need some data structure for it. -- -Mike (Iron) Orr, ir...@ms... (if mail problems: ms...@oz...) http://iron.cx/ English * Esperanto * Russkiy * Deutsch * Espan~ol |
From: Brian I. <in...@tt...> - 2002-09-10 07:32:02
|
On 10/09/02 02:55 +0000, Clark C. Evans wrote: > According to our website, YAML is first and for most a "native > object" serialization language esp for languages like Perl and Python. > We should stick hard to that goal. +1 YAML needs to have a solid purpose. Something to offer computing that is unique. For me that purpose is (and has been for the last 16 months) "YAML is a common serialization format for modern computing languages. It is modeled after the basic data structures found in almost all scripting languages: hashes, arrays and scalars. It strives to be as human readable and editable as possible." That's it. With that you can change the world. YAML was the tool I wanted for every project I was working on. I knew the power of Perl (et al) and the myriad uses people would find for a great serialization syntax. That's why I got into this game. Cheers, Brian |
From: Steve H. <sh...@zi...> - 2002-09-10 14:01:03
|
----- Original Message ----- From: "Brian Ingerson" <in...@tt...> > YAML needs to have a solid purpose. Something to offer computing that is > unique. For me that purpose is (and has been for the last 16 months) > > "YAML is a common serialization format for modern computing languages. It > is modeled after the basic data structures found in almost all scripting > languages: hashes, arrays and scalars. It strives to be as human readable > and editable as possible." > And you can also make slide shows with it. Cheers, Steve |
From: <ir...@ms...> - 2002-09-10 16:35:16
|
On Tue, Sep 10, 2002 at 10:00:11AM -0400, Steve Howell wrote: > ----- Original Message ----- > From: "Brian Ingerson" <in...@tt...> > > YAML needs to have a solid purpose. Something to offer computing that is > > unique. For me that purpose is (and has been for the last 16 months) > > > > "YAML is a common serialization format for modern computing languages. It > > is modeled after the basic data structures found in almost all scripting > > languages: hashes, arrays and scalars. It strives to be as human readable > > and editable as possible." > > > > And you can also make slide shows with it. Or keep track of when you feed your goldfish each day, what kind of food you give them and how much. -- -Mike (Iron) Orr, ir...@ms... (if mail problems: ms...@oz...) http://iron.cx/ English * Esperanto * Russkiy * Deutsch * Espan~ol |
From: <ir...@ms...> - 2002-09-10 19:00:12
|
On Tue, Sep 10, 2002 at 02:40:18PM -0400, Steve Howell wrote: > ----- Original Message ----- > From: "Mike Orr" <ir...@ms...> > > Or keep track of when you feed your goldfish each day, what kind of food > > you give them and how much. > > > > That could be PyYaml's killer app. :) Actually, you'd also want to keep track of threats to your goldfish, so you can use it as evidence in court against the perp. -- -Mike (Iron) Orr, ir...@ms... (if mail problems: ms...@oz...) http://iron.cx/ English * Esperanto * Russkiy * Deutsch * Espan~ol |
From: Steve H. <sh...@zi...> - 2002-09-10 19:28:06
|
From: "Mike Orr" <ir...@ms...> > On Tue, Sep 10, 2002 at 02:40:18PM -0400, Steve Howell wrote: > > ----- Original Message ----- > > From: "Mike Orr" <ir...@ms...> > > > Or keep track of when you feed your goldfish each day, what kind of food > > > you give them and how much. > > > > That could be PyYaml's killer app. :) > > Actually, you'd also want to keep track of threats to your goldfish, so > you can use it as evidence in court against the perp. > Maybe we could have some kind of web service for buying goldfish products too. Your Perl CGI program sends this YAML off to the web service: Items: - flakes: 5 lbs - tank cleaner: 3 liters - strainer: 3 The Ruby web service sends back: Items: - ITEM: flakes QTY: 5 lbs PRICE: $5.33 - ITEM: tank cleaner QTY: 3 liters PRICE: $17.22 - ITEM: strainer ERROR: MSG: out of stock till next Wednesday RESTOCK DATE: 10/15/2002 Finally, your Python accounting program can find the above YAML and use it to debit your goldfish account automatically. Cheers, Steve |