From: Oren Ben-K. <or...@ri...> - 2002-09-08 15:41:18
|
With regard to the models diagram and the need for a native model... I don't think you can do away with mentioning the native model. After all the transfer method is the instruction on how to do the conversion to the native model; if we'd stop at the generic model it would be rather senseless to define it as a "transfer", instead you'd have to define it as a "type identifier". Given I believe in the transfer method being optional, this would be very strange... It is one thing not to have an explicit transfer method, another to not have a type at all... How about we change the name from native model to native _representation_ (of the generic model)? Combined with appropriate wording change, I think this will have the effect you want. If you look at the models diagram you'll see it makes a lot of sense... I agree with Clark that mapping key order is out of the generic model. As for PHP, well, just like YAML doesn't reflect exactly Java's Vector vs. array, or Python's list vs. tuple, PHP will have to make a decision whenever it dumps a data structure. BTW, JavaScript has the same problem as PHP. It also has a unified collection data type. Have fun, Oren Ben-Kiki |
From: Oren Ben-K. <or...@ri...> - 2002-09-08 16:44:41
|
Steve Howell [mailto:sh...@zi...] wrote: > I really believe it's time for the information model section > to be moved to another document. I'm with Clark here; it belongs in the core and we need to get it right. > I believe Mike Orr would agree with me here, based on some > criticism he made to me of the spec in a private email. > Basically, Mike just agrees with me, and some other folks on > the list, that the spec is way too long for such a simple language. The spec is shorter than XMLs by a good degree... If the info model gives one headache, he's invited to skip over to the syntax, but at least when he tries to understand what it all means it is right there in the spec/ Have fun, Oren Ben-Kiki |
From: Brian I. <in...@tt...> - 2002-09-08 23:31:02
|
On 08/09/02 19:45 +0300, Oren Ben-Kiki wrote: > Steve Howell [mailto:sh...@zi...] wrote: > > I really believe it's time for the information model section > > to be moved to another document. > > I'm with Clark here; it belongs in the core and we need to get it right. Agreed to a point. It would be great to have it in the spec. On the other hand we've spent 16+ months working basically on the syntax. It's time to freeze that. I think it will take much longer to define an acceptable information model for all use cases. This is just an inherent problem. The info model grows from usage and language acceptance and implementations. It evolves. I just don't think we can do the big design up front here. Perhaps Steve is right but maybe not for the right reasons. We need to complete milestones. It would be nice to publish the results. Neil can/will move forward at that point. We can learn and polish our info model. It's bootstrapping, and Yes, it's hard. > > I believe Mike Orr would agree with me here, based on some > > criticism he made to me of the spec in a private email. > > Basically, Mike just agrees with me, and some other folks on > > the list, that the spec is way too long for such a simple language. YAML is a nice language. It makes the simple things simple and the hard things possible. That means it's NOT a simple language spec-wise. Where do you want us to put the hard stuff? > The spec is shorter than XMLs by a good degree... If the info model gives > one headache, he's invited to skip over to the syntax, but at least when he > tries to understand what it all means it is right there in the spec/ The info model gives *me* headaches. I'd like to believe that it's because it isn't clear/concise, and perhaps not completely thought through for all usages. Length is not the issue here. It's scope and progress. Cheers, Brian |
From: Steve H. <sh...@zi...> - 2002-09-09 00:30:33
|
----- Original Message ----- From: "Brian Ingerson" <in...@tt...> > On 08/09/02 19:45 +0300, Oren Ben-Kiki wrote: > > Steve Howell [mailto:sh...@zi...] wrote: > > > I really believe it's time for the information model section > > > to be moved to another document. > > > > I'm with Clark here; it belongs in the core and we need to get it right. > > Agreed to a point. It would be great to have it in the spec. On the other > hand we've spent 16+ months working basically on the syntax. It's time > to freeze that. I think it will take much longer to define an acceptable > information model for all use cases. > > This is just an inherent problem. The info model grows from usage and > language acceptance and implementations. It evolves. I just don't think > we can do the big design up front here. > > Perhaps Steve is right but maybe not for the right reasons. We need to > complete milestones. It would be nice to publish the results. Neil can/will > move forward at that point. We can learn and polish our info model. It's > bootstrapping, and Yes, it's hard. > There are three good reasons to move the info model section of the spec into a separate document. 1) If we had a document that focused primarily on syntax issues, we could freeze it fairly coldly. 2) The new document on the info model can grow, as we get into the issues that really require those concepts to be understood. 3) The info model makes people's head spin. Clark and Oren don't need to buy into all three reasons, though. I doubt that Clark will ever acknowledge that the info model makes people's heads spin, no matter how many people complain of head-spinning. Human nature. If the info model is so coupled to the rest of the spec that it can't be put in a separate document, then I wonder how that bodes for YAML. Quite frankly, the info model is starting to smell of overengineering. I am starting to observe the recurring "No, you can't do that because of the info model" smell. I still defer to Oren and Clark for the final decision on this, but I am reiterating my arguments, because I think Brian added to my case. Cheers, Steve |
From: Clark C. E. <cc...@cl...> - 2002-09-09 01:05:01
|
On Sun, Sep 08, 2002 at 08:29:48PM -0400, Steve Howell wrote: | 1) If we had a document that focused primarily on syntax issues, | we could freeze it fairly coldly. You can freeze the model fairly coldly. It needs a more clear/concise explanation. I could use your help here. | 2) The new document on the info model can grow, as we get into the issues that | really require those concepts to be understood. XML did this. It is one of the worst decisions they made and it is the primary cause for all the confusion and ickyness of their generic toolset. | 3) The info model makes people's head spin. Well... stop whining; help fix it. Clark |
From: Steve H. <sh...@zi...> - 2002-09-09 01:16:47
|
> | 2) The new document on the info model can grow, as we get into the issues that > | really require those concepts to be understood. > > XML did this. It is one of the worst decisions they made and > it is the primary cause for all the confusion and ickyness > of their generic toolset. > XML was wildly popular. As with most popular pieces of software, all kinds of incompatible solutions were built on top of it. Let's not make the same mistake in YAML. Let's make YAML wildly unpopular. Cheers, Steve |
From: Tom S. <tra...@tr...> - 2002-09-09 06:56:18
|
On Sun, 2002-09-08 at 17:30, Brian Ingerson wrote: > The info model gives *me* headaches. I'd like to believe that it's > because it isn't clear/concise, and perhaps not completely thought > through for all usages. > > Length is not the issue here. It's scope and progress. its poorly written honestly, not be mean or anything, but that's my take. it sufferes from "techno-talk", the use of words with specific meanings (at least i hope they have very specific meanings, or it's really bad!) used without ever given specific clearification. the result of this is that when read, much of the samantic is lost on the reader, and so must be compensated for by re-reading. as for length, well the spec has to be fully laid out, so whatever it takes to do so is what it takes. if it seems long it is probably because the spec itself needs simplification. by the way 16 months is a small time to give for a technology that could potentailly be used by many thousands for many decades. don't rush. take the extra time to get it right. -- tom sawyer, aka transami tra...@tr... |
From: Brian I. <in...@tt...> - 2002-09-09 07:23:48
|
On 09/09/02 01:06 -0600, Tom Sawyer wrote: > On Sun, 2002-09-08 at 17:30, Brian Ingerson wrote: > > The info model gives *me* headaches. I'd like to believe that it's > > because it isn't clear/concise, and perhaps not completely thought > > through for all usages. > > > > Length is not the issue here. It's scope and progress. > > its poorly written honestly, not be mean or anything, but that's my > take. > > it sufferes from "techno-talk", the use of words with specific meanings > (at least i hope they have very specific meanings, or it's really bad!) > used without ever given specific clearification. > > the result of this is that when read, much of the samantic is lost on > the reader, and so must be compensated for by re-reading. Agreed. I think you'll see a whole new section soon. > as for length, well the spec has to be fully laid out, so whatever it > takes to do so is what it takes. if it seems long it is probably because > the spec itself needs simplification. > > by the way 16 months is a small time to give for a technology that could > potentailly be used by many thousands for many decades. don't rush. take > the extra time to get it right. I completely agree. I never get discouraged, because i know we are making good progress. Thanks for the affirmation though :) Cheers, Brian |
From: Brian I. <in...@tt...> - 2002-09-08 22:09:47
|
On 08/09/02 18:42 +0300, Oren Ben-Kiki wrote: > With regard to the models diagram and the need for a native model... > > I don't think you can do away with mentioning the native model. After all > the transfer method is the instruction on how to do the conversion to the > native model; if we'd stop at the generic model it would be rather senseless > to define it as a "transfer", instead you'd have to define it as a "type > identifier". Given I believe in the transfer method being optional, this > would be very strange... It is one thing not to have an explicit transfer > method, another to not have a type at all... > > How about we change the name from native model to native _representation_ > (of the generic model)? Combined with appropriate wording change, I think > this will have the effect you want. If you look at the models diagram you'll > see it makes a lot of sense... +1 A native representation exists for most languages. I don't think it has to though. A loader could populate a completely contrived generic model using whatever it could scrape together. And them it could do YPATH or whatever. A hashless language may choose to use YAML for use cases that don't need a native representation. > I agree with Clark that mapping key order is out of the generic model. As > for PHP, well, just like YAML doesn't reflect exactly Java's Vector vs. > array, or Python's list vs. tuple, PHP will have to make a decision whenever > it dumps a data structure. BTW, JavaScript has the same problem as PHP. It > also has a unified collection data type. -0 I have a lot to say here. I really don't think this issue can be resolved in mail. Nor irc. In person maybe but you need all players. I think it's time to turn to our wiki. I'd really like to see Clark and Oren start using this tool. It's the only decent way I can think of to organize the reams of information and perspectives that need to be considered in order for us to resolve what lays in front of us. There is so much! Let me give you some examples: - Programming language internals: > We need to start looking really hard at the internal properties of all the languages that we support. What are their strengths and weaknesses. I'm just talking about the parts that apply to YAML. I'm hoping that an expert in each language will create a wiki synopsis of that language. Php, Tcl, Ksh, Lisp. We will excel at Perl/Python/Ruby. But we could improve these other languages as well. - Use cases: > We need to know what are use cases are. At least in general. And we need to weigh these properties into our decisions. Different use cases have completely different needs. I feel that Clark (and Oren to a lesser degree) are generally considering the YAML as data interchange between disseparate processing systems use case. That's important. What about YAML as an inernals dumper use case. We don't talk about that one much. When Steve and I used YAML at our last job, that was 95% of our usage. It rocked for that purpose. Some use cases need only a dumper, some only a loader. Some want to roundtrip YNY, some NYN. This is huge. Some want a *only* generic model, and native is totally unnecessary. Some want YPATH. *MOST* don't. Isn't that funny? - YAML conponents: > Can everyone rattle off from memory all the attributes of a YAML serialization. Which ones are important where? Are there ones we haven't thought of? Are you sure? - Information Models: > Where can I find a chart of all the properties surrounding any information model. Perhaps the wiki would be a nice place. - Etc: etc As far as preserving key order, what should we do? How the hell should I know without knowing all of the above and more? I can definitely describe situations where key order would be nice to preserve in the generic model. Just because YPATH doesn't need it, doesn't mean a thing. There are a lot of times when it would be nice. Think about the first YAML document ever. Clark's famous invoice for basketball supplies. It's all in a nice order. Wouldn't it be nice to load it into a generic layer (without going native) instead of having to change it through a (currently non-existent) serial interface. It begs to be loaded. But preserving that information makes Clark scream and Oren protest. They are smart people. That I know. I just don't have the background information of what they are thinking loaded into my telepathic cranial RAM all the time. I need a concise pool of thoughts to date, to review. Oren talked about making a little sign that talked about considering schema-aware vs schema-blind issues at evry step. Perhaps Oren could place this sign on our wiki. And we need a lot more signs than that! Cheers, Brian PS FWIW I've barely participated in wiki forums, so I'm not advocating this as my pet medium. Quite the contrary. I have to learn this art from scratch. But I have witnessed the power of a well maintained wiki and I think we're at the critical point where email is just not going to cut it for everything. Steve Howell has a lot of experience here. He'll be glad to assist. Right Steve? :) > > Have fun, > > Oren Ben-Kiki > > > ------------------------------------------------------- > This sf.net email is sponsored by: OSDN - Tired of that same old > cell phone? Get a new here for FREE! > https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390 > _______________________________________________ > Yaml-core mailing list > Yam...@li... > https://lists.sourceforge.net/lists/listinfo/yaml-core |
From: Clark C. E. <cc...@cl...> - 2002-09-08 23:39:30
|
On Sun, Sep 08, 2002 at 03:09:40PM -0700, Brian Ingerson wrote: | A native representation exists for most languages. I don't think it has to | though. A loader could populate a completely contrived generic model using | whatever it could scrape together. And this would be your native representation; that which is not an abstract model where the stuff really happens. | And them it could do YPATH or whatever. Yes, but I can use Ypath directly on a native Python structure *without* having a shadow object. This is the goal of the generic model, finding a LCD which describes most programming languages so that schema, path expressions, transformations, etc. can be defined on the _native_ representation of the language at hand. Admittedly it won't fit some languages... but it will work with most. | I think it's time to turn to our wiki. I'd really like to see Clark and Oren | start using this tool. It's the only decent way I can think of to organize | the reams of information and perspectives that need to be considered in order | for us to resolve what lays in front of us. Ok. Let's put yaml on HOLD at the current spec for about 12 months till YPATH and a Transform language are done. | - Programming language internals: > | We need to start looking really hard at the internal properties of all | the languages that we support. What are their strengths and weaknesses. | I'm just talking about the parts that apply to YAML. I'm hoping that an | expert in each language will create a wiki synopsis of that language. | Php, Tcl, Ksh, Lisp. We will excel at Perl/Python/Ruby. But we could | improve these other languages as well. Neat. | - Use cases: > | We need to know what are use cases are. At least in general. And we | need to weigh these properties into our decisions. Different use cases | have completely different needs. I feel that Clark (and Oren to a | lesser degree) are generally considering the YAML as data interchange | between disseparate processing systems use case. That's important. What | about YAML as an inernals dumper use case. We don't talk about that one | much. When Steve and I used YAML at our last job, that was 95% of our | usage. It rocked for that purpose. Some use cases need only a dumper, | some only a loader. Some want to roundtrip YNY, some NYN. This is huge. | Some want a *only* generic model, and native is totally unnecessary. | Some want YPATH. *MOST* don't. Isn't that funny? YPATH isn't mature yet. Seriously. The information model was laid out _before_ the syntax to meet the needs of a simple, stright-forward path and transform language. It's been this way for well over a year, the only signficnat change was adding type to it. And now, at "last call" it's all up for reconsideration? Let's give it a year or so, and if we really want to insist that all implementations must preserve key order then let's do it at that point... not now. | - YAML conponents: > | Can everyone rattle off from memory all the attributes of a YAML | serialization. Which ones are important where? Are there ones we | haven't thought of? Are you sure? Are we doing last call or a complete re-design? | - Information Models: > | Where can I find a chart of all the properties surrounding any | information model. Perhaps the wiki would be a nice place. Go for it. | As far as preserving key order, what should we do? How the hell should I know | without knowing all of the above and more? I can definitely describe | situations where key order would be nice to preserve in the generic model. | Just because YPATH doesn't need it, doesn't mean a thing. There are a lot of | times when it would be nice. Just beacuse YPATH doesn't need it is *exactly* the point. | Think about the first YAML document ever. Clark's famous invoice for | basketball supplies. It's all in a nice order. Wouldn't it be nice to load it | into a generic layer (without going native) instead of having to change it | through a (currently non-existent) serial interface. It begs to be loaded. No. The whole point of this was to be able to load that invoice into Perl, Python, etc. _without_ having a Document Object Model. Using *NATIVE* data structures of common languages. | But preserving that information makes Clark scream and Oren protest. They are | smart people. That I know. I just don't have the background information of | what they are thinking loaded into my telepathic cranial RAM all the time. I | need a concise pool of thoughts to date, to review. Ok. It comes down to looking at yaml one of two ways: (a) as a syntax format, and (b) as a representation of stuff which fits a common model shared by most programming languages. A bulk of the problem is that people on this list choose to look at it in the former way... and this is counter-productive. If you look at it in this way you'd want to preserve throw-away comments, scalar styles, and many other things which are necessary aspects/features that we've added so that the serialization is human readable. They are not intrinsic to the shared information model itself. This shared information model is what is written as the "generic model". Best, Clark |
From: Brian I. <in...@tt...> - 2002-09-09 02:06:08
|
On 08/09/02 23:41 +0000, Clark C. Evans wrote: > On Sun, Sep 08, 2002 at 03:09:40PM -0700, Brian Ingerson wrote: > | A native representation exists for most languages. I don't think it has to > | though. A loader could populate a completely contrived generic model using > | whatever it could scrape together. > > And this would be your native representation; that which is not > an abstract model where the stuff really happens. Nope. A native representation is something that is familiar to non YAML-aware programmers of the language. What i describe above is a generic representation because it was contrived to support the requirements of the YAML Generic Information Model. > | And them it could do YPATH or whatever. > > Yes, but I can use Ypath directly on a native Python structure > *without* having a shadow object. This is the goal of the generic > model, finding a LCD which describes most programming languages > so that schema, path expressions, transformations, etc. can be > defined on the _native_ representation of the language at hand. > > Admittedly it won't fit some languages... but it will work with most. That's great for you Clark. Great for Python users. But... If you are using the native representation for ypath then it is really a generic representation at that point. Because ypath only works on generic representation. As a Python user you get it for free unless type uris are involved. Then you need your viewer. But you are pushing too hard because of your Pythonic advantage. Right now you can get away with not transforming native to generic (because they're equivalent) 90% of your use cases. So when I say something like let's put key order into the generic model, you freak out. Now you'd need a transform to use ypath where one wasn't previously required. Java on the other hand would almost always need the transform to generic. I'm not even that big on key order. It's just a concept that I want to give due diligence. Your attitude gives the feeling of you protecting your current implementation advantages rather than being open minded about a new concept and viewing it from all sides. That's bad. Admittedly, requiring key order in the generic model makes less sense when you are starting with native. You need to inject it with code or schema. But when you are coming from serial to generic, there is no sense in throwing key order away. Especially if you have no intention to go all the way to native. That's what Ned Konz was saying. CYATL-like programs that read YAML documents and transformed them into new YAML docs would never need to even go to native. So why on earth would they throw away key order? Please refer to the diagrams in my earlier post: Viewer is a Flawed Concept Cheers, Brian |
From: Clark C. E. <cc...@cl...> - 2002-09-09 02:58:44
|
On Sun, Sep 08, 2002 at 07:06:01PM -0700, Brian Ingerson wrote: | Nope. A native representation is something that is familiar to non YAML-aware | programmers of the language. What i describe above is a generic | representation because it was contrived to support the requirements of the | YAML Generic Information Model. Does my terminology patch in the previous e-mail help? generic -> a wrapper or shadowed or non-builtin object used to support yaml functionality native -> a buildin object without any extra yaml shadows/wrappers models: graph: the highest level (collectinos/scalars in random access) serial: graph flattened via serial events syntax: as spec | If you are using the native representation for ypath then it is really a | generic representation at that point. Because ypath only works on generic | representation. Right. The ypath implementation would be dynamically converting the native object to its generic view... | As a Python user you get it for free unless type uris are involved. Then you | need your viewer. But you are pushing too hard because of your Pythonic | advantage. Right now you can get away with not transforming native to generic | (because they're equivalent) 90% of your use cases. Nods. When I need wrappers (generic objects) its much harder. | So when I say something like let's put key order into the generic model, | you freak out. Now you'd need a transform to use ypath where one wasn't | previously required. Well, if you want to put key order in your "generic" wrapper, this is just fine. What I'm saying is that key order should stay in the serial model. There will have to be two versions of YPath anyway, one to deal with serialized stuff and another one that works at the graph level. And there may be some mixing, just as I can see a schema with "graph" constraints, "serial" constraints, and "syntax" constraints. ;) | I'm not even that big on key order. It's just a concept that I want to | give due diligence. Your attitude gives the feeling of you protecting | your current implementation advantages rather than being open minded | about a new concept and viewing it from all sides. That's bad. Agreed. I've worked though alot of the stuff assuming the generic model and is very clean. Having to revisit this to add key order would complicate things (IMHO unnecessarly). That said, I think additions to YPATH to deal with Key Order will probably be needed as a schema may want to apply a few constraints about how the keys should be ordered when the graph is sequenced. | Admittedly, requiring key order in the generic model makes less sense when | you are starting with native. You need to inject it with code or schema. But | when you are coming from serial to generic, there is no sense in throwing key | order away. Especially if you have no intention to go all the way to native. | That's what Ned Konz was saying. I think the separation of "generic" from "graph" model has made things quite clear for me. | CYATL-like programs that read YAML documents and transformed them into new | YAML docs would never need to even go to native. So why on earth would they | throw away key order? Perhaps. And perhaps a few extensions / limitations will be needed for a CYATL which is serial instead of graph based. Thank you so much for helping to work on the issues with your last post. Its hard to defend against attacks that don't try to come at least half way... you did more than half way. Best, Clark |
From: Brian I. <in...@tt...> - 2002-09-09 02:35:49
|
On 08/09/02 23:41 +0000, Clark C. Evans wrote: > Ok. Let's put yaml on HOLD at the current spec for about 12 months > till YPATH and a Transform language are done. What is of interest to me is who is driving the need for these tools besides Clark, Oren. It has always smelled of XML envy to me. I have no use for these tools per se. A show of hands please for who wants an XPATH and XSLT replacement for YAML. > | - YAML conponents: > > | Can everyone rattle off from memory all the attributes of a YAML > | serialization. Which ones are important where? Are there ones we > | haven't thought of? Are you sure? > > Are we doing last call or a complete re-design? It's not a complete redesign. It's three steps forward, 2 steps back. We always have made progress. It's actually amazing how far we've gotten given our resources. Can you imagine what would happen if we could get the core people in the same room for even a few days? > > | - Information Models: > > | Where can I find a chart of all the properties surrounding any > | information model. Perhaps the wiki would be a nice place. > > Go for it. OK > | As far as preserving key order, what should we do? How the hell should I know > | without knowing all of the above and more? I can definitely describe > | situations where key order would be nice to preserve in the generic model. > | Just because YPATH doesn't need it, doesn't mean a thing. There are a lot of > | times when it would be nice. > > Just beacuse YPATH doesn't need it is *exactly* the point. *Boggles* Could you be more explicit on whatever point it is you're trying to make. > | Think about the first YAML document ever. Clark's famous invoice for > | basketball supplies. It's all in a nice order. Wouldn't it be nice to load it > | into a generic layer (without going native) instead of having to change it > | through a (currently non-existent) serial interface. It begs to be loaded. > > No. The whole point of this was to be able to load that invoice > into Perl, Python, etc. _without_ having a Document Object Model. > Using *NATIVE* data structures of common languages. OK. That was part of the reason, sure. So why do you make the thing pretty printed then? You should just put the keys in alpha or random order. You give the first time audience the feeling that you can control key order. > | But preserving that information makes Clark scream and Oren protest. They are > | smart people. That I know. I just don't have the background information of > | what they are thinking loaded into my telepathic cranial RAM all the time. I > | need a concise pool of thoughts to date, to review. > > Ok. It comes down to looking at yaml one of two ways: (a) as a > syntax format, and (b) as a representation of stuff which fits > a common model shared by most programming languages. > > A bulk of the problem is that people on this list choose to > look at it in the former way... and this is counter-productive. No it's not. It *is* a wonderful syntax format. And it *takes advantage* of those modern languages as much as possible. People are going to latch onto YAML for syntactic superiority first. Most Ruby people couldn't give a rat's ass about interoperability with Perl. > If you look at it in this way you'd want to preserve throw-away > comments, scalar styles, and many other things which are necessary > aspects/features that we've added so that the serialization is > human readable. They are not intrinsic to the shared information > model itself. This shared information model is what is written > as the "generic model". Well we need to have this discussion under a clear light. Everything on the table. I was going to suggest roundtripping scalar style next. :) Cheers, Brian |
From: Brian I. <in...@tt...> - 2002-09-09 01:40:06
|
On 08/09/02 18:42 +0300, Oren Ben-Kiki wrote: > With regard to the models diagram and the need for a native model... > > I don't think you can do away with mentioning the native model. After all > the transfer method is the instruction on how to do the conversion to the > native model; if we'd stop at the generic model... Stop! Look what you said! This is a huge statement! One that I've been trying to get through to Clark! What you've implied is that you go *through* the generic model to get to the native model. And the transfer method is how you get there. I AGREE 100%!! The notion of the viewer is flawed! (And thus the diagram!) It assumes that you need to get to native *before* you can get to generic. That's wrong. If you got from syntax to native without going generic, then it was purely an optimization. You probably didn't have any transfer methods in the syntax either, because then you would have realized the need or at least the implicit existence of the generic level. The basic problem is that you (Clark) see the generic level as only a view of the native. It definitely is from the N2Y side of things. But what Oren implicitly says above is that native comes from generic in the Y2N side. And that's what is missing in our diagram. You need to name a process that is the inverse of the viewer. The generic to native piece. The viewer calls a classes' to_yaml to get a generic node and a type uri. What is the piece that takes a serializations generic node and type uri and calls a classes' from_yaml to create a native node?? Here is the current flawed data flow in YNY: SYNTAX (parser) SERIAL (loader) NATIVE <-> application (viewer) GENERIC <-> application (ypath?) (dumper) SERIAL (emitter) SYNTAX I would propose that a "loader" has potentially two phases/modes. Generic loading and native loading. - It can load serial events to a generic model. Period. - It can look up (from_yaml) methods based on the transfer method URIs (if any) and turn those nodes into native, thereby creating the native repesentation graph. NOTE: If no transfer methods exist in the serialization then in some languages (but not necessarily all) the generic model might be the same as the native model. This is what would happen in a perfect world, but language deficiencies simply do not make it ubiquitous. It is very important that the the Ruby and Python (and to some degree Perl) people remember that even though the generic and native representations of simple data might be equivalent, they are not the same concept. ie IMHO you need to go through generic (at least conceptually) to get to native, even when they are the same. The "dumper" is the opposite. This is where your viewer logic lives. Here is what it should be (in YNY): SYNTAX (parser) SERIAL (generic loader) GENERIC (native loader) NATIVE <-> (application) (native dumper) GENERIC (generic dumper) SERIAL (emitter) SYNTAX See the exact symetry? Now for YPATH, we can just omit the native load: SYNTAX (parser) SERIAL (generic loader) GENERIC <-> (application - ypath) (generic dumper) SERIAL (emitter) SYNTAX And for true streaming applications: SYNTAX (parser) SERIAL <-> (application) (emitter) SYNTAX --- But that's only YNY. What about NYN? Here's the full roundtrip: NATIVE <-> (application) (native dumper) GENERIC (generic dumper) SERIAL (emitter) SYNTAX (parser) SERIAL (generic loader) GENERIC (native loader) NATIVE <-> (application) This is how you'd use ypath without ever even using the serialization. NATIVE <-> (application) (native dumper) aka viewer GENERIC <-> (application - ypath) (native loader) NATIVE <-> (application) Cheers, Brian |
From: Clark C. E. <cc...@cl...> - 2002-09-09 02:43:21
|
Brian, I think I agree with all of this; although perhaps with some slight wording changes (and a separation of concepts). Although, I think "generic" is unfortunate. Can we go back to the word "graph" to mean the highest model (graph/serial/syntax) instead of generic. Then, we can use "native" to mean a out-of-the-box language representation (non-wrapper) and "generic" to mean a language representation which exactly models one or all of the models, thus generic/graph, generic/serial. | What you've implied is that you go *through* the generic model to get to the | native model. And the transfer method is how you get there. I AGREE 100%!! Ok. Now, the question is how would you get an implicit transfer method? I propose that somehow that has to happen as one goes from the syntax to the serial model (by a regex or a serial-ypath expression). The parser would be registered with a bunch of built-in transfers so that stuff in the parenthesis would work out-of-the box. And perhaps a #IMPLICIT directive could cause the parser to attempt a download of an set of implicits from yaml.org (if they an't already cashed). In this way, the only transfers which use implicits (and stay in our model framework) are downloadable from yaml.org in a type registry. We then set a rule that they never change to enable caching for as long as necessary. | The notion of the viewer is flawed! (And thus the diagram!) It assumes that | you need to get to native *before* you can get to generic. That's wrong. If | you got from syntax to native without going generic, then it was purely an | optimization. You probably didn't have any transfer methods in the syntax | either, because then you would have realized the need or at least the | implicit existence of the generic level. Yes. At one time the spec had the statement: The native binding should only be constructed from information found in the generic model. | The basic problem is that you (Clark) see the generic level as only a | view of the native. It definitely is from the N2Y side of things. But | what Oren implicitly says above is that native comes from generic in the | Y2N side. And that's what is missing in our diagram. You need to name a | process that is the inverse of the viewer. The generic to native piece. I think that Oren's note is that when constructing a representation of a graph-model you need to use some sort of object that comes with the system. And this is what Oren views as the "native" object. Using the definition above, where "native" is a non-wrapper representation, this may or may not be true as you state below... | The viewer calls a classes' to_yaml to get a generic node and a type uri. | What is the piece that takes a serializations generic node and type uri | and calls a classes' from_yaml to create a native node?? I'd say that the "Viewer" doesn't really exist, and that only the Loader/Dumper really are there. The diagram would then go... SYNTAX (parser) SERIAL (loader) GRAPH (/w NATIVE binding) (dumper) SERIAL (emitter) SYNTAX Or to convert from NATIVE to GENERIC. GRAPH (/w NATIVE binding) (dumper) SERIAL (loader) GRAPH (/w GENERIC binding) In this way of thinking, if the Native binding doesn't support a type, then you can't go from SERIAL to GRAPH (as NATIVE) without ignoring that type; but you could go from SERIAL to GRAPH (as GENERIC). | I would propose that a "loader" has potentially two phases/modes. Generic | loading and native loading. | | - It can load serial events to a generic model. Period. | - It can look up (from_yaml) methods based on the transfer method URIs (if | any) and turn those nodes into native, thereby creating the native | repesentation graph. Exactly! Distinction between GENERIC and NATIVE binding. Where both GENERIC and NATIVE binding reflect the GRAPH model. | NOTE: If no transfer methods exist in the serialization then in some | languages (but not necessarily all) the generic model might be the | same as the native model. This is what would happen in a perfect | world, but language deficiencies simply do not make it ubiquitous. It | is very important that the the Ruby and Python (and to some degree | Perl) people remember that even though the generic and native | representations of simple data might be equivalent, they are not the | same concept. ie IMHO you need to go through generic (at least | conceptually) to get to native, even when they are the same. I'm quite sure that we're on the same page here. Mixing GENERIC and GRAPH was the problem. You've cleared it up above, with the loader having two phases/modes (going to one of two representations) In that GENERIC and NATIVE both can be seen *abstractly* as supporting the GRAPH model is the concept Oren and I were trying to achieve when we did the "viewer" thingy. | Now for YPATH, we can just omit the native load: | | SYNTAX | (parser) | SERIAL | (generic loader) | GENERIC <-> (application - ypath) | (generic dumper) | SERIAL | (emitter) | SYNTAX Right. Or the conversion could happen via a wrapper... | And for true streaming applications: | | SYNTAX | (parser) | SERIAL <-> (application) | (emitter) | SYNTAX | Yep. And YPath has two variants; one which can use the full "graph" and another which is limited to the single-pass serial model. | But that's only YNY. What about NYN? | | Here's the full roundtrip: | | NATIVE <-> (application) | (native dumper) | GENERIC | (generic dumper) | SERIAL | (emitter) | SYNTAX | (parser) | SERIAL | (generic loader) | GENERIC | (native loader) | NATIVE <-> (application) | | This is how you'd use ypath without ever even using the serialization. | | NATIVE <-> (application) | (native dumper) aka viewer | GENERIC <-> (application - ypath) | (native loader) | NATIVE <-> (application) Right. Clark |