From: why t. l. s. <yam...@wh...> - 2003-03-14 07:04:59
|
citizens, Here's the deal. I became sick and delirious about a month ago. In my deranged state, I felt it imperative that I should recode YAML.rb's parser in C. I have just now come to my senses, incident to the powerful fumes of a passing onion truck. And yet, I am sitting before a new YAML parser that is as wide and wonderful as any of the world's finest onion trucks! It is like I am an Olympic diver who has developed a serious case of amnesia mid-pike. There is nothing I can do now. So I am please to present Syck, the new YAML parser/toy on the block. I have included Ruby, Python and PHP extensions, though the last two need some work. An OCaml extension is in progress as well. Download here: http://prdownloads.sourceforge.net/yaml4r/syck-0.08.tar.gz?download To compile the Ruby extension: cd ext/ruby/ ruby extconf.rb make sudo make install To load a document: >> require 'syck' >> Syck.load( <<EOY ) - one - two - three Much is done but much is left to be done. The point is: I'm at version 0.08. And it's time to start releasing early and releasing often. Please jump in and voice your support for YAML in the Ruby distribution! YAML.rb has only been a module for eight months and has seen widespread usage. Wouldn't you like to <require 'yaml'> with no worries? Best of luck. And I can feel my amnesia wearing off already. Time for a dive. _why |
From: Brian I. <in...@tt...> - 2003-03-14 19:48:32
|
On 14/03/03 00:16 -0700, why the lucky stiff wrote: > citizens, > > Here's the deal. I became sick and delirious about a month ago. In my > deranged state, I felt it imperative that I should recode YAML.rb's > parser in C. I have just now come to my senses, incident to the > powerful fumes of a passing onion truck. And yet, I am sitting before a > new YAML parser that is as wide and wonderful as any of the world's > finest onion trucks! > > It is like I am an Olympic diver who has developed a serious case of amnesia > mid-pike. There is nothing I can do now. > > So I am please to present Syck, the new YAML parser/toy on the block. > I have included Ruby, Python and PHP extensions, though the last two > need some work. An OCaml extension is in progress as well. I took a look at Syck's interface. So you do have your own hashing built in. It's interesting because Syck is more than a parser, but less than a loader. But you could easily build a push/pull wrapper over it, or build a loader out of it. I kinda like it. (Especially if it works!) I could actually see doing the Perl extension to Syck as YAML::Parser::Syck or YAML::Loader::Syck. Eventually I'd want both. I think I'll start with the Loader though. Good work, Why. I'll be your Perl guy on this. (Aren't I always :) Cheers, Brian PS This gives me a good reason to get back into Inline again. All my projects will some day achieve oneness. Ingytopia. Pass the Dr Bronner's. Cheers, Brian |
From: why t. l. s. <yam...@wh...> - 2003-03-14 20:09:46
|
On Friday 14 March 2003 12:48 pm, Brian Ingerson wrote: > It's interesting because Syck is more than a parser, but less than a > loader. But you could easily build a push/pull wrapper over it, or buil= d > a loader out of it. I kinda like it. (Especially if it works!) > You can turn implicit typing on or off as you parse, so it really strives= to=20 be both a parser and a loader, if you want it to. The parsing of implici= t=20 types is separate, so you can use that independantly as well. (In an XML= =20 parser, for instance. :D) > I could actually see doing the Perl extension to Syck as > YAML::Parser::Syck or YAML::Loader::Syck. Eventually I'd want both. I > think I'll start with the Loader though. > YAML.rb + Syck sports similiar fashion. YAML::Loader::Syck is the base c= lass=20 of YAML::Parser::Syck. A different handler is used in the loader and=20 implicits are turned off. _why |
From: Brian I. <in...@tt...> - 2003-03-14 21:54:01
|
On 14/03/03 13:20 -0700, why the lucky stiff wrote: > On Friday 14 March 2003 12:48 pm, Brian Ingerson wrote: > > It's interesting because Syck is more than a parser, but less than a > > loader. But you could easily build a push/pull wrapper over it, or build > > a loader out of it. I kinda like it. (Especially if it works!) > > > > You can turn implicit typing on or off as you parse, so it really strives to > be both a parser and a loader, if you want it to. The parsing of implicit > types is separate, so you can use that independantly as well. (In an XML > parser, for instance. :D) So how do you think that specific type handling is going to be cross-language? If you can fully load a !foo {} from Syck, there's a problem. Because it assuredly won't work from perl. Your syck node should contain a kind and a type. And the program using Syck should call the class method to create the foo object. I would argue against putting Ruby only features in Syck. Put those in a separate C library. > > > I could actually see doing the Perl extension to Syck as > > YAML::Parser::Syck or YAML::Loader::Syck. Eventually I'd want both. I > > think I'll start with the Loader though. > > > > YAML.rb + Syck sports similiar fashion. YAML::Loader::Syck is the base class > of YAML::Parser::Syck. A different handler is used in the loader and > implicits are turned off. Actually I do things a bit differently in Perl. A YAML::Parser is definitely not a subclass of YAML::Loader. A Loader object contains a parser object, which keeps them totally orthoganal. So my end user interface is something like this: use YAML; $y = YAML->new; $y->loader('YAML::Loader::Standard'); $y->loader->parser('YAML::Parser::Syck'); $y->loader->parser->string(<<END_YAML); --- foo: barney END_YAML $data = $y->loader->next; That's long for: use YAML; $YAML::Loader = 'YAML::Loader::Standard'; $YAML::Parser = 'YAML::Parser::Syck'; $data = Load <<END_YAML; --- foo: barney END_YAML The nice thing is that I can switch out any component. When this really pays is when end users want to create their own Loaders. Let me show you how this would happen in Perl: Say that a user needed a Loader that loaded scalars as strings except for dates and currency. Say they also wanted to load strings that looked like RGB values as color objects: This is what their entire Perl YAML Loader module would look like: package My::YAML::Loader; # set up inheritance use base 'YAML::Loader::Date', 'YAML::Loader::Currency', 'YAML::Loader::Standard'; # associate this class with certain implicit values sub yaml_match { my ($self, $node) = @_; # match strings like 'ffcc55' return ($node->value =~ /[0-9a-f]{6}/); } sub yaml_load { my ($self, $node) = @_; return Color->new($node->value); } 1; And they use it like this: use YAML; $YAML::Loader = 'My::YAML::Loader'; $data = Load <<END_YAML; --- red: FF0000 white: FFFFFF blue: 0000FF END_YAML I hope that gives a clearer indication of where I want to go with this. Cheers, Brian |
From: why t. l. s. <yam...@wh...> - 2003-03-14 22:45:46
|
On Friday 14 March 2003 02:54 pm, Brian Ingerson wrote: > > So how do you think that specific type handling is going to be > cross-language? If you can fully load a !foo {} from Syck, there's a > problem. Because it assuredly won't work from perl. Your syck node > should contain a kind and a type. And the program using Syck should cal= l > the class method to create the foo object. > A SyckNode does include a kind and type. [from syck.h] struct _syck_node { // Symbol table ID SYMID id;=20 // Underlying kind enum syck_kind_tag kind; // Fully qualified tag-uri for type char *type_id; // Anchor name char *anchor; union { ... } data; } If implicits are turned on then only the types from the central repositor= y are=20 tagged. - [ "12", ! "12", !foo "12" ] In the above example, you'd have both scalar nodes listed as kind=20 `syck_str_kind'. The nodes would be typed `tag:yaml.org,2002:str',=20 `tag:yaml.org,2002:int' and `tag:yaml.org,2002:foo', respectively. It's = an=20 optimization. One of the bottlenecks in YAML.rb has been using regular expressions to d= etect=20 implicits. > I would argue against putting Ruby only features in Syck. Put those in = a > separate C library. Absolutely. I have wondered about the possibility of having !syck/* type= s,=20 though. A user recently asked if Ruby symbols could be added to the cent= ral=20 type repository. I don't know if symbols belong in the central type=20 repository or in a repository shared by languages who use a similiar load= er. > Actually I do things a bit differently in Perl. A YAML::Parser is > definitely not a subclass of YAML::Loader. A Loader object contains a > parser object, which keeps them totally orthoganal. Hmm. Hmmmm. I'll think about this. I can see the advantages. My loade= r=20 classes are just shells that farm the loading off to callbacks registered= by=20 the user. I don't know what the implications are of having the loader be= come=20 so easily decoupled. If a user changes the loader, then who knows how=20 documents would be loaded. I guess that's the idea but I can see it bein= g=20 the source of bugs. require 'yaml' # Defaults to YAML::Loader::Syck require 'okay/rpc' require 'jackslib' # Switches to YAML::Loader::Jacks=20 # for a spell. # Are the !okay/rpc types registered with Jack's loader? orc =3D Okay::RPC::Client.new( 'whytheluckystiff.net', '/okayRpc/' ) p orc.call( 'system.listMethods' ) # User emails why... I'd prefer it if Jack subclassed the YAML::Parser and used his loader cla= ss=20 directly rather than affecting the loading of YAML throughout the system.= I=20 can see wanting to switch loaders out (Syck, libyaml, pure Ruby). Anyway, you're a BDUFer and bless you for it, BI. I'm a=20 see-you-at-the-finish-liner. This kid just walked into the room with a tae-kwon-do outfit. I'm going = to=20 see what he wants. _why |
From: Brian I. <in...@tt...> - 2003-03-15 02:27:35
|
On 14/03/03 15:57 -0700, why the lucky stiff wrote: > On Friday 14 March 2003 02:54 pm, Brian Ingerson wrote: > > > > So how do you think that specific type handling is going to be > > cross-language? If you can fully load a !foo {} from Syck, there's a > > problem. Because it assuredly won't work from perl. Your syck node > > should contain a kind and a type. And the program using Syck should call > > the class method to create the foo object. > > > > A SyckNode does include a kind and type. > > [from syck.h] > struct _syck_node { > // Symbol table ID > SYMID id; > // Underlying kind > enum syck_kind_tag kind; > // Fully qualified tag-uri for type > char *type_id; > // Anchor name > char *anchor; > union { ... } data; > } > > If implicits are turned on then only the types from the central > repository are tagged. cool. > - [ "12", ! "12", !foo "12" ] > > In the above example, you'd have both scalar nodes listed as kind > `syck_str_kind'. The nodes would be typed `tag:yaml.org,2002:str', > `tag:yaml.org,2002:int' and `tag:yaml.org,2002:foo', respectively. It's an > optimization. > > One of the bottlenecks in YAML.rb has been using regular expressions > to detect implicits. Interesting. I wouldn't have guessed this. At least in Perl, regexp operations are usually close to pure C. Not often the source of bottlenecks unless the regexps are poorly written. > > > I would argue against putting Ruby only features in Syck. Put those in a > > separate C library. > > Absolutely. I have wondered about the possibility of having !syck/* types, > though. A user recently asked if Ruby symbols could be added to the central > type repository. I don't know if symbols belong in the central type > repository or in a repository shared by languages who use a similiar loader. What are symbols? I see no reason for not putting lots of types in the repository. That's what it is there for. The common wisdom is that type checking is done at the loader level though. If your loader is designed for multiple inheritance, then you can simply add a new type class as new types are added, without have to change the parser or core loader. > > Actually I do things a bit differently in Perl. A YAML::Parser is > > definitely not a subclass of YAML::Loader. A Loader object contains a > > parser object, which keeps them totally orthoganal. > > Hmm. Hmmmm. I'll think about this. I can see the advantages. My loader > classes are just shells that farm the loading off to callbacks registered by > the user. I don't know what the implications are of having the loader become > so easily decoupled. If a user changes the loader, then who knows how > documents would be loaded. I guess that's the idea but I can see it being > the source of bugs. > > require 'yaml' # Defaults to YAML::Loader::Syck > require 'okay/rpc' > require 'jackslib' # Switches to YAML::Loader::Jacks > # for a spell. > > # Are the !okay/rpc types registered with Jack's loader? > orc = Okay::RPC::Client.new( 'whytheluckystiff.net', '/okayRpc/' ) > p orc.call( 'system.listMethods' ) > > # User emails why... > > I'd prefer it if Jack subclassed the YAML::Parser and used his loader class > directly rather than affecting the loading of YAML throughout the system. I > can see wanting to switch loaders out (Syck, libyaml, pure Ruby). I am not suggesting that requiring a loader class effect the whole system. That's ludicrous. Each YAML Load operation is done from a particular YAML::Loader object. That Loader object may inherit certain properties (like implicits) from various Loading classes, but it has one predictable beahviour. In fact if a load operation involved another load operation (like with !include processing) then a separate loader object would be used. And that separate object might be from a completely different class. > Anyway, you're a BDUFer and bless you for it, BI. I'm a > see-you-at-the-finish-liner. Silly programmer, there's no basement in the Alamo, er I mean, there's no finish line. Anyway, you and I are the only ones currently implementing YAML stuff, so we really shouldn't fight. It's all about peace and love. > This kid just walked into the room with a tae-kwon-do outfit. I'm going to > see what he wants. KICK HIS ASS. Cheers, Brian |
From: why t. l. s. <yam...@wh...> - 2003-03-15 03:06:19
|
Brian Ingerson (in...@tt...) wrote: > > > > One of the bottlenecks in YAML.rb has been using regular expressions > > to detect implicits. > > Interesting. I wouldn't have guessed this. At least in Perl, regexp > operations are usually close to pure C. Not often the source of > bottlenecks unless the regexps are poorly written. > See the problem is having to use the ten or twenty regexps that handle int, float, timestamps, nulls, nools, etc. The tokenizer in C can whittle that down to what can be thought of as a single expression. > > What are symbols? > A symbol is an entry in the symbol table represented by a name. In Ruby, symbols are designated by a colon followed by word characters. For sumbols, the following is true: :Fred.id == :Fred.id Whereas for strings: "Fred".id != "Fred".id (ID is the entry number in the symbol table.) Symbols are very efficient and often serialized. (In comparison to classes like Range and Regexp, which are serialized less often.) > > I see no reason for not putting lots of types in the repository. That's > what it is there for. The common wisdom is that type checking is done at > the loader level though. If your loader is designed for multiple > inheritance, then you can simply add a new type class as new types are > added, without have to change the parser or core loader. > Cool. > I am not suggesting that requiring a loader class effect the whole > system. That's ludicrous. Each YAML Load operation is done from a > particular YAML::Loader object. That Loader object may inherit certain > properties (like implicits) from various Loading classes, but it has one > predictable beahviour. > > In fact if a load operation involved another load operation (like with > !include processing) then a separate loader object would be used. And > that separate object might be from a completely different class. Ok. I see your angle. YAML.rb is similar. _why |