YAML Ain't Markup Language

On Apr 19, 2:16 pm, Dan Stromberg <dan.stromb...@...> wrote:
> We have a build process that uses YAML - pyyaml specifically, not
> pyyaml-legacy.
>
> When our automated processes put things in the YAML files, the YAML
> parsing works.  But every once in a while a human adds something to a
> YAML file, and sometimes these humans don't add blanks after the colons,
> and then the process blows up.
>
> I've been asked to come up with a YAML parser that will be more lenient
> about not having a blank after a colon.  EG, right now, this doesn't
> work for us:
>
> invoice: 34843
> date   : 2001-01-23
> bill-to: &id001
>      given  :Chris
>      family : Dumars
>
> ...because "given  :Chris" has no space after the colon.
>
> So I have some questions:
> 1) Is there already a YAML parser that relaxes this?
> 2) How can I tell what version of pyyaml we are using?  I see the module
> names match those of pyyaml, but when grep'ing those modules for
> "version", I don't see anything that looks like a version string
> 3) I see that pyyaml's trunk has a "tests" directory - how would I
> invoke these?  Can anyone comment on how comprehensive these tests are?
> 4) Does pyyaml currently insist on a blank after a colon for a good
> reason?  Would eliminating the requirement introduce an ambiguity?

Maybe a simple pre-parser suffice?

On 04/19/2010 11:44 AM, Trans wrote:
> On Apr 19, 2:16 pm, Dan Stromberg<dan.stromb...@...>  wrote:
>    
>> We have a build process that uses YAML - pyyaml specifically, not
>> pyyaml-legacy.
>>
>> When our automated processes put things in the YAML files, the YAML
>> parsing works.  But every once in a while a human adds something to a
>> YAML file, and sometimes these humans don't add blanks after the colons,
>> and then the process blows up.
>>
>> I've been asked to come up with a YAML parser that will be more lenient
>> about not having a blank after a colon.  EG, right now, this doesn't
>> work for us:
>>
>> invoice: 34843
>> date   : 2001-01-23
>> bill-to:&id001
>>       given  :Chris
>>       family : Dumars
>>
>> ...because "given  :Chris" has no space after the colon.
>>
>> So I have some questions:
>> 1) Is there already a YAML parser that relaxes this?
>> 2) How can I tell what version of pyyaml we are using?  I see the module
>> names match those of pyyaml, but when grep'ing those modules for
>> "version", I don't see anything that looks like a version string
>> 3) I see that pyyaml's trunk has a "tests" directory - how would I
>> invoke these?  Can anyone comment on how comprehensive these tests are?
>> 4) Does pyyaml currently insist on a blank after a colon for a good
>> reason?  Would eliminating the requirement introduce an ambiguity?
>>      
> Maybe a simple pre-parser suffice?
>    
I was thinking a sed -i might do it, but then I'm not quite sure where 
to insert the sed at this point.  Also, I have a feeling my management 
will prefer a change to the parser.

I found that you can run "python setup.py test" for part of my question 
#3 above.  It gives a C compile error, and then appears to continue 
testing using pure python code.  The final line of output is "TESTS: 
1268" - how does one interpret that?

Thanks!

On 04/19/2010 12:17 PM, Stromberg, Dan wrote:
> On 04/19/2010 11:44 AM, Trans wrote:
>    
>> On Apr 19, 2:16 pm, Dan Stromberg<dan.stromb...@...>   wrote:
>>
>>      
>>> We have a build process that uses YAML - pyyaml specifically, not
>>> pyyaml-legacy.
>>>
>>> When our automated processes put things in the YAML files, the YAML
>>> parsing works.  But every once in a while a human adds something to a
>>> YAML file, and sometimes these humans don't add blanks after the colons,
>>> and then the process blows up.
>>>
>>> I've been asked to come up with a YAML parser that will be more lenient
>>> about not having a blank after a colon.  EG, right now, this doesn't
>>> work for us:
>>>
>>> invoice: 34843
>>> date   : 2001-01-23
>>> bill-to:&id001
>>>        given  :Chris
>>>        family : Dumars
>>>
>>> ...because "given  :Chris" has no space after the colon.
>>>
>>> So I have some questions:
>>> 1) Is there already a YAML parser that relaxes this?
>>> 2) How can I tell what version of pyyaml we are using?  I see the module
>>> names match those of pyyaml, but when grep'ing those modules for
>>> "version", I don't see anything that looks like a version string
>>> 3) I see that pyyaml's trunk has a "tests" directory - how would I
>>> invoke these?  Can anyone comment on how comprehensive these tests are?
>>> 4) Does pyyaml currently insist on a blank after a colon for a good
>>> reason?  Would eliminating the requirement introduce an ambiguity?
>>>
>>>        
>> Maybe a simple pre-parser suffice?
>>
>>      
> I was thinking a sed -i might do it, but then I'm not quite sure where
> to insert the sed at this point.  Also, I have a feeling my management
> will prefer a change to the parser.
>
> I found that you can run "python setup.py test" for part of my question
> #3 above.  It gives a C compile error, and then appears to continue
> testing using pure python code.  The final line of output is "TESTS:
> 1268" - how does one interpret that?
>
> Thanks!
>    
It just occured to me - we could replace the file open with a 
subprocess.Popen - but I still suspect my management may favor a change 
to the python parser, assuming that won't introduce an ambiguity.

On 04/19/2010 12:22 PM, Stromberg, Dan wrote:
> On 04/19/2010 12:17 PM, Stromberg, Dan wrote:
>    
>> On 04/19/2010 11:44 AM, Trans wrote:
>>
>>      
>>> On Apr 19, 2:16 pm, Dan Stromberg<dan.stromb...@...>    wrote:
>>>
>>>
>>>        
>>>> We have a build process that uses YAML - pyyaml specifically, not
>>>> pyyaml-legacy.
>>>>
>>>> When our automated processes put things in the YAML files, the YAML
>>>> parsing works.  But every once in a while a human adds something to a
>>>> YAML file, and sometimes these humans don't add blanks after the colons,
>>>> and then the process blows up.
>>>>
>>>> I've been asked to come up with a YAML parser that will be more lenient
>>>> about not having a blank after a colon.  EG, right now, this doesn't
>>>> work for us:
>>>>
>>>> invoice: 34843
>>>> date   : 2001-01-23
>>>> bill-to:&id001
>>>>         given  :Chris
>>>>         family : Dumars
>>>>
>>>> ...because "given  :Chris" has no space after the colon.
>>>>
>>>> So I have some questions:
>>>> 1) Is there already a YAML parser that relaxes this?
>>>> 2) How can I tell what version of pyyaml we are using?  I see the module
>>>> names match those of pyyaml, but when grep'ing those modules for
>>>> "version", I don't see anything that looks like a version string
>>>> 3) I see that pyyaml's trunk has a "tests" directory - how would I
>>>> invoke these?  Can anyone comment on how comprehensive these tests are?
>>>> 4) Does pyyaml currently insist on a blank after a colon for a good
>>>> reason?  Would eliminating the requirement introduce an ambiguity?
>>>>
>>>>
>>>>          
>>> Maybe a simple pre-parser suffice?
>>>
>>>
>>>        
>> I was thinking a sed -i might do it, but then I'm not quite sure where
>> to insert the sed at this point.  Also, I have a feeling my management
>> will prefer a change to the parser.
>>
>> I found that you can run "python setup.py test" for part of my question
>> #3 above.  It gives a C compile error, and then appears to continue
>> testing using pure python code.  The final line of output is "TESTS:
>> 1268" - how does one interpret that?
>>
>> Thanks!
>>
>>      
> It just occured to me - we could replace the file open with a
> subprocess.Popen - but I still suspect my management may favor a change
> to the python parser, assuming that won't introduce an ambiguity.
>    

About question #3:

    I see that the README says the unit tests are comprehensive.

    Also, I intentionally introduced an error and then ran the tests,
    and got:

        TESTS: 1268
        ERRORS: 986

    ...which seems pretty self-explanatory now.

I'm still interested in answers to questions 1, 2, and 4 above.  And if 
someone has something further to add to #3, I certainly wouldn't be 
displeased ^)^

Thanks!

> 4) Does pyyaml currently insist on a blank after a colon for a good
reason? Would eliminating the requirement introduce an ambiguity?

>> I'm still interested in answers to questions 1, 2, and 4 above.  And if
someone has something further to add to #3, I certainly wouldn't be
displeased ^)^

My understanding is that the answer to 4. is, e.g.,

urls:
- http://www.google.com
- http://www.amazon.com

If you didn't insist on a blank after the colon, there would be
extra hashes/mappings in there.

-- 
Brad

Brad Baxter wrote:
> 
>  > 4) Does pyyaml currently insist on a blank after a colon for a good 
> reason? Would eliminating the requirement introduce an ambiguity?
>  
>  >> I'm still interested in answers to questions 1, 2, and 4 above.  And 
> if someone has something further to add to #3, I certainly wouldn't be 
> displeased ^)^
> 
> My understanding is that the answer to 4. is, e.g.,
> 
> urls:
> - http://www.google.com
> - http://www.amazon.com
> 
> If you didn't insist on a blank after the colon, there would be
> extra hashes/mappings in there.
> 
> -- 
> Brad

Making a space *before* the colon work as well should help. It would fix 
the example in the original post:

      given  :Chris

Trying this out I do have to say that the error messages are completely 
cryptic and it is VERY hard to figure out what YAML is trying to do or 
what you typed wrong. This may be a good sign that this design is 
incorrect and should be changed.