module-build-general

[Module::Build] broken META.yml file

[<and...@fr...>]

This seems to be a bug in Module::Build that is still there in 0.28.

Get the distro DJO/DBIx-Class-FormTools-0.000003.tar.gz, hide your
YAML from perl and run

  perl Build.PL
  ./Build distmeta

The resulting META.yml contains (among others) the following lines:

----snip----
abstract: Utility module for building forms with multiple
related L<DBIx::Class> objects.
----snip----

This cannot be parsed by YAML with this error message:

YAML Error: Invalid element in map
   Code: YAML_LOAD_ERR_BAD_MAP_ELEMENT
   Line: 7
   Document: 1



-- 
andreas

Re: [Module::Build] broken META.yml file

[<and...@fr...>]

>>>>> On Wed, 10 May 2006 08:40:33 +0200, and...@fr... (Andreas J. Koenig) said:

  > This seems to be a bug in Module::Build that is still there in 0.28.

Another case where Module::Build without YAML fails is PHILCROW/Gantry-3.30.tar.gz

M:B generates a YAML file that contains

----snip----
author:
  - Tim Keefer <tk...@gm...>
Phil Crow <phi...@ya...>
----snip----

and YAML complains about this one with

% perl -le 'use YAML 0.58; YAML::LoadFile("META.yml")'
YAML Error: Invalid element in map
   Code: YAML_LOAD_ERR_BAD_MAP_ELEMENT
   Line: 6
   Document: 1
 at /usr/local/perl-5.8.7/lib/site_perl/5.8.7/YAML.pm line 60


-- 
andreas

Re: [Module::Build] broken META.yml file

[Johan V. <jvr...@sq...>]

and...@fr... (Andreas J. Koenig) writes:

>>>>>> On Wed, 10 May 2006 08:40:33 +0200, and...@fr... (Andreas J. Koenig) said:
>
>   > This seems to be a bug in Module::Build that is still there in 0.28.
>
> Another case where Module::Build without YAML fails is PHILCROW/Gantry-3.30.tar.gz
>
> M:B generates a YAML file that contains
>
> ----snip----
> author:
>   - Tim Keefer <tk...@gm...>
> Phil Crow <phi...@ya...>
> ----snip----

In the current Module::Build::YAML, line 105, read:

  # XXX doesn't handle embedded newlines

Shouldn't be so hard to fix...

-- Johan

Re: [Module::Build] broken META.yml file

[Stephen A. <spa...@gm...>]

Hi,

I wrote Module::Build::YAML.
I offer these patches.

Both of the previously mentioned broken YAML files were a
result of embedded newlines.  I opted to tackle newlines and
all of the other special characters which I could find which
YAML considers special.

I think this version will be quite an improvement in terms of
robustness in the presence of "odd" characters in the scalar
values.  I patched

   lib/Module/Build/YAML.pm
   t/mbyaml.t

The "svn diff" is below. I also attach a tar.gz of the two full files in ca=
se
I did the patch wrong.

Stephen

spadkins@pompeii:/usr/rubicon/spadkins/src/app/Module-Build> svn diff
Index: t/mbyaml.t
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- t/mbyaml.t  (revision 6288)
+++ t/mbyaml.t  (working copy)
@@ -9,9 +9,12 @@
 $dir =3D "t" if (-d "t");

 {
-  use_ok("Module::Build::YAML");
-  my ($expected, $got, $var);
-  $var =3D {
+    use_ok("Module::Build::YAML");
+    my ($expected, $got, $var);
+    ##########################################################
+    # Test a typical-looking Module::Build structure (alphabetized)
+    ##########################################################
+    $var =3D {
           'resources' =3D> {
                            'license' =3D>
'http://opensource.org/licenses/artistic-license.php'
                          },
@@ -43,11 +46,11 @@
                         },
           'abstract' =3D> 'A framework for building dynamic widgets or
full applications in Javascript'
         };
-  $expected =3D <<EOF;
+    $expected =3D <<'EOF';
 ---
 abstract: A framework for building dynamic widgets or full
applications in Javascript
 author:
-  - '"Stephen Adkins" <spadkins\@gmail.com>'
+  - '"Stephen Adkins" <spa...@gm...>'
 build_requires:
   App::Build: 0
   File::Spec: 0
@@ -72,12 +75,15 @@
     $got =3D &Module::Build::YAML::Dump($var);
     is($got, $expected, "Dump(): single deep hash");

-  $expected =3D <<EOF;
+    ##########################################################
+    # Test a typical-looking Module::Build structure (ordered)
+    ##########################################################
+    $expected =3D <<'EOF';
 ---
 name: js-app
 version: 0.13
 author:
-  - '"Stephen Adkins" <spadkins\@gmail.com>'
+  - '"Stephen Adkins" <spa...@gm...>'
 abstract: A framework for building dynamic widgets or full
applications in Javascript
 license: lgpl
 resources:
@@ -102,13 +108,16 @@
     $got =3D &Module::Build::YAML::Dump($var);
     is($got, $expected, "Dump(): single deep hash, ordered");

+    ##########################################################
+    # Test that an array turns into multiple documents
+    ##########################################################
     $var =3D [
         "e",
         2.71828,
         [ "pi", "is", 3.1416 ],
         { fun =3D> "under_sun", 6 =3D> undef, "more", undef },
     ];
-    $expected =3D <<EOF;
+    $expected =3D <<'EOF';
 ---
 e
 ---
@@ -118,14 +127,17 @@
 - is
 - 3.1416
 ---
-6: ""
+6: ~
 fun: under_sun
-more: ""
+more: ~
 EOF
     $got =3D &Module::Build::YAML::Dump(@$var);
     is($got, $expected, "Dump(): multiple, various");

-    $expected =3D <<EOF;
+    ##########################################################
+    # Test that a single array ref turns into one document
+    ##########################################################
+    $expected =3D <<'EOF';
 ---
 - e
 - 2.71828
@@ -134,16 +146,115 @@
   - is
   - 3.1416
 -
-  6: ""
+  6: ~
   fun: under_sun
-  more: ""
+  more: ~
 EOF
     $got =3D &Module::Build::YAML::Dump($var);
     is($got, $expected, "Dump(): single array of various");

+    ##########################################################
+    # Test Object-Oriented Flavor of the API
+    ##########################################################
     my $y =3D Module::Build::YAML->new();
     $got =3D $y->Dump($var);
     is($got, $expected, "Dump(): single array of various (OO)");
+
+    ##########################################################
+    # Test Quoting Conditions (newlines, quotes, tildas, undefs)
+    ##########################################################
+    $var =3D {
+        'foo01' =3D> '`~!@#$%^&*()_+-=3D{}|[]\\;\':",./?<>
+<nl>',
+        'foo02' =3D> '~!@#$%^&*()_+-=3D{}|[]\\;:,./<>?',
+        'foo03' =3D> undef,
+        'foo04' =3D> '~',
+    };
+    $expected =3D <<'EOF';
+---
+foo01: "`~!@#$%^&*()_+-=3D{}|[]\;':\",./?<>\n<nl>"
+foo02: "~!@#$%^&*()_+-=3D{}|[]\;:,./<>?"
+foo03: ~
+foo04: "~"
+EOF
+    $got =3D &Module::Build::YAML::Dump($var);
+    is($got, $expected, "Dump(): tricky embedded characters");
+
+    $var =3D {
+        'foo10' =3D> undef,
+        'foo40' =3D> '!',
+        'foo41' =3D> '@',
+        'foo42' =3D> '#',
+        'foo43' =3D> '$',
+        'foo44' =3D> '%',
+        'foo45' =3D> '^',
+        'foo47' =3D> '&',
+        'foo48' =3D> '*',
+        'foo49' =3D> '(',
+        'foo50' =3D> ')',
+        'foo51' =3D> '_',
+        'foo52' =3D> '+',
+        'foo53' =3D> '-',
+        'foo54' =3D> '=3D',
+        'foo55' =3D> '{',
+        'foo56' =3D> '}',
+        'foo57' =3D> '|',
+        'foo58' =3D> '[',
+        'foo59' =3D> ']',
+        'foo60' =3D> '\\',
+        'foo61' =3D> ';',
+        'foo62' =3D> ':',
+        'foo63' =3D> ',',
+        'foo64' =3D> '.',
+        'foo65' =3D> '/',
+        'foo66' =3D> '<',
+        'foo67' =3D> '>',
+        'foo68' =3D> '?',
+        'foo69' =3D> '\'',
+        'foo70' =3D> '"',
+        'foo71' =3D> '`',
+        'foo72' =3D> '
+',
+    };
+    $expected =3D <<'EOF';
+---
+foo10: ~
+foo40: "!"
+foo41: '@'
+foo42: "#"
+foo43: $
+foo44: %
+foo45: "^"
+foo47: "&"
+foo48: "*"
+foo49: "("
+foo50: ")"
+foo51: _
+foo52: +
+foo53: -
+foo54: =3D
+foo55: "{"
+foo56: "}"
+foo57: "|"
+foo58: "["
+foo59: "]"
+foo60: \
+foo61: ;
+foo62: :
+foo63: ,
+foo64: .
+foo65: /
+foo66: '<'
+foo67: '>'
+foo68: "?"
+foo69: "'"
+foo70: '"'
+foo71: "`"
+foo72: "\n"
+EOF
+    $got =3D &Module::Build::YAML::Dump($var);
+    is($got, $expected, "Dump(): tricky embedded characters (singles)");
+
 }


Index: lib/Module/Build/YAML.pm
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- lib/Module/Build/YAML.pm    (revision 6288)
+++ lib/Module/Build/YAML.pm    (working copy)
@@ -102,26 +102,31 @@
 }

 sub _yaml_value {
-  # XXX doesn't handle embedded newlines
   my ($value) =3D @_;
-  # undefs and empty strings will become empty strings
-  if (! defined $value || $value eq "") {
+  # undefs become ~
+  if (! defined $value) {
+    return("~");
+  }
+  # empty strings will become empty strings
+  elsif (! defined $value || $value eq "") {
     return('""');
   }
+  # quote and escape strings with special values
+  elsif ($value =3D~ /["'`~\n!\@\#^\&\*\(\)\{\}\[\]\|<>\?]/) {
+    if ($value !~ /['`~\n!\#^\&\*\(\)\{\}\[\]\|\?]/) {  # nothing but
" or @ or < or > (email addresses)
+      return("'" . $value . "'");
+    }
+    else {
+      $value =3D~ s/\n/\\n/g;    # handle embedded newlines
+      $value =3D~ s/"/\\"/g;     # handle embedded quotes
+      return('"' . $value . '"');
+    }
+  }
   # allow simple scalars (without embedded quote chars) to be unquoted
-  elsif ($value !~ /["'\\]/) {
+  # (includes $%_+=3D-\;:,./)
+  else {
     return($value);
   }
-  # strings without double-quotes get double-quoted
-  elsif ($value !~ /\"/) {
-    $value =3D~ s{\\}{\\\\}g;
-    return qq{"$value"};
-  }
-  # other strings get single-quoted
-  else {
-    $value =3D~ s{([\\'])}{\\$1}g;
-    return qq{'$value'};
-  }
 }

 1;
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

On 5/11/06, Johan Vromans <jvr...@sq...> wrote:
> and...@fr... (Andreas J. Koenig) writes:
>
> >>>>>> On Wed, 10 May 2006 08:40:33 +0200, andreas.koenig.gmwojprw@franz.=
ak.mind.de (Andreas J. Koenig) said:
> >
> >   > This seems to be a bug in Module::Build that is still there in 0.28=
.
> >
> > Another case where Module::Build without YAML fails is PHILCROW/Gantry-=
3.30.tar.gz
> >
> > M:B generates a YAML file that contains
> >
> > ----snip----
> > author:
> >   - Tim Keefer <tk...@gm...>
> > Phil Crow <phi...@ya...>
> > ----snip----
>
> In the current Module::Build::YAML, line 105, read:
>
>   # XXX doesn't handle embedded newlines
>
> Shouldn't be so hard to fix...
>
> -- Johan
>
>
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job ea=
sier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronim=
o
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=
=3D121642
> _______________________________________________
> Module-build-general mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/module-build-general
>

Re: [Module::Build] broken META.yml file

[Johan V. <jvr...@sq...>]

"Stephen Adkins" <spa...@gm...> writes:

> I wrote Module::Build::YAML.
> I offer these patches.

Thanks!

For the sake of acceptance, I strongly suggest to release a 0.29
version ASAP.

Stephen, how about turning M::B::YAML into YAML::Light and let the
world benefit from it?

-- Johan

Re: [Module::Build] broken META.yml file

[Ken W. <ke...@ma...>]

Hi Stephen,

The patch looks good except that it got mangled a little in the email  
message.

I've applied it and all tests pass for me.  Thanks!

  -Ken

On May 11, 2006, at 11:50 PM, Stephen Adkins wrote:

> Hi,
>
> I wrote Module::Build::YAML.
> I offer these patches.
>
> Both of the previously mentioned broken YAML files were a
> result of embedded newlines.  I opted to tackle newlines and
> all of the other special characters which I could find which
> YAML considers special.
>
> I think this version will be quite an improvement in terms of
> robustness in the presence of "odd" characters in the scalar
> values.  I patched
>
>   lib/Module/Build/YAML.pm
>   t/mbyaml.t
>
> The "svn diff" is below. I also attach a tar.gz of the two full  
> files in case
> I did the patch wrong.
>
> Stephen
>
> spadkins@pompeii:/usr/rubicon/spadkins/src/app/Module-Build> svn diff
> Index: t/mbyaml.t
> ===================================================================
> --- t/mbyaml.t  (revision 6288)
> +++ t/mbyaml.t  (working copy)
> @@ -9,9 +9,12 @@
> $dir = "t" if (-d "t");
>
> {
> -  use_ok("Module::Build::YAML");
> -  my ($expected, $got, $var);
> -  $var = {
> +    use_ok("Module::Build::YAML");
> +    my ($expected, $got, $var);
> +    ##########################################################
> +    # Test a typical-looking Module::Build structure (alphabetized)
> +    ##########################################################
> +    $var = {
>           'resources' => {
>                            'license' =>
> 'http://opensource.org/licenses/artistic-license.php'
>                          },
> @@ -43,11 +46,11 @@
>                         },
>           'abstract' => 'A framework for building dynamic widgets or
> full applications in Javascript'
>         };
> -  $expected = <<EOF;
> +    $expected = <<'EOF';
> ---
> abstract: A framework for building dynamic widgets or full
> applications in Javascript
> author:
> -  - '"Stephen Adkins" <spadkins\@gmail.com>'
> +  - '"Stephen Adkins" <spa...@gm...>'
> build_requires:
>   App::Build: 0
>   File::Spec: 0
> @@ -72,12 +75,15 @@
>     $got = &Module::Build::YAML::Dump($var);
>     is($got, $expected, "Dump(): single deep hash");
>
> -  $expected = <<EOF;
> +    ##########################################################
> +    # Test a typical-looking Module::Build structure (ordered)
> +    ##########################################################
> +    $expected = <<'EOF';
> ---
> name: js-app
> version: 0.13
> author:
> -  - '"Stephen Adkins" <spadkins\@gmail.com>'
> +  - '"Stephen Adkins" <spa...@gm...>'
> abstract: A framework for building dynamic widgets or full
> applications in Javascript
> license: lgpl
> resources:
> @@ -102,13 +108,16 @@
>     $got = &Module::Build::YAML::Dump($var);
>     is($got, $expected, "Dump(): single deep hash, ordered");
>
> +    ##########################################################
> +    # Test that an array turns into multiple documents
> +    ##########################################################
>     $var = [
>         "e",
>         2.71828,
>         [ "pi", "is", 3.1416 ],
>         { fun => "under_sun", 6 => undef, "more", undef },
>     ];
> -    $expected = <<EOF;
> +    $expected = <<'EOF';
> ---
> e
> ---
> @@ -118,14 +127,17 @@
> - is
> - 3.1416
> ---
> -6: ""
> +6: ~
> fun: under_sun
> -more: ""
> +more: ~
> EOF
>     $got = &Module::Build::YAML::Dump(@$var);
>     is($got, $expected, "Dump(): multiple, various");
>
> -    $expected = <<EOF;
> +    ##########################################################
> +    # Test that a single array ref turns into one document
> +    ##########################################################
> +    $expected = <<'EOF';
> ---
> - e
> - 2.71828
> @@ -134,16 +146,115 @@
>   - is
>   - 3.1416
> -
> -  6: ""
> +  6: ~
>   fun: under_sun
> -  more: ""
> +  more: ~
> EOF
>     $got = &Module::Build::YAML::Dump($var);
>     is($got, $expected, "Dump(): single array of various");
>
> +    ##########################################################
> +    # Test Object-Oriented Flavor of the API
> +    ##########################################################
>     my $y = Module::Build::YAML->new();
>     $got = $y->Dump($var);
>     is($got, $expected, "Dump(): single array of various (OO)");
> +
> +    ##########################################################
> +    # Test Quoting Conditions (newlines, quotes, tildas, undefs)
> +    ##########################################################
> +    $var = {
> +        'foo01' => '`~!@#$%^&*()_+-={}|[]\\;\':",./?<>
> +<nl>',
> +        'foo02' => '~!@#$%^&*()_+-={}|[]\\;:,./<>?',
> +        'foo03' => undef,
> +        'foo04' => '~',
> +    };
> +    $expected = <<'EOF';
> +---
> +foo01: "`~!@#$%^&*()_+-={}|[]\;':\",./?<>\n<nl>"
> +foo02: "~!@#$%^&*()_+-={}|[]\;:,./<>?"
> +foo03: ~
> +foo04: "~"
> +EOF
> +    $got = &Module::Build::YAML::Dump($var);
> +    is($got, $expected, "Dump(): tricky embedded characters");
> +
> +    $var = {
> +        'foo10' => undef,
> +        'foo40' => '!',
> +        'foo41' => '@',
> +        'foo42' => '#',
> +        'foo43' => '$',
> +        'foo44' => '%',
> +        'foo45' => '^',
> +        'foo47' => '&',
> +        'foo48' => '*',
> +        'foo49' => '(',
> +        'foo50' => ')',
> +        'foo51' => '_',
> +        'foo52' => '+',
> +        'foo53' => '-',
> +        'foo54' => '=',
> +        'foo55' => '{',
> +        'foo56' => '}',
> +        'foo57' => '|',
> +        'foo58' => '[',
> +        'foo59' => ']',
> +        'foo60' => '\\',
> +        'foo61' => ';',
> +        'foo62' => ':',
> +        'foo63' => ',',
> +        'foo64' => '.',
> +        'foo65' => '/',
> +        'foo66' => '<',
> +        'foo67' => '>',
> +        'foo68' => '?',
> +        'foo69' => '\'',
> +        'foo70' => '"',
> +        'foo71' => '`',
> +        'foo72' => '
> +',
> +    };
> +    $expected = <<'EOF';
> +---
> +foo10: ~
> +foo40: "!"
> +foo41: '@'
> +foo42: "#"
> +foo43: $
> +foo44: %
> +foo45: "^"
> +foo47: "&"
> +foo48: "*"
> +foo49: "("
> +foo50: ")"
> +foo51: _
> +foo52: +
> +foo53: -
> +foo54: =
> +foo55: "{"
> +foo56: "}"
> +foo57: "|"
> +foo58: "["
> +foo59: "]"
> +foo60: \
> +foo61: ;
> +foo62: :
> +foo63: ,
> +foo64: .
> +foo65: /
> +foo66: '<'
> +foo67: '>'
> +foo68: "?"
> +foo69: "'"
> +foo70: '"'
> +foo71: "`"
> +foo72: "\n"
> +EOF
> +    $got = &Module::Build::YAML::Dump($var);
> +    is($got, $expected, "Dump(): tricky embedded characters  
> (singles)");
> +
> }
>
>
> Index: lib/Module/Build/YAML.pm
> ===================================================================
> --- lib/Module/Build/YAML.pm    (revision 6288)
> +++ lib/Module/Build/YAML.pm    (working copy)
> @@ -102,26 +102,31 @@
> }
>
> sub _yaml_value {
> -  # XXX doesn't handle embedded newlines
>   my ($value) = @_;
> -  # undefs and empty strings will become empty strings
> -  if (! defined $value || $value eq "") {
> +  # undefs become ~
> +  if (! defined $value) {
> +    return("~");
> +  }
> +  # empty strings will become empty strings
> +  elsif (! defined $value || $value eq "") {
>     return('""');
>   }
> +  # quote and escape strings with special values
> +  elsif ($value =~ /["'`~\n!\@\#^\&\*\(\)\{\}\[\]\|<>\?]/) {
> +    if ($value !~ /['`~\n!\#^\&\*\(\)\{\}\[\]\|\?]/) {  # nothing but
> " or @ or < or > (email addresses)
> +      return("'" . $value . "'");
> +    }
> +    else {
> +      $value =~ s/\n/\\n/g;    # handle embedded newlines
> +      $value =~ s/"/\\"/g;     # handle embedded quotes
> +      return('"' . $value . '"');
> +    }
> +  }
>   # allow simple scalars (without embedded quote chars) to be unquoted
> -  elsif ($value !~ /["'\\]/) {
> +  # (includes $%_+=-\;:,./)
> +  else {
>     return($value);
>   }
> -  # strings without double-quotes get double-quoted
> -  elsif ($value !~ /\"/) {
> -    $value =~ s{\\}{\\\\}g;
> -    return qq{"$value"};
> -  }
> -  # other strings get single-quoted
> -  else {
> -    $value =~ s{([\\'])}{\\$1}g;
> -    return qq{'$value'};
> -  }
> }
>
> 1;
> =================================================
>
> On 5/11/06, Johan Vromans <jvr...@sq...> wrote:
>> and...@fr... (Andreas J. Koenig) writes:
>>
>> >>>>>> On Wed, 10 May 2006 08:40:33 +0200,  
>> and...@fr... (Andreas J. Koenig) said:
>> >
>> >   > This seems to be a bug in Module::Build that is still there  
>> in 0.28.
>> >
>> > Another case where Module::Build without YAML fails is PHILCROW/ 
>> Gantry-3.30.tar.gz
>> >
>> > M:B generates a YAML file that contains
>> >
>> > ----snip----
>> > author:
>> >   - Tim Keefer <tk...@gm...>
>> > Phil Crow <phi...@ya...>
>> > ----snip----
>>
>> In the current Module::Build::YAML, line 105, read:
>>
>>   # XXX doesn't handle embedded newlines
>>
>> Shouldn't be so hard to fix...
>>
>> -- Johan
>>
>>
>> -------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services,  
>> security?
>> Get stuff done quickly with pre-integrated technology to make your  
>> job easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
>> Geronimo
>> http://sel.as-us.falkag.net/sel? 
>> cmd=lnk&kid=120709&bid=263057&dat=121642
>> _______________________________________________
>> Module-build-general mailing list
>> Mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/module-build-general
>>
>> <mbyaml-patch.tar.gz>

Re: [Module::Build] broken META.yml file

[Randy W. S. <ml...@th...>]

Stephen Adkins wrote:
> Hi,
> 
> I wrote Module::Build::YAML.
> I offer these patches.
> 
> Both of the previously mentioned broken YAML files were a
> result of embedded newlines.  I opted to tackle newlines and
> all of the other special characters which I could find which
> YAML considers special.
> 
> I think this version will be quite an improvement in terms of
> robustness in the presence of "odd" characters in the scalar
> values.  I patched

I'm not sure this is correct; although, it is much improved. The second 
instance with the module 'Gantry' should have an 'author' node that is 
an array containing 2 elements. With your patch, I untared Gantry and ran

# ysh
ysh > <META.yml
YAML Error: Invalid element in map
    Code: YAML_LOAD_ERR_BAD_MAP_ELEMENT
    Line: 6
    Document: 1
  at C:\devel\perl\5.8.7\site\lib/YAML.pm line 60
ysh > :q

Which is the original error. After running

# Build.PL
[ignore output]

# perl Build distmeta
Deleting Makefile.PL
Creating Makefile.PL
Deleting META.yml
Creating META.yml

# ysh
ysh > <META.yml
$VAR1 = {
   'name' => 'Gantry',
   'version' => '3.30',
   'author' => [
     'Tim Keefer <tk...@gm...>
Phil Crow <phi...@ya...>'
   ],
   ...
};

Notice that author is an array with *one* element that is a multiline 
string.

In the other case, we probably have a problem up stream(???): I would 
expect the abstract to be returned as a single string removing the 
formatting from the pod. I.E. Single \n should be collapsed along with 
surrounding whitespace, while \n\n should remain a paragraph break.

But do notice that YAML.pm when give a string with embedded \n 
characters writes the result out as verbatim text:

use YAML;
require YAML::Node;

my $data = { text =>
   "YAML::Node is a class for generating and manipulating these
   containers. A YAML node (or ynode) is a tied hash, array or
   scalar. In most ways it behaves just like the plain thing. But you
   can assign and retrieve and YAML type tag URI to it. For the hash
   flavor, you can also assign the order that the keys will be
   retrieved in. By default a ynode will offer its keys in the same
   order that they were assigned."
};

my $node = YAML::Node->new($data);
print Dump $node;

__END__

outputs:
---
text: |-
   YAML::Node is a class for generating and manipulating these
     containers. A YAML node (or ynode) is a tied hash, array or
     scalar. In most ways it behaves just like the plain thing. But you
     can assign and retrieve and YAML type tag URI to it. For the hash
     flavor, you can also assign the order that the keys will be
     retrieved in. By default a ynode will offer its keys in the same
     order that they were assigned.


Whereas the same text without embedded \n gets printed as:
---
text: 'YAML::Node is a class for generating and manipulating these 
containers. A YAML node (or ynode) is a tied hash, array or scalar. In 
most ways it behaves just like the plain thing. But you can assign and 
retrieve and YAML type tag URI to it. For the hash flavor, you can also 
assign the order that the keys will be retrieved in. By default a ynode 
will offer its keys in the same order that they were assigned.'

Regards,
Randy.

Re: [Module::Build] broken META.yml file

[Stephen A. <spa...@gm...>]

Hi,

I am happy to address any real issue that you've found.
It appears you are reporting two issues.
Please clarify if I am misunderstanding the issues you reported.

1. The Gantry distribution does "author" wrong

I think I am doing this right.  The fact that the two authors show
up as one string is not a YAML or Module::Build::YAML problem.
The "author" element is extracted from the "=3Dhead1 AUTHOR"
section of "Gantry.pm" because "module_name =3D> 'Gantry',"
is in Build.PL (and no "author" entry exists).

Here are the relevant files.

http://search.cpan.org/src/PHILCROW/Gantry-3.30/META.yml
http://search.cpan.org/src/PHILCROW/Gantry-3.30/Build.PL
http://search.cpan.org/src/PHILCROW/Gantry-3.30/lib/Gantry.pm

The META.yml which comes with the Gantry-3.30 distribution
was generated with Module::Build version 0.2611, without YAML.pm,
and it creates illegal YAML (as you pointed out). So does the
initial 0.28 which uses Module::Build::YAML.

--- #YAML:1.0
name: Gantry
version: 3.30
author:
  - Tim Keefer <tk...@gm...>
Phil Crow <phi...@ya...>
abstract: Web application framework for mod_perl, cgi, etc.
license: perl
generated_by: Module::Build version 0.2611, without YAML.pm

On my system, I have YAML.pm installed, and a "./Build distmeta"
creates:

author:
  - |-
    Tim Keefer <tk...@gm...>
    Phil Crow <phi...@ya...>

This is legal YAML (good) but still not the array that is desired (bad).
It is also a bit nicer to look at than the output of Module::Build::YAML,
but it is functionally equivalent (below).

author:
  - "Tim Keefer <tk...@gm...>\nPhil Crow <phi...@ya...>"

I don't know if somewhere else in Module::Build needs to do something
different in scanning Gantry.pm (to get the desired arrayref), but
Module::Build::YAML is behaving correctly (and creating the same
result as YAML.pm).

2. It would be nicer if long text sections were formatted nicer

I think the requirement that Module::Build::YAML is meeting is to
produce valid YAML which works.

Module::Build::YAML - Provides just enough YAML support so that
Module::Build works even if YAML.pm is not installed

I wanted to stay away from the many flavors of multi-line formatting
which are available in the full YAML spec.  However, I believe that
Module::Build::YAML does work properly.

Instead of

text: |-
   YAML::Node is a class for generating and manipulating these
     containers. A YAML node (or ynode) is a tied hash, array or
     scalar. In most ways it behaves just like the plain thing. But you
     can assign and retrieve and YAML type tag URI to it. For the hash
     flavor, you can also assign the order that the keys will be
     retrieved in. By default a ynode will offer its keys in the same
     order that they were assigned.

it produces

text: "YAML::Node is a class for generating and manipulating
these\ncontainers. A YAML node (or ynode) is a tied hash, array
or\nscalar. In most ways it behaves just like the plain thing. But
you\ncan assign and retrieve and YAML type tag URI to it. For the
hash\nflavor, you can also assign the order that the keys will
be\nretrieved in. By default a ynode will offer its keys in the
same\norder that they were assigned."

in a single line.  This is uglier, but it is still functionally correct.
If people think I need to implement the more aesthetically pleasing
format, I will consider adding it.

(I was considering implementing the Load() subroutine with the promise
that Module::Build::YAML::Load() can read anything it can Dump().
I wanted to stay away from the fancy multi-line YAML formats to
make parsing simpler.)

Stephen

On 5/15/06, Randy W. Sims <ml...@th...> wrote:
> Stephen Adkins wrote:
> > Hi,
> >
> > I wrote Module::Build::YAML.
> > I offer these patches.
> >
> > Both of the previously mentioned broken YAML files were a
> > result of embedded newlines.  I opted to tackle newlines and
> > all of the other special characters which I could find which
> > YAML considers special.
> >
> > I think this version will be quite an improvement in terms of
> > robustness in the presence of "odd" characters in the scalar
> > values.  I patched
>
> I'm not sure this is correct; although, it is much improved. The second
> instance with the module 'Gantry' should have an 'author' node that is
> an array containing 2 elements. With your patch, I untared Gantry and ran
>
> # ysh
> ysh > <META.yml
> YAML Error: Invalid element in map
>     Code: YAML_LOAD_ERR_BAD_MAP_ELEMENT
>     Line: 6
>     Document: 1
>   at C:\devel\perl\5.8.7\site\lib/YAML.pm line 60
> ysh > :q
>
> Which is the original error. After running
>
> # Build.PL
> [ignore output]
>
> # perl Build distmeta
> Deleting Makefile.PL
> Creating Makefile.PL
> Deleting META.yml
> Creating META.yml
>
> # ysh
> ysh > <META.yml
> $VAR1 =3D {
>    'name' =3D> 'Gantry',
>    'version' =3D> '3.30',
>    'author' =3D> [
>      'Tim Keefer <tk...@gm...>
> Phil Crow <phi...@ya...>'
>    ],
>    ...
> };
>
> Notice that author is an array with *one* element that is a multiline
> string.
>
> In the other case, we probably have a problem up stream(???): I would
> expect the abstract to be returned as a single string removing the
> formatting from the pod. I.E. Single \n should be collapsed along with
> surrounding whitespace, while \n\n should remain a paragraph break.
>
> But do notice that YAML.pm when give a string with embedded \n
> characters writes the result out as verbatim text:
>
> use YAML;
> require YAML::Node;
>
> my $data =3D { text =3D>
>    "YAML::Node is a class for generating and manipulating these
>    containers. A YAML node (or ynode) is a tied hash, array or
>    scalar. In most ways it behaves just like the plain thing. But you
>    can assign and retrieve and YAML type tag URI to it. For the hash
>    flavor, you can also assign the order that the keys will be
>    retrieved in. By default a ynode will offer its keys in the same
>    order that they were assigned."
> };
>
> my $node =3D YAML::Node->new($data);
> print Dump $node;
>
> __END__
>
> outputs:
> ---
> text: |-
>    YAML::Node is a class for generating and manipulating these
>      containers. A YAML node (or ynode) is a tied hash, array or
>      scalar. In most ways it behaves just like the plain thing. But you
>      can assign and retrieve and YAML type tag URI to it. For the hash
>      flavor, you can also assign the order that the keys will be
>      retrieved in. By default a ynode will offer its keys in the same
>      order that they were assigned.
>
>
> Whereas the same text without embedded \n gets printed as:
> ---
> text: 'YAML::Node is a class for generating and manipulating these
> containers. A YAML node (or ynode) is a tied hash, array or scalar. In
> most ways it behaves just like the plain thing. But you can assign and
> retrieve and YAML type tag URI to it. For the hash flavor, you can also
> assign the order that the keys will be retrieved in. By default a ynode
> will offer its keys in the same order that they were assigned.'
>
> Regards,
> Randy.
>

Re: [Module::Build] broken META.yml file

[Randy W. S. <ml...@th...>]

Stephen Adkins wrote:
> Hi,
> 
> I am happy to address any real issue that you've found.
> It appears you are reporting two issues.
> Please clarify if I am misunderstanding the issues you reported.
> 
> 1. The Gantry distribution does "author" wrong
> 
> I think I am doing this right.  The fact that the two authors show
> up as one string is not a YAML or Module::Build::YAML problem.
> The "author" element is extracted from the "=head1 AUTHOR"
> section of "Gantry.pm" because "module_name => 'Gantry',"
> is in Build.PL (and no "author" entry exists).

Oops, sorry. I didn't look deep enough. Your are correct.

> 2. It would be nicer if long text sections were formatted nicer
> 
> I think the requirement that Module::Build::YAML is meeting is to
> produce valid YAML which works.
> 
> Module::Build::YAML - Provides just enough YAML support so that
> Module::Build works even if YAML.pm is not installed
> 
> I wanted to stay away from the many flavors of multi-line formatting
> which are available in the full YAML spec.  However, I believe that
> Module::Build::YAML does work properly.

Yes, this does work. I didn't know if it was worth the trouble of adding 
the additional behavior. I'm very happy with what you've already 
provided though.

Thanks,
Randy.

Re: [Module::Build] broken META.yml file

[Eric W. <scr...@gm...>]

# from Stephen Adkins
# on Tuesday 16 May 2006 05:55 am:

>1. The Gantry distribution does "author" wrong

>I don't know if somewhere else in Module::Build needs to do something
>different in scanning Gantry.pm (to get the desired arrayref), but
>Module::Build::YAML is behaving correctly (and creating the same
>result as YAML.pm).

=3Dhead1 AUTHOR

=46oo <fo...@ba...>

Bar <ba...@ba...>

=3Dcut

=2E..is two paragraphs (renders on separate lines in perldoc)

=3Dhead1 AUTHOR

=46oo <fo...@ba...>
Bar <ba...@ba...>

=3Dcut

=2E..is one paragraph

=3Dhead1 AUTHOR

  Foo <fo...@ba...>
  Bar <ba...@ba...>

=3Dcut

=2E..is one literal paragraph of two lines

=3Dhead1 AUTHOR

  Foo <foo at bar dot com>

=3Dcut

=2E.. is ignored without my patch, which was ignored.

But it looks like the behavior WRT lines/paragraphs/literals may depend=20
on whether Pod::Parser is installed as textblock() is fed a paragraph=20
while the M::B::PodParser::_myparse_from_filehandle() is going=20
line-by-line.

=46rom the looks of things, the first three cases should turn into=20
something like this (where $f =3D "Foo ..." and $b =3D "Bar ...".)

1.
M::B::PP [$f, $b]
P::P [$f, $b]

2.
M::B::PP [$f, $b]
P::P ["$f $b"]

3.
M::B::PP ["  $f", "  $b"]
P::P ["  $f\n  $b"]

The automated test is not trivial, but is still left as an exercise for=20
the reader.

=2D-Eric
=2D-=20
Minus 1 million points for not setting your clock forward and trying the
code.
=2D-Michael Schwern
=2D--------------------------------------------------
    http://scratchcomputing.com
=2D--------------------------------------------------

Re: [Module::Build] broken META.yml file

[Randy W. S. <ml...@th...>]

Eric Wilhelm wrote:
> # from Stephen Adkins
> # on Tuesday 16 May 2006 05:55 am:
> 
> 
>>1. The Gantry distribution does "author" wrong
> 
> 
>>I don't know if somewhere else in Module::Build needs to do something
>>different in scanning Gantry.pm (to get the desired arrayref), but
>>Module::Build::YAML is behaving correctly (and creating the same
>>result as YAML.pm).
> 
> 
> =head1 AUTHOR
> 
> Foo <fo...@ba...>
> 
> Bar <ba...@ba...>
> 
> =cut
> 
> ...is two paragraphs (renders on separate lines in perldoc)
> 
> =head1 AUTHOR
> 
> Foo <fo...@ba...>
> Bar <ba...@ba...>
> 
> =cut
> 
> ...is one paragraph
> 
> =head1 AUTHOR
> 
>   Foo <fo...@ba...>
>   Bar <ba...@ba...>
> 
> =cut
> 
> ...is one literal paragraph of two lines
> 
> =head1 AUTHOR
> 
>   Foo <foo at bar dot com>
> 
> =cut
> 
> ... is ignored without my patch, which was ignored.

Not ignored; just not applied. ;)  I saw the discussion, but kept quiet 
because I have a stong bias against obfuscation: I stubbornly refuse to 
give ground to spammers-I was here first. Besides, obfuscated addresses 
are as easy to detect and de-obfuscate as a normal address, and there 
are several ways to obfuscate email where your patch only detects one 
particular method.

> But it looks like the behavior WRT lines/paragraphs/literals may depend 
> on whether Pod::Parser is installed as textblock() is fed a paragraph 
> while the M::B::PodParser::_myparse_from_filehandle() is going 
> line-by-line.
> 
> From the looks of things, the first three cases should turn into 
> something like this (where $f = "Foo ..." and $b = "Bar ...".)
> 
> 1.
> M::B::PP [$f, $b]
> P::P [$f, $b]
> 
> 2.
> M::B::PP [$f, $b]
> P::P ["$f $b"]
> 
> 3.
> M::B::PP ["  $f", "  $b"]
> P::P ["  $f\n  $b"]
> 
> The automated test is not trivial, but is still left as an exercise for 
> the reader.

Maybe we should just complain and ask the user to specify the 
'dist_author' argument to the constructor if we detect multiple lines? 
Or we could require format (2) above in the AUTHOR section where there 
is one address perl line; we should be able to detect that reliably enough.

Randy.

Re: [Module::Build] broken META.yml file

[Eric W. <scr...@gm...>]

# from Randy W. Sims
# on Tuesday 16 May 2006 12:58 pm:

>> 1.
>> M::B::PP [$f, $b]
>> P::P [$f, $b]
>>
>> 2.
>> M::B::PP [$f, $b]
>> P::P ["$f $b"]
>>
>> 3.
>> M::B::PP [" =A0$f", " =A0$b"]
>> P::P [" =A0$f\n =A0$b"]
>>
>> The automated test is not trivial, but is still left as an exercise
>> for the reader.
>
>Maybe we should just complain and ask the user to specify the
>'dist_author' argument to the constructor if we detect multiple lines?
>Or we could require format (2) above in the AUTHOR section where there
>is one address perl line; we should be able to detect that reliably
> enough.

Er, do you mean "format (1)" and "one address per paragraph"?

It's the only one that works consistently with the current code.  Plus,=20
(2) and (3) are both somewhat incorrect anyway (any pod renderer gets=20
to put the stuff from (2) on a single line and as for (3), pod2html=20
doesn't render links in literal paragraphs.)

=2D-Eric
=2D-=20
"Beware of bugs in the above code; I have only proved it correct, not
tried it."
=2D-Donald Knuth
=2D--------------------------------------------------
    http://scratchcomputing.com
=2D--------------------------------------------------

Re: [Module::Build] broken META.yml file

[Randy W. S. <ml...@th...>]

Eric Wilhelm wrote:
> # from Randy W. Sims
> # on Tuesday 16 May 2006 12:58 pm:
> 
>>> 1.
>>> M::B::PP [$f, $b]
>>> P::P [$f, $b]
>>>
>>> 2.
>>> M::B::PP [$f, $b]
>>> P::P ["$f $b"]
>>>
>>> 3.
>>> M::B::PP ["  $f", "  $b"]
>>> P::P ["  $f\n  $b"]
>>>
>>> The automated test is not trivial, but is still left as an exercise
>>> for the reader.
>> Maybe we should just complain and ask the user to specify the
>> 'dist_author' argument to the constructor if we detect multiple lines?
>> Or we could require format (2) above in the AUTHOR section where there
>> is one address perl line; we should be able to detect that reliably
>> enough.
> 
> Er, do you mean "format (1)" and "one address per paragraph"?
> 
> It's the only one that works consistently with the current code.  Plus, 
> (2) and (3) are both somewhat incorrect anyway (any pod renderer gets 
> to put the stuff from (2) on a single line and as for (3), pod2html 
> doesn't render links in literal paragraphs.)

Actually, I meant (3). When looking at the top part of you previous 
message, I mis-counted. However, your point about pod2html kills that. 
So, yeah, we could require (1), the only viable option, or we could 
display an informative message to the user if we detect an author 
section spanning multiple lines.

Randy.