module-build-general

[Module::Build] Some updates, and catching up...

[Adam K. <ad...@ph...>]

Hi folks

I was asked to point you in the direction of the following

http://use.perl.org/~Alias/journal/29427

If it fits what you want and you'd like to embed it, you are welcome to 
hack on it directly.

http://use.perl.org/~Alias/journal/29327

Also, is there somewhere that has a nntp server for this list, like you 
can get for most of the other Perl mailing lists?

Oh, and there seems to be some confusion about the meaning of META.yml?

As I understand it from doing CPAN.pm and Module::Install stuff, 
META.yml is intrinsically dynamic and platform dependant, where 
"platform" actually means environment, and is going to be different for 
every single host.

A META.yml file is packed up with the dist for the convenience of larger 
systems, and META.yml in this case is known to only be an approximation.

This is enforced by the dynamic_config flag, which defaults to true.

When true, the CPAN client should be aware it needs to run the installer 
script first, which will localise the META.yml content, to get a 
definitive result for that environment.

Of course, in the case where dynamic_config is explicitly set to false, 
the installer is advertising (via inserting the entry at dist-time) that 
the META.yml will not change when the installer is run, and so the 
client is free to shortcut the dependency installation if it wishes.

BOTH of these situations (the version in the tarball and the version 
post-installer.PL) have their uses in a few different cases, and both 
really need to be retained.

The problem with doing some sort of optional feature is that it 
introduces branch logic into your data structures.

So you need a way to describe the branch... platform => 'Win32'... [ 
platform => 'Win32', perl => '> 5.005 || < 5.008004' ], and so on and so on.

But because you are trying to express logic (code) as data, you are 
always going to cases where you need just that bit more syntax and 
flexibility, until before you know it you have invented your own new 
mini-language, and well, then we might as well use Perl.

But then of course we have Perl inside META.yml, and we're back to square 1.

I don't know whether or not it was planned, but the current situation, 
with a flagged-imperfect uploaded META.yml, which is localised at 
Makefile|Build.PL-time quite comfortably covers 100% of all use cases 
I'm aware of.

I'm more than happy the way things are in this case.

Adam K

Re: [Module::Build] Some updates, and catching up...

[David G. <da...@hy...>]

Adam Kennedy wrote:
> A META.yml file is packed up with the dist for the convenience of larger 
> systems, and META.yml in this case is known to only be an approximation.

I think it's for the convenience of telling indexers about:

(a) metadata (e.g. version numbers, dependencies) that would otherwise 
require them to run (potentially unsafe) code on all distributions submitted

(b) metadata about the distribution structure, e.g. files that 
should/shouldn't be indexed

> When true, the CPAN client should be aware it needs to run the installer 
> script first, which will localise the META.yml content, to get a 
> definitive result for that environment.

That seems redundant -- I think the installers can notify clients of 
dependencies directly without a new META.yml being created.  Once an 
installer has committed to running Makefile|Build.PL, the META.yml is 
irrelevant.

> The problem with doing some sort of optional feature is that it 
> introduces branch logic into your data structures.
> 
> So you need a way to describe the branch... platform => 'Win32'... [ 
> platform => 'Win32', perl => '> 5.005 || < 5.008004' ], and so on and so 
> on.
> 
> But because you are trying to express logic (code) as data, you are 
> always going to cases where you need just that bit more syntax and 
> flexibility, until before you know it you have invented your own new 
> mini-language, and well, then we might as well use Perl.

As long as it's a mini-language that can be parsed safely, then having 
some expressiveness is fine.  Most of where this will crop up is in 
module dependencies anyway, and the number of variables there is 
relatively contained:

* platforms (in some set)
* perl versions (in some set of ranges of versions)
* module versions (in some set of ranges of versions)

The logic is pretty simple, too.  If a given set of those conditions 
above is true, then a given set of dependencies exist (which can be 
expressed again as a set of ranges of versions).

> But then of course we have Perl inside META.yml, and we're back to 
> square 1.

A mini language for that shouldn't be too challenging to write and parse.

> I don't know whether or not it was planned, but the current situation, 
> with a flagged-imperfect uploaded META.yml, which is localised at 
> Makefile|Build.PL-time quite comfortably covers 100% of all use cases 
> I'm aware of.

Use case: Create an index of all dependencies for all distributions on 
the Win32 platform without running Makefile|Build.PL for all 
distributions.  (E.g. for a potential "CP-PAN" i.e. "CamelPack-PAN")

Personally, I think that YAML is complete overkill for CPAN distribution 
metadata.  Its major advantage seems to be that it's easily read by humans.

Regards,
David Golden

Re: [Module::Build] Some updates, and catching up...

[Ken W. <ke...@ma...>]

On Apr 24, 2006, at 11:49 AM, David Golden wrote:

> Adam Kennedy wrote:
>> A META.yml file is packed up with the dist for the convenience of  
>> larger systems, and META.yml in this case is known to only be an  
>> approximation.
>
> I think it's for the convenience of telling indexers about:
>
> (a) metadata (e.g. version numbers, dependencies) that would  
> otherwise require them to run (potentially unsafe) code on all  
> distributions submitted
>
> (b) metadata about the distribution structure, e.g. files that  
> should/shouldn't be indexed

(c) Simple metadata like author name, distribution name, abstract,  
support URLs, license, and so on, that would otherwise be buried in  
various places in the tarball

and

(d) Letting humans browse this same data too


>> When true, the CPAN client should be aware it needs to run the  
>> installer script first, which will localise the META.yml content,  
>> to get a definitive result for that environment.
>
> That seems redundant -- I think the installers can notify clients  
> of dependencies directly without a new META.yml being created.   
> Once an installer has committed to running Makefile|Build.PL, the  
> META.yml is irrelevant.

Yeah, the META.yml is really not the right place for installers to be  
looking for dependencies.  A better place is _build/prereq after the  
Build.PL is run.


>
>> The problem with doing some sort of optional feature is that it  
>> introduces branch logic into your data structures.
>> So you need a way to describe the branch... platform => 'Win32'...  
>> [ platform => 'Win32', perl => '> 5.005 || < 5.008004' ], and so  
>> on and so on.
>> But because you are trying to express logic (code) as data, you  
>> are always going to cases where you need just that bit more syntax  
>> and flexibility, until before you know it you have invented your  
>> own new mini-language, and well, then we might as well use Perl.
>
> As long as it's a mini-language that can be parsed safely, then  
> having some expressiveness is fine.  Most of where this will crop  
> up is in module dependencies anyway, and the number of variables  
> there is relatively contained:
>
> * platforms (in some set)
> * perl versions (in some set of ranges of versions)
> * module versions (in some set of ranges of versions)
>
> The logic is pretty simple, too.  If a given set of those  
> conditions above is true, then a given set of dependencies exist  
> (which can be expressed again as a set of ranges of versions).
>
>> But then of course we have Perl inside META.yml, and we're back to  
>> square 1.
>
> A mini language for that shouldn't be too challenging to write and  
> parse.

Yeah, by no means should we ever put Perl inside the META.yml.  That  
would be serious trouble.  We'd be back to square zero as far as I'm  
concerned.

The mini-language is something we do plan to do, we've made some  
headway in the past but it's waiting for some other more pressing  
issues.


> Use case: Create an index of all dependencies for all distributions  
> on the Win32 platform without running Makefile|Build.PL for all  
> distributions.  (E.g. for a potential "CP-PAN" i.e. "CamelPack-PAN")
>
> Personally, I think that YAML is complete overkill for CPAN  
> distribution metadata.  Its major advantage seems to be that it's  
> easily read by humans.

It's definitely overkill, but the small subset of YAML that we  
actually use is a pretty good fit.  Human readability is a pretty  
huge factor IMO.  And now we have a simple YAML generator bundled  
with M::B too, so we don't need to worry about YAML's dependency chain.

  -Ken