Thread: [Module-build-general] Proposal for Module::Build installation dirs
Status: Beta
Brought to you by:
kwilliams
|
From: Ken W. <ke...@ma...> - 2003-05-14 17:14:19
|
Hi,
I'd like to get the ball rolling on the installation map stuff for
M::B, as I think it's the only thing holding up the 0.19 release.
Here's my idea of what I think we should do, please give me your
feedback on it.
MakeMaker gives us a pretty good test-case for how an installation map
should & shouldn't work, and what features we probably need to support
to be "competitive". The main MakeMaker thing we probably *don't* want
to replicate is the way it handles PREFIX, because I understand from
Schwern that it's been a nightmare to support in a way that works
nicely on all the various platforms & setups.
Right now M::B has a concept that I sort of like, called the
"Installation Map". In the CVS code, have a look at the install_map(),
install_types(), and install_destination() methods and how they
cooperate to tell ExtUtils::Install where to put things.
The idea is that there's a fairly limited set of "things" that we know
how to install. Each corresponds to a subdirectory of the blib/
directory:
lib - architecture-independent items like .pm files
arch - architecture-dependent items like compiled .xs files
script - architecture-independent executables
bin - architecture-dependent executables
man1 - man pages in the .1 section
man3 - man pages in the .3 section
(Actually, bin, man1, and man3 aren't actually generated yet by M::B,
but I suppose they will be before too long.) (And ExtUtils::Install
will actually move things in lib/ to arch/ if it is anything in arch/ .
Which seems sensible.)
So all we really need to know is where each of these things should get
installed to. (Sounds so easy.)
Here's what I've come up with for a proposal:
** Do what MakeMaker does, minus the fancy PREFIX support. **
This means we can let people specify a family of defaults using an
"installdirs" parameter set to one of three options:
'installdirs' set to:
core site vendor
results in the following defaults from Config.pm:
arch => installarchlib installsitearch installvendorarch
lib => installprivlib installsitelib installvendorlib
bin => installbin installsitebin installvendorbin
script => installscript installscript installscript
man1 => installman1dir installsiteman1dir
installvendorman1dir
man3 => installman3dir installsiteman3dir
installvendorman3dir
(I've changed the "perl" setting to "core" here, since that seems
like a better moniker for the concept.)
Then, with these defaults in place, we need a way for people to
override them. The first way, which is already possible, is to
override specific Config.pm entries like "installsitelib". The second
way, which is not yet possible, is to specify where a specific "thing"
like the "lib" dir contents should get installed.
If we want to support something like PREFIX, I would like it to not do
any guesswork or DWIM. If you specify a PREFIX of '/foo/bar', I would
like it to simply install to /foo/bar/lib, /foo/bar/bin, and so on.
(It wouldn't install to /foo/bar/arch, but rather to
/foo/bar/$Config{archname} .)
In the future, we need to think about extensibility too. The matrix
above should be able to grow in both directions - distributions might
install other types of stuff, so as long as they can figure out a
suitable default or ask the user for a destination, they can just plug
into the normal installation mechanism. And if we ever get into the
business of using user preference files, we could offer people a way to
define default sets besides "core", "site", and "vendor", or to
override entries in those three. This would essentially replace the
fancy-PREFIX stuff of MakeMaker, but in a more explicitly specified way.
Finally, note that an orthogonal 'destdir' parameter lets people
specify a directory prefix to prepend to all installation paths after
all paths have been determined in the usual way. This is mainly to
help rpm/deb/etc. package maintainers. And this is nice and simple and
probably doesn't need to be changed in any way.
*pant* *pant*
-Ken
|
|
From: Jeff M. <jef...@e-...> - 2003-05-14 17:59:26
|
On Wed, 2003-05-14 at 13:14, Ken Williams wrote: > Hi, > > I'd like to get the ball rolling on the installation map stuff for > M::B, as I think it's the only thing holding up the 0.19 release. > Here's my idea of what I think we should do, please give me your > feedback on it. > > MakeMaker gives us a pretty good test-case for how an installation map > should & shouldn't work, and what features we probably need to support > to be "competitive". The main MakeMaker thing we probably *don't* want > to replicate is the way it handles PREFIX, because I understand from > Schwern that it's been a nightmare to support in a way that works > nicely on all the various platforms & setups. > I use PREFIX and LIB all the time now and I haven't had any problems with it yet. Currently M::B doesn't work correctly with builddir or sitelib when doing a build test or install (see earlier postings about @INC from a fellow subscriber). So while there may be magic behind PREFIX, I don't think it is fair for the average module builder to have to know specifics about %Config either. I need M::B to be able to install stuff outside the system perl tree. MakeMaker allows me to do that today using PREFIX and LIB (I set both to the same value). All I have to do in my code is add a: use lib qw(<path specified by PREFIX>); -- Jeff Macdonald <jef...@e-...> |
|
From: Ken W. <ke...@ma...> - 2003-05-14 19:54:01
|
On Wednesday, May 14, 2003, at 12:59 PM, Jeff Macdonald wrote: > > I use PREFIX and LIB all the time now and I haven't had any problems > with it yet. Currently M::B doesn't work correctly with builddir or > sitelib when doing a build test or install (see earlier postings about > @INC from a fellow subscriber). You mean the thread where I called M::B's support for local installation directories "woefully inadequate"? I know M::B doesn't install (nicely) to arbitrary directories, that's sort of my whole point. What do you mean "builddir" and "sitelib"? > > So while there may be magic behind PREFIX, I don't think it is fair for > the average module builder to have to know specifics about %Config > either. In the scheme I proposed, the user has to know *less* about %Config than under MakeMaker, because under MakeMaker you have to understand how PREFIX interacts with your particular %Config settings in order to arrive at the installation paths. My scheme would just install in the obvious way under wherever you tell it to install. > I need M::B to be able to install stuff outside the system perl tree. > MakeMaker allows me to do that today using PREFIX and LIB (I set both > to > the same value). All I have to do in my code is add a: > > use lib qw(<path specified by PREFIX>); Yeah, that's why we're working on getting this working. Incidentally, Module::Build already lets you install to whatever directory you want, it's just not pretty: perl Build.PL config='siteperl=/foo/bar sitearch=/foo/bar/i386-linux installscript=/foo/bar/bin' -Ken |
|
From: Jeff M. <jef...@e-...> - 2003-05-16 15:30:11
|
On Wed, 2003-05-14 at 15:53, Ken Williams wrote:
> On Wednesday, May 14, 2003, at 12:59 PM, Jeff Macdonald wrote:
> >
> > I use PREFIX and LIB all the time now and I haven't had any problems
> > with it yet. Currently M::B doesn't work correctly with builddir or
> > sitelib when doing a build test or install (see earlier postings about
> > @INC from a fellow subscriber).
>
> You mean the thread where I called M::B's support for local
> installation directories "woefully inadequate"? I know M::B doesn't
> install (nicely) to arbitrary directories, that's sort of my whole
> point.
>
> What do you mean "builddir" and "sitelib"?
Ack, builddir should of been destdir and when I rerun the compile
destdir does what it should do. It also turns out it's not what I want.
It seems I want sitelib (but maybe I'm mis-understanding what sitelib
is):
>From the README:
Alternatively, you could specify the "sitelib" parameter when you run
the "Build.PL" script:
perl Build.PL sitelib=/my/secret/place/
Under normal circumstances, you'll need superuser privileges to install
into your system's default "sitelib" directory.
> >
> > So while there may be magic behind PREFIX, I don't think it is fair for
> > the average module builder to have to know specifics about %Config
> > either.
>
> In the scheme I proposed, the user has to know *less* about %Config
> than under MakeMaker, because under MakeMaker you have to understand
> how PREFIX interacts with your particular %Config settings in order to
> arrive at the installation paths. My scheme would just install in the
> obvious way under wherever you tell it to install.
I re-read your earlier message and I think I understand what your are saying now.
> > I need M::B to be able to install stuff outside the system perl tree.
> > MakeMaker allows me to do that today using PREFIX and LIB (I set both
> > to
> > the same value). All I have to do in my code is add a:
> >
> > use lib qw(<path specified by PREFIX>);
>
> Yeah, that's why we're working on getting this working.
>
> Incidentally, Module::Build already lets you install to whatever
> directory you want, it's just not pretty:
>
> perl Build.PL config='siteperl=/foo/bar sitearch=/foo/bar/i386-linux
> installscript=/foo/bar/bin'
Yes, it's butt ugly and requires me to know about %Config and
what my archname is. I guess just setting sitelib isn't enough or
doesn't do what I thought it would do. Here's some output of Build,
I apologize for long lines below. I realize that it pass the tests
but it does issue warnings and it also shows the @INC problem and
it trying to install into a directory it doesn't have permission to.
Let me know how I can help (I'm about to try that patch you just sent out):
PERL5LIB=/home/jeff/projects/mail-parser-5.0.1/build/YAML-0.35/blib/lib:/home/jeff/projects/mail-parser-5.0.1/build/Module-Info-0.19/blib/lib:/home/jeff/projects/mail-parser-5.0.1/build/ExtUtils-ParseXS-2.02/blib/lib:/home/jeff/projects/mail-parser-5.0.1/build/Archive-Tar-0.23/blib/lib perl Build.PL sitelib=/home/jeff/projects/mail-parser-5.0.1/test-install/lib
Checking whether your kit is complete...
Looks good
Creating new 'Build' script for 'Module-Build' version '0.18'
...
PERL5LIB=/home/jeff/projects/mail-parser-5.0.1/build/YAML-0.35/blib/lib:/home/jeff/projects/mail-parser-5.0.1/build/Module-Info-0.19/blib/lib:/home/jeff/projects/mail-parser-5.0.1/build/ExtUtils-ParseXS-2.02/blib/lib:/home/jeff/projects/mail-parser-5.0.1/build/Archive-Tar-0.23/blib/lib ./Build test && ./Build install;
t/basic.........ok
t/runthrough....ok 4/13B::Module::Info,packages use failed with 2 saying:
Can't locate B/Module/Info.pm in @INC (@INC contains: /home/jeff/projects/mail-parser-5.0.1/build/Module-Build-0.18/blib/lib /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl .) at (eval 1) line 18.
BEGIN failed--compilation aborted at (eval 1) line 18.
BEGIN failed--compilation aborted.
t/runthrough....ok 6/13B::Module::Info,packages use failed with 2 saying:
Can't locate B/Module/Info.pm in @INC (@INC contains: /home/jeff/projects/mail-parser-5.0.1/build/Module-Build-0.18/blib/lib /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl .) at (eval 1) line 18.
BEGIN failed--compilation aborted at (eval 1) line 18.
BEGIN failed--compilation aborted.
t/runthrough....ok
t/xs............ok
All tests successful.
Files=3, Tests=32, 161 wallclock secs ( 9.76 cusr + 0.81 csys = 10.57 CPU)
Warning: You do not have permissions to install into /usr/lib/perl5/site_perl/5.8.0 at /usr/lib/perl5/5.8.0/ExtUtils/Install.pm line 84.
mkdir /usr/lib/perl5/site_perl/5.8.0/Module: Permission denied at /usr/lib/perl5/5.8.0/ExtUtils/Install.pm line 137
make: *** [/home/jeff/projects/mail-parser-5.0.1/test-install/lib/Module/Build.pm] Error 255
--
Jeff Macdonald <jef...@e-...>
|
|
From: Ken W. <ke...@ma...> - 2003-05-16 16:52:07
|
On Friday, May 16, 2003, at 10:29 AM, Jeff Macdonald wrote: > Ack, builddir should of been destdir and when I rerun the compile > destdir does what it should do. Good. > It also turns out it's not what I want. destdir is mostly useful for package managers, it's not generally useful for installing into a private location like ~/lib . > It seems I want sitelib (but maybe I'm mis-understanding what sitelib > is): > > From the README: > > > Alternatively, you could specify the "sitelib" parameter when you run > the "Build.PL" script: > > perl Build.PL sitelib=/my/secret/place/ Ack, sorry - that part of the documentation has been wrong for several releases now. It'll get fixed for the next release. >> Incidentally, Module::Build already lets you install to whatever >> directory you want, it's just not pretty: >> >> perl Build.PL config='siteperl=/foo/bar sitearch=/foo/bar/i386-linux >> installscript=/foo/bar/bin' > > Yes, it's butt ugly and requires me to know about %Config and > what my archname is. Yup. And I guess I shouldn't comment on this technique anymore, since it's now "the old way", and won't even necessarily work with version 0.19 once we get something better figured out. > I guess just setting sitelib isn't enough or > doesn't do what I thought it would do. Here's some output of Build, > I apologize for long lines below. I realize that it pass the tests > but it does issue warnings and it also shows the @INC problem and > it trying to install into a directory it doesn't have permission to. > Let me know how I can help (I'm about to try that patch you just sent > out): I'm hopeful that the patch will fix the warnings about B::Module::Info, by fixing the @INC problem. The permission warnings are of course because it's not paying attention to where you told it to install, which is a separate issue and what we'll have to hash out here. -Ken |
|
From: Abigail <ab...@ab...> - 2003-07-03 13:25:16
|
On Wed, May 14, 2003 at 12:14:02PM -0500, Ken Williams wrote: > > This means we can let people specify a family of defaults using an > "installdirs" parameter set to one of three options: > > 'installdirs' set to: > core site vendor > > results in the following defaults from Config.pm: > > arch => installarchlib installsitearch installvendorarch > lib => installprivlib installsitelib installvendorlib > bin => installbin installsitebin installvendorbin > script => installscript installscript installscript > man1 => installman1dir installsiteman1dir > installvendorman1dir > man3 => installman3dir installsiteman3dir > installvendorman3dir Is the row for 'script' a typo, or are indeed all scripts, whether they are core, site and vendor always placed in the same directory? Abigail |
|
From: Ken W. <ke...@ma...> - 2003-07-03 15:11:40
|
On Thursday, July 3, 2003, at 08:24 AM, Abigail wrote:
> On Wed, May 14, 2003 at 12:14:02PM -0500, Ken Williams wrote:
>>
>> This means we can let people specify a family of defaults using an
>> "installdirs" parameter set to one of three options:
>>
>> 'installdirs' set to:
>> core site vendor
>>
>> results in the following defaults from Config.pm:
>>
>> arch => installarchlib installsitearch
>> installvendorarch
>> lib => installprivlib installsitelib installvendorlib
>> bin => installbin installsitebin installvendorbin
>> script => installscript installscript installscript
>> man1 => installman1dir installsiteman1dir
>> installvendorman1dir
>> man3 => installman3dir installsiteman3dir
>> installvendorman3dir
>
>
> Is the row for 'script' a typo, or are indeed all scripts, whether
> they are core, site and vendor always placed in the same directory?
Unfortunately it wasn't a typo. That's the way MakeMaker does it too,
because there's no such thing as 'installsitescript' or
'installvendorscript' in Config.pm.
However, I've since decided that this is unacceptable, since it really
does get in the way of sane layout management. It's also responsible
for the problem of LWP's /usr/bin/HEAD overwriting /usr/bin/head on Mac
OS X.
So in recent Module::Build betas, I've changed to this:
------------------------------------------------------------------
=head3 installdirs
The default destinations for these installable things come from
entries in your system's C<Config.pm>. You can select from three
different sets of default locations by setting the C<installdirs>
parameter as follows:
'installdirs' set to:
core site vendor
uses the following defaults from Config.pm:
lib => installprivlib installsitelib installvendorlib
arch => installarchlib installsitearch installvendorarch
script => installscript installsitebin installvendorbin
bin => installbin installsitebin installvendorbin
libdoc => installman3dir installsiteman3dir installvendorman3dir
bindoc => installman1dir installsiteman1dir installvendorman1dir
The default value of C<installdirs> is "site". If you're creating
vendor distributions of module packages, you may want to do something
like this:
perl Build.PL installdirs=vendor
or
Build install installdirs=vendor
If you're installing an updated version of a module that was included
with perl itself (i.e. a "core module"), then you may set
C<installdirs> to "core" to overwrite the module in its present
location.
(Note that the 'script' line is different from MakeMaker -
unfortunately there's no such thing as "installsitescript" or
"installvendorscript" entry in C<Config.pm>, so we use the
"installsitebin" and "installvendorbin" entries to at least get the
general location right. In the future, if C<Config.pm> adds some more
appropriate entries, we'll start using those.)
------------------------------------------------------------------
-Ken
|
|
From: Andy D. <dou...@la...> - 2003-07-03 17:49:28
|
On Thu, 3 Jul 2003, Ken Williams wrote:
>
> On Thursday, July 3, 2003, at 08:24 AM, Abigail wrote:
>
> >> 'installdirs' set to:
> >> core site vendor
> >>
> >> bin => installbin installsitebin installvendorbin
> >> script => installscript installscript installscript
> > Is the row for 'script' a typo, or are indeed all scripts, whether
> > they are core, site and vendor always placed in the same directory?
>
> Unfortunately it wasn't a typo. That's the way MakeMaker does it too,
> because there's no such thing as 'installsitescript' or
> 'installvendorscript' in Config.pm.
Well, there were supposed to be, and they are even documented in 5.6.1's
INSTALL file, but due to a variety of oversights, the actual code to set
them didn't make it into either 5.6.1 or 5.8.0, though it is now in
5.8.1-to-be.
> (Note that the 'script' line is different from MakeMaker -
> unfortunately there's no such thing as "installsitescript" or
> "installvendorscript" entry in C<Config.pm>, so we use the
> "installsitebin" and "installvendorbin" entries to at least get the
> general location right. In the future, if C<Config.pm> adds some more
> appropriate entries, we'll start using those.)
That sounds like a good plan. Those entries should appear in 5.8.1's
Config.pm, when that is released.
--
Andy Dougherty dou...@la...
|
|
From: Ken W. <ke...@ma...> - 2003-07-03 17:53:59
|
On Thursday, July 3, 2003, at 12:49 PM, Andy Dougherty wrote:
> On Thu, 3 Jul 2003, Ken Williams wrote:
>
>> (Note that the 'script' line is different from MakeMaker -
>> unfortunately there's no such thing as "installsitescript" or
>> "installvendorscript" entry in C<Config.pm>, so we use the
>> "installsitebin" and "installvendorbin" entries to at least get the
>> general location right. In the future, if C<Config.pm> adds some more
>> appropriate entries, we'll start using those.)
>
> That sounds like a good plan. Those entries should appear in 5.8.1's
> Config.pm, when that is released.
Oh, good. That means I don't have to do the lobbying/convincing that I
thought I was going to have to do. =)
The code in the soon-to-be-released version of Module::Build looks like
this:
core => {
lib => $c->{installprivlib},
arch => $c->{installarchlib},
bin => $c->{installbin},
script => $c->{installscript},
bindoc => $c->{installman1dir},
libdoc => $c->{installman3dir},
},
site => {
lib => $c->{installsitelib},
arch => $c->{installsitearch},
bin => $c->{installsitebin},
script => $c->{installsitescript} ||
$c->{installsitebin},
bindoc => $c->{installsiteman1dir},
libdoc => $c->{installsiteman3dir},
},
vendor => {
lib => $c->{installvendorlib},
arch => $c->{installvendorarch},
bin => $c->{installvendorbin},
script => $c->{installvendorscript} ||
$c->{installvendorbin},
bindoc => $c->{installvendorman1dir},
libdoc => $c->{installvendorman3dir},
},
where $c is a reference to %Config. So it should just start working if
the %Config entries appear.
Thanks for the status update.
-Ken
|