luadist-devel

[LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

Hello list,
since we are planing for the 1.0.0 release of LuaDist by the end of next 
month I wanted to include one last feature that I feel improves the 
possibilities LD offers and solves few issues we have been observing 
while using the current SVN version. It would be too late after the release.

The SVN version now supports conditionals in dist.info for source 
modules, this combined with the availability of all build variables that 
users can define using the -D[CMAKE_VARIABLE]=[value] or pass using the 
API in the dist.info environment offers following possibilities to dist 
maintainers.

Case A: Adding Architecture Conditioned Dependency
As an example I take LD itself, on Windows LD needs luadist_utils 
dependency that is not required on other platforms. Up till now the 
syntax to write architecture conditioned dependencies was following

Windows.dependencies = {
    dep1 => ">= 0.1",
    ..
    ..
}

This required support in the LD code to read, dependencies had to be 
duplicated and no logic could be implemented. Now we can use 
LUADIST_ARCH variable instead.

-- For all platforms
depencencies = {
    lua = ">=5.1"
    luasocket ...
    ..
}

if LUADIST_ARCH == "Windows" then
    dependencies["luadist_utils"] = ">=0.1"
end

Any and all other variables can be used in dist.info, this opens up 
other possibilities.

Case B: Conditional builds and "USE" variables.

Inspired by the portage build system we can now implement similar 
behavior to provide users with the choice of configuring the source dist 
build settings.

I will use IUP as an example. IUP offers users the choice to build 
against Motif, GTK and on Windows it can use the native GUI. We can now 
conditionally build the dist depending on user choice and possibilities:

eg: user provides -DUSE_MOTIF=true variable when building IUP

IUP dist info contains conditional dependency to motif that would look 
similar to following example:

if USE_MOTIF then
    dependencies["motif"] = ">=1.0.0"
end

if USE_GTK then
    dependencies["gtk"] = ">=1.0.0"
end

Now the maintainer can modify his CMakeLists.txt to support the use of 
these variables. The changes would look similar to the following loose 
example.

...
IF(USE_MOTIF)
    FIND_MODULE(Motif REQUIRED)
    SET(LIBS ${MOTIF_LIBRARIES)
    ADD_DEFINITIONS(-DIUP_MOTIF=true)
    ...
ENDIF()
IF(Windows)
    ADD_DEFINITIONS(-DIUP_WIN32=true)
ENDIF()
..

Case C: Conditional Installation of Components and other Conventions

Additionally this approach can be used to disable installation of 
certain components depending on user choice but would require rewriting 
of all existing dists. If our guide for dist maintainers contained 
conventions such as using of INSTALL_ variables for controlling 
installation or other distribution wide aspects we could build luadist 
with no documentation in cases when its desired by using 
-DINSTALL_DOC=false.

These conventions are up for discussion and I would like to hear your 
suggestions on different uses of this feature.

Now, how does this affect binary dists ?

The nice thing about this approach is that LD evaluates the contents of 
dist.info in the source dists and automatically produces appropriate 
dist.info without any conditionals in the resulting binary dists. This 
ensures dependencies always match regardless of any USE variables being 
set when installing a binary dist.

I hope this rather long explanation made sense and would like to hear 
your comments on this feature.

pd

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Sun, Aug 30, 2009 at 10:51 AM, Peter Drahos wrote:
> The SVN version now supports conditionals in dist.info for source
> modules
> ...
> if LUADIST_ARCH == "Windows" then
>    dependencies["luadist_utils"] = ">=0.1"
> end
> ...
> The nice thing about this approach is that LD evaluates the contents of
> dist.info in the source dists and automatically produces appropriate
> dist.info without any conditionals in the resulting binary dists.


A theoretical downside is that the dist.info format is now less
declarative.  The set of all actual (i.e. evaluated) dist.info
structures is only determinable by evaluating the dist.info as
arbitrary Lua code for all values of the variables, and this set of
values might be infinite.  This could have implications if I write a
tool like search.cpan.org that doesn't install dists but rather
indexes them and displays this information to the user.  For example,
it's now less obvious for a program to determine the set of all dists
that are or might be dependencies of a given dist.

One workaround to the above problem is to require dists to enumerate
all variables and possible values:

  variable.USE_MOTIF = {false, true}
  variable.USE_GTK = {false, true}

Now, assuming the maintainer correctly defined these enumerations, we
can make the above determination given a Lua sandbox.

If this were in an object-oriented world, the IUP dist would accept
any dist that provided a certain interface X, even dists unknown to
IUP.  The Motif and GTK dists (or adapters for them) would provide
this interface X.  LuaDist (not IUP) would find all dists that provide
the interface X and choose the most appropriate one.  This might not
be so straightforward in the case of IUP.  We might break up IUP into
multiple dists: iup, iup-core, iup-gtk, iup-motif, etc. and define
dependencies.  It might be much simpler to templatize the IUP dist --
iup<gtk>, iup<motif>, etc. -- which in fact could be equivalent to
what you are suggesting.  Note that this suggests possibly allowing a
dist to depend on a particular instantiation:

  dependencies = { ["iup<gtk>"] = ... }

luaglut is another example where the dependencies are not well specified:

  dependencies = {
      lua = ">= 5.1",
      openglut = ">= 0.6.3",
      freeglut = ">= 2.4.0",
      glut = ">=3.7.6"
  }

That currently attempts to install all three glut dists.  Many users
would probably be satisfied with LuaDist by default attempting to
install each one and stopping once one successfully installs (e.g. if
freeglut fails to compile, automatically try openglut).  I suggest
providing for this behavior in whichever solution is chosen.  I would
be fine with

  dependencies = {
      lua = ">= 5.1",
      OR {
         openglut = ">= 0.6.3",
         freeglut = ">= 2.4.0",
         glut = ">=3.7.6"
      }
  }

The IUP dist.info might be expressed similarly.


> if LUADIST_ARCH == "Windows" then
>    dependencies["luadist_utils"] = ">=0.1"
> end


An alternative is to require luadist to depend on luadist_utils on all
platforms.  Then make the CMakeList.txt for luadist_utils do nothing
on all platforms except Windows.  More precisely, even on Windows,
these utilities may exist if Cygwin or MinGW binaries are in the PATH.
 Or consider if there were some other platform that might have all
commands except one.  We should have a FIND_PROGRAM in CMakeLists.txt
for each of these commands.  I think there's a consistency in this
approach.  The luadist_utils are required on all platforms, and cmake
has the ability to check for each of them and build any one that is
missing.


> Additionally this approach can be used to disable installation of
> certain components depending on user choice but would require rewriting
> of all existing dists. If our guide for dist maintainers contained
> conventions such as using of INSTALL_ variables for controlling
> installation or other distribution wide aspects we could build luadist
> with no documentation in cases when its desired by using
> -DINSTALL_DOC=false.


Not building/installing documentation may be useful.  Again, we can
solve this my splitting the dist into two (program + documentation) or
templating the dist.  We probably want to templatize.  Debug v.s.
Release builds are another case.

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

On 1-Sep-09, at 5:46 AM, David Manura wrote:

> ...
> A theoretical downside is that the dist.info format is now less
> declarative.  The set of all actual (i.e. evaluated) dist.info
> structures is only determinable by evaluating the dist.info as
> arbitrary Lua code for all values of the variables, and this set of
> values might be infinite.  This could have implications if I write a
> tool like search.cpan.org that doesn't install dists but rather
> indexes them and displays this information to the user.  For example,
> it's now less obvious for a program to determine the set of all dists
> that are or might be dependencies of a given dist.
>
> One workaround to the above problem is to require dists to enumerate
> all variables and possible values:
>
>  variable.USE_MOTIF = {false, true}
>  variable.USE_GTK = {false, true}
>
> Now, assuming the maintainer correctly defined these enumerations, we
> can make the above determination given a Lua sandbox.
Yes, I see your point. I don't see this as an issue as usually users  
don't care about dependencies when searching for modules and  LuaDist  
would "do the right thing" when they decide to install it. As for  
finding all the dependencies it  would be relatively easy to collect  
them all with pattern matching for the search portal avoiding all the  
problems. It would however be up to the maintainers to describe the  
"USE" behavior correctly.
>
> If this were in an object-oriented world, the IUP dist would accept
> any dist that provided a certain interface X, even dists unknown to
> IUP.  The Motif and GTK dists (or adapters for them) would provide
> this interface X.  LuaDist (not IUP) would find all dists that provide
> the interface X and choose the most appropriate one.  This might not
> be so straightforward in the case of IUP.  We might break up IUP into
> multiple dists: iup, iup-core, iup-gtk, iup-motif, etc. and define
> dependencies.  It might be much simpler to templatize the IUP dist --
> iup<gtk>, iup<motif>, etc. -- which in fact could be equivalent to
> what you are suggesting.  Note that this suggests possibly allowing a
> dist to depend on a particular instantiation:
>
>  dependencies = { ["iup<gtk>"] = ... }
>
> luaglut is another example where the dependencies are not well  
> specified:
>
>  dependencies = {
>      lua = ">= 5.1",
>      openglut = ">= 0.6.3",
>      freeglut = ">= 2.4.0",
>      glut = ">=3.7.6"
>  }
>
> That currently attempts to install all three glut dists.  Many users
> would probably be satisfied with LuaDist by default attempting to
> install each one and stopping once one successfully installs (e.g. if
> freeglut fails to compile, automatically try openglut).  I suggest
> providing for this behavior in whichever solution is chosen.  I would
> be fine with
>
>  dependencies = {
>      lua = ">= 5.1",
>      OR {
>         openglut = ">= 0.6.3",
>         freeglut = ">= 2.4.0",
>         glut = ">=3.7.6"
>      }
>  }
I like this idea, its more consistent with the current LD behavior  
which is try and see. Should be trivial to implement also. However  
there might be more "OR" segments involved in certain packages.

One way around it would be to provide "meta" packages eg: glut-meta.dist
...
dependencies={
	OR {
		openglut = ">= 0.6.3",
		freeglut = ">= 2.4.0",
		glut = ">=3.7.6"
	}
}

So your case in luaglut would end up being:

dependencies = {
	lua = ">=5.1",
	glut = "meta"
}

>> if LUADIST_ARCH == "Windows" then
>>   dependencies["luadist_utils"] = ">=0.1"
>> end
>
> An alternative is to require luadist to depend on luadist_utils on all
> platforms.  Then make the CMakeList.txt for luadist_utils do nothing
> on all platforms except Windows.  More precisely, even on Windows,
> these utilities may exist if Cygwin or MinGW binaries are in the PATH.
> Or consider if there were some other platform that might have all
> commands except one.  We should have a FIND_PROGRAM in CMakeLists.txt
> for each of these commands.  I think there's a consistency in this
> approach.  The luadist_utils are required on all platforms, and cmake
> has the ability to check for each of them and build any one that is
> missing.
Nice idea for dist_utils but there will be dists in the future where  
its certain they will not be available to other platforms. eg. cocoa  
on OSX, direct input on Windows etc. While the behavior you describe  
would work with source dists it would generate "empty" binary dists  
for other platforms since the CMake inside skips building it. I'd  
rather not even consider the dist on other platform if possible. Also  
when dealing with binary only dists this would lead to confusing user  
messages such as "luadist_utils not found, please install this package  
manually or using your systems package manager", because LD simply  
would not know its not needed.
>
>> Additionally this approach can be used to disable installation of
>> certain components depending on user choice but would require  
>> rewriting
>> of all existing dists. If our guide for dist maintainers contained
>> conventions such as using of INSTALL_ variables for controlling
>> installation or other distribution wide aspects we could build  
>> luadist
>> with no documentation in cases when its desired by using
>> -DINSTALL_DOC=false.
>
> Not building/installing documentation may be useful.  Again, we can
> solve this my splitting the dist into two (program + documentation) or
> templating the dist.  We probably want to templatize.  Debug v.s.
> Release builds are another case.
I would prefer to keep all things related to a package in one dist  
only. (doc, libs, headers, executable, data) as this simplifies things  
allot and keeps repo sizes manageable. To implement a filter on  
certain content differently we can use the dist deployment process  
which can filter out parts the user doesn't want without any CMake  
variables. eg. don't deploy /doc directory from a dist,this would work  
on binary dists too. The downside is that dists downloads are bigger  
than might be necessary, however less dists means less maintenance and  
we would still be true to the original goal to keep all dists usable  
without luadists so our work can be reused. Splitting packages into  
several parts seems counterproductive. Simply put I'd rather have less  
bigger dists than many small ones.

Debug and Release can be separated with two distinct binary trees with  
common architecture but one type being "x86" and other "x86-debug".

pd

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Tue, Sep 1, 2009 at 4:42 PM, Peter Drahos wrote:
> Nice idea for dist_utils but there will be dists in the future where its
> certain they will not be available to other platforms. eg. cocoa on OSX,
> direct input on Windows etc.

My concern is that this is redundant information.  Dist A probably
knows best that it can be deployed only on a set P of platforms
(especially after cmake checks).  If Dist B wants to allow using A,
then we are suggesting the B dist.info repeat the check for P, or at
least an easier-to-specify superset of P since dist.info
expressiveness is less than cmake and it doesn't hurt to try
additional platforms and fail.  If P changes, then all dists that use
A potentially need to be updated.  There is also a potential for B
getting P wrong (e.g. check for "Windows" but neglect to take into
account "sort-of Windows" nuances like Cygwin/MinGW).  Platform
restrictions are not always obvious: tee might exist on Windows (MinGW
in PATH), GTK might exist on Windows (win32 build) [2], and a Win32
API [1] might even exist on Linux (Wine), etc.  However, it might be
the case that even though two dists might both be deployable under two
platforms, different ones are much preferred on each, and even though
A might be deployable on Windows, B might still be incompatible with
the Windows version of A.

I think it's good to look at more actual examples.

[1] http://use.perl.org/articles/09/02/05/0422239.shtml
[2] http://www.tecgraf.puc-rio.br/iup/en/guide.html

> While the behavior you describe would work with source
> dists it would generate "empty" binary dists for other
> platforms since the CMake inside skips building it. I'd
> rather not even consider the dist on other platform if possible.
> Also when dealing with binary only dists
> this would lead to confusing user messages such as "luadist_utils not found,
> please install this package manually or using your systems package manager",
> because LD simply would not know its not needed.

The dist.info for luaexpat contains a dependency on libexpat.  When
installing luaexpat (in binary or source mode), how does LuaDist know
whether it needs to download libexpat?

> To implement a filter on certain content
> differently we can use the dist deployment process which can filter out
> parts the user doesn't want without any CMake variables. eg. don't deploy
> /doc directory from a dist,this would work on binary dists too.

That allows for "deploy only docs" but not "build only docs".
(However, most dists don't currently have complicated documentation
build procedures for this to matter much.)

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

Dňa 2.9.2009, o 6:12, David Manura <dm...@ma...> napísal:

> On Tue, Sep 1, 2009 at 4:42 PM, Peter Drahos wrote:
>> Nice idea for dist_utils but there will be dists in the future  
>> where its
>> certain they will not be available to other platforms. eg. cocoa on  
>> OSX,
>> direct input on Windows etc.
>
> My concern is that this is redundant information.  Dist A probably
> knows best that it can be deployed only on a set P of platforms
> (especially after cmake checks).  If Dist B wants to allow using A,
> then we are suggesting the B dist.info repeat the check for P, or at
> least an easier-to-specify superset of P since dist.info
> expressiveness is less than cmake and it doesn't hurt to try
> additional platforms and fail.  If P changes, then all dists that use
> A potentially need to be updated.  There is also a potential for B
> getting P wrong (e.g. check for "Windows" but neglect to take into
> account "sort-of Windows" nuances like Cygwin/MinGW).  Platform
> restrictions are not always obvious: tee might exist on Windows (MinGW
> in PATH), GTK might exist on Windows (win32 build) [2], and a Win32
> API [1] might even exist on Linux (Wine), etc.  However, it might be
> the case that even though two dists might both be deployable under two
> platforms, different ones are much preferred on each, and even though
> A might be deployable on Windows, B might still be incompatible with
> the Windows version of A.
>
> I think it's good to look at more actual examples.
>
> [1] http://use.perl.org/articles/09/02/05/0422239.shtml
> [2] http://www.tecgraf.puc-rio.br/iup/en/guide.html
>
Looking back on the comments, implementing just the suggested OR  
behaviour would be a better solution. So pets try that approach and  
roll back my last changes.
>> While the behavior you describe would work with source
>> dists it would generate "empty" binary dists for other
>> platforms since the CMake inside skips building it. I'd
>> rather not even consider the dist on other platform if possible.
>> Also when dealing with binary only dists
>> this would lead to confusing user messages such as "luadist_utils  
>> not found,
>> please install this package manually or using your systems package  
>> manager",
>> because LD simply would not know its not needed.
>
> The dist.info for luaexpat contains a dependency on libexpat.  When
> installing luaexpat (in binary or source mode), how does LuaDist know
> whether it needs to download libexpat?
>
It's simple, it always will when the dist is available. That's why we  
separated library repo from the luamodules repo. Simply disabling it  
will render libexpat unavailable and thus luadist will skip over it.  
To a
Provide more control over this we can add dist masking eg: luadist  
install expat -mask=libexpat, this would be usefull to skip over  
certain dists.
>> To implement a filter on certain content
>> differently we can use the dist deployment process which can filter  
>> out
>> parts the user doesn't want without any CMake variables. eg. don't  
>> deploy
>> /doc directory from a dist,this would work on binary dists too.
>
> That allows for "deploy only docs" but not "build only docs".
> (However, most dists don't currently have complicated documentation
> build procedures for this to matter much.)
Small price to pay, this would apply only to source dists. Extracting  
documentation from binary versions should be straightforward.

pd

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Wed, Sep 2, 2009 at 2:03 AM, Peter Drahos wrote:
> Dňa 2.9.2009, o 6:12, David Manura napísal:
>> The dist.info for luaexpat contains a dependency on libexpat.  When
>> installing luaexpat (in binary or source mode), how does LuaDist know
>> whether it needs to download libexpat?
> It's simple, it always will when the dist is available. That's why we
> separated library repo from the luamodules repo. Simply disabling it will
> render libexpat unavailable and thus luadist will skip over it. To a
> Provide more control over this we can add dist masking eg: luadist install
> expat -mask=libexpat, this would be usefull to skip over certain dists.

Attempting to install local versions of libraries by default before
attempting to link to system libraries might not be what all *nix
users prefer, but it is probably the safest thing to do without more
detailed knowledge of system library versions (e.g. integration with
apt/rpm).  The LuaRocks "external_dependencies" may be in the realm of
cmake responsibility.

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

On 3-Sep-09, at 7:26 AM, David Manura wrote:
> ...
> Attempting to install local versions of libraries by default before
> attempting to link to system libraries might not be what all *nix
> users prefer, but it is probably the safest thing to do without more
> detailed knowledge of system library versions (e.g. integration with
> apt/rpm).  The LuaRocks "external_dependencies" may be in the realm of
> cmake responsibility.
The overall plan was to disable the Libraries repository by default  
for unix systems  in order to reuse available host libraries. However  
I believe that leaving it enabled by default would render the system  
generally more robust as we cant test and debug with all host supplied  
libraries on all systems. It remains to be seen how this will work out  
in the end but testing so far suggests that making binary  
distributions of LD on unix platforms is possible without the  
Libraries repo. For Windows its best to rely on Libraries all the time.

pd

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Thu, Sep 3, 2009 at 6:16 PM, Peter Drahos wrote:
> On 3-Sep-09, at 10:27 AM, Peter Drahos wrote:
>> I'm planning to change this to strings in order to be able to write
>> ordered dependencies for the "OR" feature you suggested. Recent discussions
>> with PK about this we came up with the following suggestion:
>>
>> dependencies = {
>>        "lua >= 5.1",
>>        { -- OR
>>                "openglut >= 1.2", -- 1st candidate
>>                "freeglut >= 2.3", -- 2nd
>>                "glut >= 3.1", -- last
>>        }
>> }

Just a few comments tonight...  I came across a similar design
question when developing the SQL::Interp(olate) [1] Perl module as an
alternative to SQL::Abstract [2].  That is, how should we represent
Boolean and comparison operations as data?  The approach of
SQL::Abstract was to use curly braces (hashes) to represent logical
AND and square braces (arrays) to represent logical OR.  The problem
is that hashes can't have duplicate elements, and they don't preserve
order (thereby preventing short-circuiting as you've alluded to), so
this has semantic issues, and SQL::Abstract develops a workaround for
corner cases.  Another complaint noted in [1], I think by Stosberg,
was that this "syntax may impair readability because it uses the
subtle difference between a brace and bracket to denote the difference
between AND and OR".  Unlike Perl, Lua merges the concepts of hashes
and arrays, so we only have one type of bracket.  The convention you
apparently use is to alternate the meaning, so that { { { ... } } }
means AND(OR(AND(...))), or perhaps you only allow two levels.  That
also prohibits writing AND(AND(...)) and OR(OR(...)) constructions as
is, though any Boolean expression (at least without circuiting?) can
be expanded to product-of-sum canonical form.  I'm not certain if we
should pursue it, but one could go further down the stringification
route:

   dependencies = [[
        lua >= 5.1 and ( openglut >= 1.2 or
                                freeglut >= 2.3 or
                                glut >= 3.1 )
    ]]


[1] http://search.cpan.org/dist/SQL-Interp/lib/SQL/Interp.pm#SQL::Abstract
(b) http://search.cpan.org/dist/SQL-Interp/lib/SQL/Interp.pm#Philosophy
[2] http://search.cpan.org/~ribasushi/SQL-Abstract/lib/SQL/Abstract.pm#Logic_and_nesting_operators

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Mon, Aug 31, 2009 at 11:46 PM, David Manura wrote:
> On Sun, Aug 30, 2009 at 10:51 AM, Peter Drahos wrote:
>> if LUADIST_ARCH == "Windows" then
>>    dependencies["luadist_utils"] = ">=0.1"
>> end
> An alternative is to require luadist to depend on luadist_utils on all
> platforms.

On a look through ( http://luarocks.luaforge.net/rocks/ ), I see that
although LuaRocks does use platform-specific build sections (analogous
to CMakeLists.txt), no modules currently use platform specific
distribution dependencies.

It is plausible that one might prefer a platform specific module
(luaposix) over a generic one (luafilesystem).  The question is who
should select the module.  Note that it's not essential to this
selection to have platform specific information in the dist.info.  The
dist.info could, for example, indicate to attempt installation of
luaposix first (which may immediately fail on some platforms) and on
failure resort to luafilesystem.

  dependencies = {
    lua = ">= 5.1",
    OR {
      {luaposix => "..."},
      {luafilesystem => "..."}
    }
  }

(BTW, I would like to see the dist.info format eventually be a subset
of the rockspec format.)

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

On 1-Sep-09, at 6:48 AM, David Manura wrote:

> ...
> On a look through ( http://luarocks.luaforge.net/rocks/ ), I see that
> although LuaRocks does use platform-specific build sections (analogous
> to CMakeLists.txt), no modules currently use platform specific
> distribution dependencies.
>
> It is plausible that one might prefer a platform specific module
> (luaposix) over a generic one (luafilesystem).  The question is who
> should select the module.  Note that it's not essential to this
> selection to have platform specific information in the dist.info.  The
> dist.info could, for example, indicate to attempt installation of
> luaposix first (which may immediately fail on some platforms) and on
> failure resort to luafilesystem.
>
>  dependencies = {
>    lua = ">= 5.1",
>    OR {
>      {luaposix => "..."},
>      {luafilesystem => "..."}
>    }
>  }
>
I would leave these question on the maintainers of the packages  
involved as they would probably have better insight into their  
internals as I do. As mentioned in the previous post we could create a  
"meta" package instead.

As I yet again fail to provide any luadist roadmap I'd like to point  
out that in order to solve these issues we would like to take a  
different road instead. Once the workings of LD are established and  
version 1.0.0 is more or less stable we are planning to start work on  
"Standard Library" repository similar to your idea with[1] where we  
would pick up code from available MIT dists and modify their naming,  
API and documentation to follow more strict standards. This would  
include splitting of functionality into more dists or merging similar  
dists. In case of the filesystem library we would merge code available  
in lfs with bindings to win32 api or posix when apropriate.

> (BTW, I would like to see the dist.info format eventually be a subset
> of the rockspec format.)

I don't know if we can keep it a subset but we can certainly make a  
generator .. dist2rock perhaps.

pd

[1] http://lua-users.org/wiki/StandardLibraries

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Tue, Sep 1, 2009 at 5:13 PM, Peter Drahos wrote:
>> (BTW, I would like to see the dist.info format eventually be a subset
>> of the rockspec format.)
>
> I don't know if we can keep it a subset but we can certainly make a
> generator .. dist2rock perhaps.

There are ways to reduce the difference.  For example, LuaRock's
"summary" field is called "short" in LuaDist, and "detailed" is called
"full".

It's not clear to me yet exactly what the inherent differences are and
in which cases, dist2rock(rock2dist(dist1)) == dist1, for example.
There's "external_dependencies", but many rocks don't use this.

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

David Manura wrote:
> On Thu, Sep 3, 2009 at 6:16 PM, Peter Drahos wrote:
>   
> ...
> Just a few comments tonight...  I came across a similar design
> question when developing the SQL::Interp(olate) [1] Perl module as an
> alternative to SQL::Abstract [2].  That is, how should we represent
> Boolean and comparison operations as data?  The approach of
> SQL::Abstract was to use curly braces (hashes) to represent logical
> AND and square braces (arrays) to represent logical OR.  The problem
> is that hashes can't have duplicate elements, and they don't preserve
> order (thereby preventing short-circuiting as you've alluded to), so
> this has semantic issues, and SQL::Abstract develops a workaround for
> corner cases.  Another complaint noted in [1], I think by Stosberg,
> was that this "syntax may impair readability because it uses the
> subtle difference between a brace and bracket to denote the difference
> between AND and OR".  Unlike Perl, Lua merges the concepts of hashes
> and arrays, so we only have one type of bracket.  The convention you
> apparently use is to alternate the meaning, so that { { { ... } } }
> means AND(OR(AND(...))), or perhaps you only allow two levels.  That
> also prohibits writing AND(AND(...)) and OR(OR(...)) constructions as
> is, though any Boolean expression (at least without circuiting?) can
> be expanded to product-of-sum canonical form.  I'm not certain if we
> should pursue it, but one could go further down the stringification
> route:
>
>    dependencies = [[
>         lua >= 5.1 and ( openglut >= 1.2 or
>                                 freeglut >= 2.3 or
>                                 glut >= 3.1 )
>     ]]
>   
I'm not sure it will be necessary to go that far in near future .. it 
could be added later as an alternative to the table format if theres 
need for it. I'm trying to keep LD as simple as possible and I already 
see the current "OR" implementation as ugly but maybe necessary evil. 
However I think we could solve this whole issue by other means. We could 
simply support only one "recomended" external library when there are 
more options (eg: only openglut in luaglut), people that want to use 
other option can simply install it manually or explicitly and turn off 
deps when building the package. Dists cmake files would still have to be 
written with all the options in mind.

pd

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Fri, Sep 4, 2009 at 3:34 PM, Peter Drahos wrote:
> I'm not sure it will be necessary to go that far in near future .. it could
> be added later as an alternative to the table format if theres need for it.
> I'm trying to keep LD as simple as possible and I already see the current
> "OR" implementation as ugly but maybe necessary evil.
> However I think we
> could solve this whole issue by other means. We could simply support only
> one "recomended" external library when there are more options (eg: only
> openglut in luaglut), people that want to use other option can simply
> install it manually or explicitly and turn off deps when building the
> package. Dists cmake files would still have to be written with all the
> options in mind.

I think there are good uses for the OR logic

 - Platform specific libraries (libgtk2 or win32)
 - Alternate C and pure Lua libraries (e.g. struct, bit, xml, etc.)
 - Alternate versions of a library (version 1.x and 2.x)
 - Alternate implementations (pcre or posix re)

I'd prefer that complex builds like "luadist install iup" just work
with suitable defaults and not require manual fiddling with
configuration options.

It's probably worthwhile to research the Debian .deb format
specifications [1] for comparison.  For example, [2] lists some
examples with alternation and negation:

  Depends: libc6 (>= 2.2.1), exim | mail-transport-agent

  Build-Depends: foo [!i386] | bar [!amd64]

I think we may also be able to treat platforms and architectures in a
consistent way as packages.  For example, "windows >= 5.1" would be
Windows XP or above.  This is not too unlike the "lua >= 5.1" usage.
I think we can express iup dependencies as something like

  dependencies = {
    "cd >= 5.1.1",
    "im >= 3.4.1",
    {
      "windows",
      "libgtk2 >= 2.0",
      {"libmotif >= 2.1", "libxpm >= 3.5.7", "libxp >= 1.0.0"}
         -- note: we might avoid this deep nesting by splitting
         -- iup-motif into a separate package
    }
    -- "libcairo2 >= 1.8.0" gtk?
  }

A debian-like syntax would be more concise though.  Something like

  dependencies = "cd (>=5.1.1), im (>=3.4.1), windows | libgtk2 (>=2.0) | ..."

[1] http://en.wikipedia.org/wiki/Deb_%28file_format%29
[2] http://www.debian.org/doc/debian-policy/ch-relationships.html

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Sat, Sep 5, 2009 at 2:10 AM, David Manura wrote:
> It's probably worthwhile to research the Debian .deb format
> specifications [1] for comparison.  For example, [2] lists some
> examples with alternation and negation:

There's an interesting analysis of existing package systems in [3],
particularly concerning dependency definition.

For example, deb allows ORed (disjunctive) dependencies, without deep
nesting.  The Portage system allows complicated nested logic
expressions.  RPM does not allow ORed dependencies, but RPM (as well
as deb and Portage) can achieve a similar affect using a "virtual
packages"/"provides" capability.  An example of how virtual packages
might work in LuaDist is for the luajit dist to be marked as providing
the lua virtual package (provides = {"lua 5.1"}).  Then any package
containing depends={"lua >= 5.1"} would also work with luajit.

Those notes also have some proposals like allowing

  Package: hevea
  Build-Depends: ocaml-compiler == N, ...
  Depends: ocaml-runtime-system == N, ...
  Where: N >= 3.0

which means that the package hevea depends on both ocaml-compiler (for
building) and ocaml-runtime-system, with the additional constraint
that the version numbers of both must be the same (N) and at least
3.0.

[3] http://www.mancoosi.org/edos/index.html

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Sat, Sep 5, 2009 at 2:12 PM, David Manura wrote:
> There's an interesting analysis of existing package systems in [3],
> particularly concerning dependency definition.

The Arch system [1], based on BSD ports, looks worthwhile to look at.
See its PKGBUILD format [2].

Its "depends" field is just a simple list like in LuaDist.  Instead of
OR logic it supports virtual packages.  It doesn't support negation in
depends either because there is a separate list for that
("conflicts"), and there are additional lists for other things (e.g.
makedepends, optdepends, provides, and replaces).  I like how the
optdepends lists contains a short description of why each dependency
is optional (e.g. 'cups: printing support'), for this could be
displayed from the user interface.

The packages [3] are maintained in an svn tree [4].  Only the package
definition + patches are maintained in the svn tree (e.g. [5]), not
the full source code, which is instead referenced by a URL and md5
checksum (like luarocks).  That keeps the svn tree small.
Furthermore, keeping these separate seems common in package management
systems (including LuaRocks).  Personally, I found upgrading the
sources for LuaDist packages slightly confusing because it's not
immediately clear which files belong to the original sources and which
were added/patched by the dist package maintainer.  I also like how
the package definitions can be viewed from the svn web interface
without extracting [6], like LuaRocks [7] but unlike LuaDist [8].

So, Arch may be a reasonable, well tested, and not-too-different model
to base LuaDist and LuaRocks off of, especially if you want to keep
things simple.  (Search for "lua" in [3,5] to see examples of Lua arch
packages.)

[1] http://en.wikipedia.org/wiki/Arch_Linux
[2] http://www.archlinux.it/wiki/index.php?title=Building_Packages_in_Arch_Linux
[3] http://www.archlinux.org/packages/
[4] http://www.archlinux.org/svn/
[5] http://repos.archlinux.org/viewvc.cgi/lua/trunk/
[6] http://repos.archlinux.org/viewvc.cgi/lua/trunk/PKGBUILD?revision=42988&view=markup
[7] http://luarocks.luaforge.net/rocks/luafilesystem-1.4.2-1.rockspec
[8] http://luadist.org/repo/luadist/LuaUtils/lua-5.1.4.dist

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Sat, Sep 5, 2009 at 5:06 PM, David Manura wrote:
> The Arch system [1], based on BSD ports, looks worthwhile to look at.
> See its PKGBUILD format [2].

The Arch PKGBUILD file is a bash script.  There was a proposal to
rework it into a format that is more easily parseable and represent
nested data structures better [9], which interesting enough actually
resembles Lua syntax, and Lua may be a good candidate for what they
are looking for.  One feature that's apparently occasionally used is
bash conditional statements to apply platform-specific dependencies
[9-11] and customizations in the manner you originally suggested:

  depends=('foo' 'baz')
  [ "$(uname -m)" == 'x86_64' ] && depends=('lib32-foo' 'lib32-baz')

  [ "$CARCH" = "x86_64" ] && source=(${source[@]} 'other source')
  [ "$CARCH" = "x86_64" ] && md5sums=(${md5sums[@]} 'other md5sum')

Bash syntax is detrimental to parsing though [9][11].  [9] proposes
the more LuaRocks-like syntax of

  package {
    name foo,
    version 1.3,
    depends { depa, depb, depc },
    arch {
      i686 { depends { depd, depe } },
      x86_64 { depends { lib32-depd, lib32-depe } }
    },

These both contrast to the .deb approach mentioned earlier:

  Build-Depends: foo [!i386] | bar [!amd64]

[9] http://xyne.archlinux.ca/ideas/pkgmeta
[10] http://wiki.archlinux.org/index.php/Arch64_FAQ
[11] http://www.mail-archive.com/au...@ar.../msg00583.html

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

David Manura wrote:
> On Sat, Sep 5, 2009 at 2:12 PM, David Manura wrote:
>   
>> There's an interesting analysis of existing package systems in [3],
>> particularly concerning dependency definition.
>>     
>
> The Arch system [1], based on BSD ports, looks worthwhile to look at.
> See its PKGBUILD format [2].
>   
Thank you for the extensive analysis of the problem, I was already 
looking how macports/bsdports do things. I really like the archlinux 
system as its very close to the current implementation.
> Its "depends" field is just a simple list like in LuaDist.  Instead of
> OR logic it supports virtual packages.  It doesn't support negation in
> depends either because there is a separate list for that
> ("conflicts"), and there are additional lists for other things (e.g.
> makedepends, optdepends, provides, and replaces).  I like how the
> optdepends lists contains a short description of why each dependency
> is optional (e.g. 'cups: printing support'), for this could be
> displayed from the user interface.
>   
I think this approach plays very well with the goals we have although 
for the time being we probably wont make heavy use of the features so 
for version 1.0.0 I'd like to go and implement only a subset of it. 
However following these changes will further widen the gap with the 
luarocks format.
> The packages [3] are maintained in an svn tree [4].  Only the package
> definition + patches are maintained in the svn tree (e.g. [5]), not
> the full source code, which is instead referenced by a URL and md5
> checksum (like luarocks).  That keeps the svn tree small.
> Furthermore, keeping these separate seems common in package management
> systems (including LuaRocks).  Personally, I found upgrading the
> sources for LuaDist packages slightly confusing because it's not
> immediately clear which files belong to the original sources and which
> were added/patched by the dist package maintainer.  I also like how
> the package definitions can be viewed from the svn web interface
> without extracting [6], like LuaRocks [7] but unlike LuaDist [8].
>   
We can of course create dists that only contain dist.info and 
CMakeLists.txt and patches in uncompressed form inside SVN. CMake can 
download the sources from any source (similary to the current svn dists 
like luarocks-svn). The reason why we prefer to keep all files in a 
single dist file with sources is that they can simply be downloaded and 
used without LD with no additional resources (other than its 
dependencies) needed. I agree that the approach may be confusing 
compared to other package managers but it has its advantages, for 
example resources being referenced may not be available for download or 
they may change at any time breaking the build.
> So, Arch may be a reasonable, well tested, and not-too-different model
> to base LuaDist and LuaRocks off of, especially if you want to keep
> things simple.  (Search for "lua" in [3,5] to see examples of Lua arch
> packages.)
>   
The approach taken by Arch may very well be the most reasonable system 
to borrow ideas from. Seeing LuaDist is somewhat on new ground as it 
completely relies on CMake and strives to unify the differences between 
multiple platforms, architectures, binary types and compilers we may be 
forced to do things differently. Compared to LD most package managers 
are usually specific only to certain platform (Linux, BSD, Mac etc.), 
few types and specific compiler.

pd

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Sun, Sep 6, 2009 at 9:57 AM, Peter Drahos wrote:
> I think this approach plays very well with the goals we have although for
> the time being we probably wont make heavy use of the features so for
> version 1.0.0 I'd like to go and implement only a subset of it. However
> following these changes will further widen the gap with the luarocks format.

LuaRocks might be open to some changes at this time.  A upcoming
LuaRocks 2.0 was mentioned on the LuaRocks list, and rockspecs appear
to currently use only a subset of the features in the specification
[12].  I didn't see any rockspecs using "supported_platforms", "~=",
"~> 2.4", nor ">= 2.4, < 2.5" styles.  Styles in use include

  "luafilesystem cvs",
  "lua >= 5.1",
  "luasocket",
  "loop == 2.3beta",
  "luaidl == 0.8.9beta",
  "lposix >= 20031107-1",
  "lua-iconv >= r3",

There is a mapping [14] between Haskell packages [13] and Arch, which
Lua could emulate.  A large cross-reference of mapped packages is at
[17].

Haskell cabal supports a "foo >= 1.2 && < 1.3" syntax (also "||") and
a shorthand "foo ==1.2.*" similar to the "~> 1.2" feature in LuaRocks.
 As I understand, Arch supports installing only a single version of a
package at a time and occasionally a workaround is needed of giving a
package two different names (foo and foo2).  The version constraints
supported by Arch seem to be looser than those supported by Haskell.

The Haskell cabal includes fields for compiling C source code. The
cabal file for zlib [16] or bzip2 is instructive: it conditionally
links to libz on non-Windows or builds C sources on Windows.

See also [15] from Haskell, which gives a more formal specification of
the Arch PKGBUILD format.

> Seeing LuaDist is somewhat on new ground as it completely
> relies on CMake and strives to unify the differences between multiple
> platforms, architectures, binary types and compilers we may be forced to do
> things differently. Compared to LD most package managers are usually
> specific only to certain platform (Linux, BSD, Mac etc.), few types and
> specific compiler.

Yes, LuaDist is more generally a CMake-based distribution.  It is not
limited to Lua.  For example, one might bootstrap Python [18] with it.

[12] http://www.luarocks.org/en/Rockspec_format
[13] http://www.haskell.org/ghc/docs/latest/html/Cabal/authors.html
[14] http://wiki.archlinux.org/index.php/Haskell_package_guidelines
[15] http://hackage.haskell.org/packages/archive/archlinux/0.2.3/doc/html/Distribution-ArchLinux-PkgBuild.html
[16] http://hackage.haskell.org/packages/archive/zlib/0.5.2.0/zlib.cabal
[17] http://www.galois.com/~dons/arch-haskell-status.html
[18] http://mail.python.org/pipermail/python-dev/2007-July/073912.html

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

David Manura wrote:
> On Sun, Sep 6, 2009 at 9:57 AM, Peter Drahos wrote:
>   
>> I think this approach plays very well with the goals we have although for
>> the time being we probably wont make heavy use of the features so for
>> version 1.0.0 I'd like to go and implement only a subset of it. However
>> following these changes will further widen the gap with the luarocks format.
>>     
>
> LuaRocks might be open to some changes at this time.  A upcoming
> LuaRocks 2.0 was mentioned on the LuaRocks list, and rockspecs appear
> to currently use only a subset of the features in the specification
> [12].  I didn't see any rockspecs using "supported_platforms", "~=",
> "~> 2.4", nor ">= 2.4, < 2.5" styles.  Styles in use include
>   
>   "luafilesystem cvs",
>   "lua >= 5.1",
>   "luasocket",
>   "loop == 2.3beta",
>   "luaidl == 0.8.9beta",
>   "lposix >= 20031107-1",
>   "lua-iconv >= r3",
>   
Also "author" field is missing (which is disrespectful). I don't like 
the rockspec format very much, it includes some completely unnecessary 
fields, parentheses and concentrates too much on the "build" section 
which is obsolete for LD. Main reason for starting our own deployment in 
LD was to get rid of all the complexity LR accumulates to reinvent the 
wheel. CMake fills that spot pretty well and generally offers more 
flexibility (independent packages, multiple compilers, cross-compilation 
etc).
> There is a mapping [14] between Haskell packages [13] and Arch, which
> Lua could emulate.  A large cross-reference of mapped packages is at
> [17].
>   
Great find thanks.
> Haskell cabal supports a "foo >= 1.2 && < 1.3" syntax (also "||") and
> a shorthand "foo ==1.2.*" similar to the "~> 1.2" feature in LuaRocks.
>  As I understand, Arch supports installing only a single version of a
> package at a time and occasionally a workaround is needed of giving a
> package two different names (foo and foo2).  The version constraints
> supported by Arch seem to be looser than those supported by Haskell.
>   
I like the version constraints with operators, easy to read and write. 
Arch is currently the top model I consider to follow as its goals are 
closest in line with LD of all the package managers available. I would 
like to keep support for only single version of dists as our experiments 
with multiple versions proved to be a nightmare when dealing with 
various dynamic linkers between platforms. Splitting the dist into 
foo/foo2 is accepted solution. We could probably write a tool that 
creates dists from most of its packages automatically which would be 
even simpler if we follow naming conventions in Arch (and Arch and other 
package managers can benefit from our dists too).
> The Haskell cabal includes fields for compiling C source code. The
> cabal file for zlib [16] or bzip2 is instructive: it conditionally
> links to libz on non-Windows or builds C sources on Windows.
>
> See also [15] from Haskell, which gives a more formal specification of
> the Arch PKGBUILD format.
>   
I don't think that the package info files should act as "makefiles". It 
would be reinventing the wheel all over again .. thats a job for CMake. 
I like the rest of the format especially the syntax for constraints. 
However LD should just resolve deps, fetch, unpack, call cmake and 
deploy/remove.
>> Seeing LuaDist is somewhat on new ground as it completely
>> relies on CMake and strives to unify the differences between multiple
>> platforms, architectures, binary types and compilers we may be forced to do
>> things differently. Compared to LD most package managers are usually
>> specific only to certain platform (Linux, BSD, Mac etc.), few types and
>> specific compiler.
>>     
>
> Yes, LuaDist is more generally a CMake-based distribution.  It is not
> limited to Lua.  For example, one might bootstrap Python [18] with it.
>   
Yeah, PK used it as a package manager on Windows to install libs he 
needed. However he now uses Ubuntu so his motivation to write dists for 
libraries is low. I can see LD expanding beyond lua if there is 
interest, filling the repo can be very quick if we write a tool that 
converts available Arch(or other) packages to dists, manual work would 
still be needed to tweak generated CMakeLists.txt files.

Thanks to your great pointers and research, I'll start working forward 
to follow Arch later this week.

pd

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Tue, Sep 8, 2009 at 3:02 AM, Peter Drahos wrote:
> the rockspec format...
> concentrates too much on the "build" section which is
> obsolete for LD. Main reason for starting our own deployment in LD was to
> get rid of all the complexity LR accumulates to reinvent the wheel. CMake
> fills that spot pretty well and generally offers more flexibility
> (independent packages, multiple compilers, cross-compilation etc).

I think there is still a strong desire for autoconf builds, especially
given that most packages have autoconf builds already, an autoconf
build is often better than nothing, and the translation to CMake takes
some time.

LuaRocks (via the "command" build type) and Linux Arch are not tied to
any one build system.  Both can run arbitrary build commands, such as
autoconf, CMake, or a custom build system.  The same might hold for
LuaDist in a way too (calling autoconf+make via a thin CMake
"execute_process" wrapper), but LuaDist doesn't currently allow
bypassing cmake.

If a package has more than one build system, I think the user should
be offered a choice.  Currently LuaRocks doesn't allow defining more
than one build method (autoconf v.s. cmake) in a standard way, but I
was thinking of suggesting that this restriction be removed.

> We could probably write a tool that creates dists from most of its packages
> automatically which would be even simpler if we follow naming conventions in
> Arch (and Arch and other package managers can benefit from our dists too).
...
> I can see LD expanding beyond lua if there is interest, filling the
> repo can be very quick if we write a tool that converts available Arch(or
> other) packages to dists, manual work would still be needed to tweak
> generated CMakeLists.txt files.

By default, that tool might invoke the current build system (as above)
until a CMakeLists.txt can be defined.

> I don't think that the package info files should act as "makefiles". It
> would be reinventing the wheel all over again .. thats a job for CMake. I
> like the rest of the format especially the syntax for constraints. However
> LD should just resolve deps, fetch, unpack, call cmake and deploy/remove.

Maybe.  On the other hand, the CMakeLists.txt for luafilesystem is
relatively verbose compared to the LuaRocks "module" build type [19].
See also the "rockspec-to-Makefile converter" [20]  It would be easy
to write a rockspec-to-CMakeLists.txt converter likewise.

Do you have any ideas for how cpack would be supported [21] in the
LuaDist framework?

[19] http://luarocks.luaforge.net/rocks/luafilesystem-1.4.2-1.rockspec
[20] http://lists.luaforge.net/pipermail/luarocks-developers/2009-September/001181.html
[21] http://lua-users.org/lists/lua-l/2009-09/msg00121.html

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Wed, Sep 9, 2009 at 1:47 AM, David Manura  wrote:
> ...

MacPorts [22] is another (referenced recently on the LuaRocks list
[28]).  Particularly, see examples of Lua modules [23].  Some things
that stand out:

  - The Portfile is a TCL script.

  - No version numbers on "ports" (similar to Arch Linux).

  - Distinguishes build and runtime dependencies
    (like various other systems).

  - Supports both port and file dependencies (section 5.4.1 [22]).
    A file dependency can be written as "lib:libX11.6:xorg",
    which means search for the system library file libX11.6.so
    and failing that install the port "xorg".

  - Does not support virtual packages or ORed dependencies,
    except maybe in the restricted sense above, although there
    have been requests for this [27].

  - Supports the concept of "variants" (see [24] and examples
    in [23]).  For example, the port for luarocks  [25] contains
    "curl" and "wget" variants, which if selected append the
    dependencies "port:curl" and "port:wget" respectively
    and adjust build options.  Variants also are a workaround
    for the lack of virtual packages or ORed dependencies
    (e.g. variants for postgresql83 or postgresql84).
    The above are "user configurable variants". "Platform variants"
    (5.5.3) form another type of variant that are selected
    automatically based on platform.  GIMP [26] applies
    dependencies based on platform and conditional logic.
    Each variant has a description and is displayed in the
    web view [23].

  - The source is external and checksummed.  If downloading
    from the original location fails, it resorts to a "MacPorts
    Project mirror" (5.3.3). [29]

  - The sources can be patched to adapt it to the port
    (like various other systems).
    Patches can also be simple regex substitutions.

  - The build is largely external to the port, other than the port
     configuring options passed to configure and make (this is
     like various other systems).  The build type (build.type) is
     configurable to gnumake or bsdmake, or you can set
     an arbitrary build command (build.cmd defaults to "make").
     For example, Python modules use build.cmd =
     "${python.bin} setup.py --no-user-cfg".

In the case of [28], it was efficient to fork the repository because
only Portfile's (not source) are stored in the repository.  The
concern about links becoming unavailable can be addressed in the
MacPorts manner [29] by LuaDist maintaining at least two separate
repositories: a dist.info repository and a source repository.

[22] http://guide.macports.org/#reference.dependencies
[23] http://www.macports.org/ports.php?by=name&substr=lua
[24] http://guide.macports.org/#reference.variants
[25] http://trac.macports.org/browser/trunk/dports/devel/luarocks/Portfile
[26] http://trac.macports.org/browser/trunk/dports/graphics/gimp/Portfile
[27] http://trac.macports.org/ticket/16336
[28] http://lists.luaforge.net/pipermail/luarocks-developers/2009-September/001189.html
[29] http://trac.macports.org/wiki/Mirrors

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[Peter Drahos <dr...@gm...>]

On Sep 16, 2009, at 4:28 AM, David Manura wrote:

> On Wed, Sep 9, 2009 at 1:47 AM, David Manura  wrote:
> ...
> In the case of [28], it was efficient to fork the repository because
> only Portfile's (not source) are stored in the repository.  The
> concern about links becoming unavailable can be addressed in the
> MacPorts manner [29] by LuaDist maintaining at least two separate
> repositories: a dist.info repository and a source repository.
I looked into MacPorts few months ago and actually use it on my OSX  
machines. Its a mature system as well but I'm following Arch for the  
upcoming changes (mail to follow later this week).

  As for the dist repository issue I would like to propose following  
approach:
- Store unpacked dist.info, CMakeLists.txt, patches + extra files  
(def, doc, etc) in repo SVN.
- Add reference to original sources to dist.info (similarly to  
homepage entry) eg, "source = 'http://path/to/source.tar.gz'".
- Write a cron that will download the sources by looking into  
dists.info and pack them periodically (optionally it can apply the  
patches as well) into the public repository.

This approach would be flexible for the developers as the repo tree  
would be pretty clean and updating to higher versions would only  
require to modify the dist.info entries. Above all, packed dists would  
remain self contained with no need for source mirrors (one of primary  
goals).

pd

Re: [LuaDist-devel] Support for conditional evaluation of dist.info

[David Manura <dm...@ma...>]

On Wed, Sep 16, 2009 at 7:39 AM, Peter Drahos wrote:
>  As for the dist repository issue I would like to propose following
> approach:
> - Store unpacked dist.info, CMakeLists.txt, patches + extra files (def, doc,
> etc) in repo SVN.
> - Add reference to original sources to dist.info (similarly to homepage
> entry) eg, "source = 'http://path/to/source.tar.gz'".
> - Write a cron that will download the sources by looking into dists.info and
> pack them periodically (optionally it can apply the patches as well) into
> the public repository.
> This approach would be flexible for the developers as the repo tree would be
> pretty clean and updating to higher versions would only require to modify
> the dist.info entries. Above all, packed dists would remain self contained
> with no need for source mirrors (one of primary goals).

Yes, that sounds like a reasonable approach and more in-line with
other ports systems.  The cron job might alternately be replaced with
an svn hook [30].

[30] http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.ref.reposhooks