Octave Forge

Hi

tir, 21 12 2010 kl. 15:50 -0700, skrev Orion Poplawski:
> I'm starting to take a look at creating a standard method of building rpms of 
> octave packages for Fedora.  I'm starting with the old octave-forge package as 
> a starting point.

I don't really know that much about RPM's, so I can't offer much helpful
insight. David Bateman did some work on autogenerating RPM spec files
for packages. Did you look at this?

> Although, now that I look at the octave-forge repo, I don't see a single 
> package that has an on_unistall.m file.  Could we reserve this for distro use?

Can't you just add an 'on_rpm_uninstall.m' and do a 

  echo "on_rpm_uninstall" >> on_uninstall.m

to ensure that this script is called?

> Finally, when a user tries to remove and rpm install octave package, they get:
> 
> octave:3> pkg uninstall octcdf
> warning: some of the packages you want to uninstall are not installed
> 
> which does not seem correct.

Hmm, I guess we should check if the package in question is installed by
'root' and adapt the error message accordingly. We could say something
like: "warning: you do not have the necessary privileges to remove
package 'XXX'". Could you open a bug report on http://bugs.octave.org
for this?

Søren

On 21 December 2010 16:50, Orion Poplawski <orion@...> wrote:
> I'm starting to take a look at creating a standard method of building rpms
> of octave packages for Fedora.  I'm starting with the old octave-forge
> package as a starting point.

Did you see these already?

     http://git.debian.org/?p=pkg-octave/octave-pkg-dev.git;a=tree;h=c93c63e13b6d75feabd5e11329474f5e06439625;hb=c93c63e13b6d75feabd5e11329474f5e06439625

I'm not sure how easily could you turn that into a .spec files... But
the overall structure of a Makefile with targets and is similar to the
targets of a .spec. It doesn't address your concerns about end users
mixing Octave's pkg function and the system package manager, though.
On Debian, if I call pkg uninstall on a Debian-installed 'Forge
package, pkg reports that the package I want to uninstall isn't
installed. This seems sort of ok to me.

HTH,
- Jordi G. H.

Orion Poplawski wrote:
> I'm starting to take a look at creating a standard method of building rpms of
> octave packages for Fedora.  I'm starting with the old octave-forge package as
> a starting point.
:
<snip>
:
>
> Finally, when a user tries to remove and rpm install octave package, they get:
>
> octave:3>  pkg uninstall octcdf
> warning: some of the packages you want to uninstall are not installed
>
> which does not seem correct.
>
> Thoughts?

Perhaps no answer to your question but I can share some user experiences.

This (warning messages) is what I also see when I try to 
uninstall/install (renew/update) individual octave-forge packages in 
Mandriva (2010.1).
Mandriva has made renewal/updating individual octave-forge packages a 
cumbersome process, with unexpected dependencies all over the place, 
perhaps due to the still "monolithic" character of its octave-forge rpm, 
and rpm and (octave's) pkg stepping onto each other's toes.

Hopefully you can make it much easier for Fedora users, up to the point 
that rpm isn't needed (or only needed for initial installation), i.e., 
so that just Octave's pkg command will do. (I'd prefer pkg over rpm in 
this case.)
I think development of octave-forge packages proceeds more rapidly than 
rpm's can be built, tested & updated - if you automate building rpms 
from individual octave-forge packages this can be alleviated a bit.
Thus either Q-control by Fedora will be hampered, or octave users will 
be withheld a bit from trying the newest octave-forge packages (in turn, 
hampering package development) if they must wait until tested rpms 
become available.

FYI, "pkg" has an option "forge" that will d/l & install octave-forge 
packages directly from octave.sf.net. Permissions (root) may interfere 
when packages go into /usr/[local/]share, but that's probably resolvable.

P.

On 12/21/2010 4:01 PM, Søren Hauberg wrote:
> Hi
>
> tir, 21 12 2010 kl. 15:50 -0700, skrev Orion Poplawski:
>> I'm starting to take a look at creating a standard method of building rpms of
>> octave packages for Fedora.  I'm starting with the old octave-forge package as
>> a starting point.
> I don't really know that much about RPM's, so I can't offer much helpful
> insight. David Bateman did some work on autogenerating RPM spec files
> for packages. Did you look at this?
>
I suspect that his work is the basis for much of what is already in 
place, and what I'm basing my current templates on.
>> Although, now that I look at the octave-forge repo, I don't see a single
>> package that has an on_unistall.m file.  Could we reserve this for distro use?
> Can't you just add an 'on_rpm_uninstall.m' and do a
>
>    echo "on_rpm_uninstall">>  on_uninstall.m
>
> to ensure that this script is called?
>
That may be a bit simpler to implement.  Thanks.
>> Finally, when a user tries to remove and rpm install octave package, they get:
>>
>> octave:3>  pkg uninstall octcdf
>> warning: some of the packages you want to uninstall are not installed
>>
>> which does not seem correct.
> Hmm, I guess we should check if the package in question is installed by
> 'root' and adapt the error message accordingly. We could say something
> like: "warning: you do not have the necessary privileges to remove
> package 'XXX'". Could you open a bug report on http://bugs.octave.org
> for this?
>
> Søren
Done: https://savannah.gnu.org/bugs/index.php?31946

- Orion

On 12/21/2010 4:14 PM, Jordi Gutiérrez Hermoso wrote:
> On 21 December 2010 16:50, Orion Poplawski<orion@...>  wrote:
>> I'm starting to take a look at creating a standard method of building rpms
>> of octave packages for Fedora.  I'm starting with the old octave-forge
>> package as a starting point.
> Did you see these already?
>
>       http://git.debian.org/?p=pkg-octave/octave-pkg-dev.git;a=tree;h=c93c63e13b6d75feabd5e11329474f5e06439625;hb=c93c63e13b6d75feabd5e11329474f5e06439625
>
> I'm not sure how easily could you turn that into a .spec files... But
> the overall structure of a Makefile with targets and is similar to the
> targets of a .spec. It doesn't address your concerns about end users
> mixing Octave's pkg function and the system package manager, though.
> On Debian, if I call pkg uninstall on a Debian-installed 'Forge
> package, pkg reports that the package I want to uninstall isn't
> installed. This seems sort of ok to me.
>
> HTH,
> - Jordi G. H.
Thanks, I'll take a look.  From first glance it doesn't appear to have 
more than what we're already doing, but we'll see.

- Orion

On 12/21/2010 5:01 PM, Philip Nienhuis wrote:
>
> Perhaps no answer to your question but I can share some user experiences.
>
> This (warning messages) is what I also see when I try to 
> uninstall/install (renew/update) individual octave-forge packages in 
> Mandriva (2010.1).
> Mandriva has made renewal/updating individual octave-forge packages a 
> cumbersome process, with unexpected dependencies all over the place, 
> perhaps due to the still "monolithic" character of its octave-forge 
> rpm, and rpm and (octave's) pkg stepping onto each other's toes.
>
> Hopefully you can make it much easier for Fedora users, up to the 
> point that rpm isn't needed (or only needed for initial installation), 
> i.e., so that just Octave's pkg command will do. (I'd prefer pkg over 
> rpm in this case.)
> I think development of octave-forge packages proceeds more rapidly 
> than rpm's can be built, tested & updated - if you automate building 
> rpms from individual octave-forge packages this can be alleviated a bit.
> Thus either Q-control by Fedora will be hampered, or octave users will 
> be withheld a bit from trying the newest octave-forge packages (in 
> turn, hampering package development) if they must wait until tested 
> rpms become available.
>
> FYI, "pkg" has an option "forge" that will d/l & install octave-forge 
> packages directly from octave.sf.net. Permissions (root) may interfere 
> when packages go into /usr/[local/]share, but that's probably resolvable.
>
> P.
I think the monolithic octave-forge package has proved to be too 
cumbersome.  Individual packages should be easier to maintain.  Updates 
to packages can be done quite quickly.  However, nothing in the rpm 
octave package prevents anyone from using the pkg command directly, and 
is in fact probably the way to go for individual users.   For managed 
environments though, it more convenient to install packages 
automatically via the distributions packaging methods.

- Orion

On Tue, Dec 21, 2010 at 10:11:46PM -0700, Orion Poplawski wrote:
> On 12/21/2010 5:01 PM, Philip Nienhuis wrote:
> >
> > Perhaps no answer to your question but I can share some user experiences.
> >
> > This (warning messages) is what I also see when I try to 
> > uninstall/install (renew/update) individual octave-forge packages in 
> > Mandriva (2010.1).
> > Mandriva has made renewal/updating individual octave-forge packages a 
> > cumbersome process, with unexpected dependencies all over the place, 
> > perhaps due to the still "monolithic" character of its octave-forge 
> > rpm, and rpm and (octave's) pkg stepping onto each other's toes.
> >
> > Hopefully you can make it much easier for Fedora users, up to the 
> > point that rpm isn't needed (or only needed for initial installation), 
> > i.e., so that just Octave's pkg command will do. (I'd prefer pkg over 
> > rpm in this case.)
> > I think development of octave-forge packages proceeds more rapidly 
> > than rpm's can be built, tested & updated - if you automate building 
> > rpms from individual octave-forge packages this can be alleviated a bit.
> > Thus either Q-control by Fedora will be hampered, or octave users will 
> > be withheld a bit from trying the newest octave-forge packages (in 
> > turn, hampering package development) if they must wait until tested 
> > rpms become available.
> >
> > FYI, "pkg" has an option "forge" that will d/l & install octave-forge 
> > packages directly from octave.sf.net. Permissions (root) may interfere 
> > when packages go into /usr/[local/]share, but that's probably resolvable.
> >
> > P.
> I think the monolithic octave-forge package has proved to be too 
> cumbersome.  Individual packages should be easier to maintain.  

Having quite some experience with maintaining the splitted packages in
Debian I can assure you that maintaining the small packages in a
distribution is at least *an order of magnitude* more work than the
monolithic build ever was.

	Thomas

ons, 22 12 2010 kl. 23:05 +0100, skrev Thomas Weber:
> Having quite some experience with maintaining the splitted packages in
> Debian I can assure you that maintaining the small packages in a
> distribution is at least *an order of magnitude* more work than the
> monolithic build ever was.

Is there anything that can be changed in Octave-Forge that would
simplify this work?

Søren

On 22 Dec 2010, at 23:17, Søren Hauberg wrote:

> ons, 22 12 2010 kl. 23:05 +0100, skrev Thomas Weber:
>> Having quite some experience with maintaining the splitted packages in
>> Debian I can assure you that maintaining the small packages in a
>> distribution is at least *an order of magnitude* more work than the
>> monolithic build ever was.
> 
> Is there anything that can be changed in Octave-Forge that would
> simplify this work?

I vaguely remember David had implemented a method to automatically produce an RPM out of an OF package, 
maybe we could have something like 

pkg build -rpm package_name-version.tar.gz

and 

pkg build -deb package_name-version.tar.gz

directly in pkg.m?

> Søren
c.

On Wed, Dec 22, 2010 at 11:17:47PM +0100, Søren Hauberg wrote:
> ons, 22 12 2010 kl. 23:05 +0100, skrev Thomas Weber:
> > Having quite some experience with maintaining the splitted packages in
> > Debian I can assure you that maintaining the small packages in a
> > distribution is at least *an order of magnitude* more work than the
> > monolithic build ever was.
> 
> Is there anything that can be changed in Octave-Forge that would
> simplify this work?

No, not really. Packages have a sort of fixed overhead that just needs
to be done.

	Thomas

On Thu, Dec 23, 2010 at 10:32:06AM +0100, c. wrote:
> 
> On 22 Dec 2010, at 23:17, Søren Hauberg wrote:
> 
> > ons, 22 12 2010 kl. 23:05 +0100, skrev Thomas Weber:
> >> Having quite some experience with maintaining the splitted packages in
> >> Debian I can assure you that maintaining the small packages in a
> >> distribution is at least *an order of magnitude* more work than the
> >> monolithic build ever was.
> > 
> > Is there anything that can be changed in Octave-Forge that would
> > simplify this work?
> 
> I vaguely remember David had implemented a method to automatically produce an RPM out of an OF package, 
> maybe we could have something like 
> 
> pkg build -rpm package_name-version.tar.gz
> 
> and 
> 
> pkg build -deb package_name-version.tar.gz
> 
> directly in pkg.m?

I don't think that will work in the general case. I will probably work
for packages which contain only .m files (in fact I think that for a
bunch of .m files, pkg.m is overkill). 

But for a package which compiles code, you have not much chance in
building something that will work across distributions.

	Thomas

> On Wed, Dec 22, 2010 at 11:17:47PM +0100, Søren Hauberg wrote:
> > ons, 22 12 2010 kl. 23:05 +0100, skrev Thomas Weber:
> > > Having quite some experience with maintaining the splitted
> > > packages in
> > > Debian I can assure you that maintaining the small packages in a
> > > distribution is at least *an order of magnitude* more work than
> > > the
> > > monolithic build ever was.
> >
> > Is there anything that can be changed in Octave-Forge that would
> > simplify this work?
> 
> No, not really. Packages have a sort of fixed overhead that just needs
> to be done.

For Fedora at least, the largest part of the fixed overhead is the initial
one: getting through package review.  There's a surplus of packages to
review and a deficit of reviewers.  Unfortunately the way the review system
currently works is that each package will be treated as an isolated package,
there's currently no mechanism to "batch review" a lot of very similar
packages.  If they are considered separate packages by the buildsystem,
i.e. separate tarballs, they have to be reviewed one by one.  That's why
having a semi-monolithic build of octave-forge would still be useful, perhaps 
grouped into 4 or 5 different groups, e.g. the way gstreamer has their
plugins grouped into "good", "bad" and "ugly":

http://www.gstreamer.net/

There is, of course, an ongoing fixed cost to maintaining each individual
package separately, but that cost is much smaller in comparison to the cost
of getting through the initial review, IMHO.

Alex