linuxcoe-devel — LinuxCOE development discussion

[Linuxcoe-devel] Proposal of documentation patch

[Bruno C. <Bru...@hp...>]

Hello,

I think there is an issue in th documentation. I'd like to propose the
following patch:

-bash-3.2$ cvs diff ./docs/guides/configuration.dkb
Index: ./docs/guides/configuration.dkb
===================================================================
RCS file: /cvsroot/linuxcoe/docs/guides/configuration.dkb,v
retrieving revision 1.4
diff -r1.4 configuration.dkb
65c65
<               /etc/@prefix@/
---
>               @prefix@/etc
137c137
<           /etc/@prefix@/
---
>           @prefix@/etc

In fact when I use --sysconfdir in configure the dir generated is not
referenced correctly in the documentation. I think the above patch
should fix it but there may bu side effects so I'd like confirmation
before applying.

Greetings,
Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Hello,

Back to building packages for LinuxCOE, phase II (sorry for the delay).

Bruno Cornec said on Fri, Oct 26, 2007 at 04:30:59PM +0200:
> What I can propose is a tool in pb which would:
> 
> Take a pb ChangeLog file (under pbconf/pkg) and process it to create a
> ChangeLog file in the tar file, the %changelog section in the spec and
> the changelog file for deb. THat's pretty easy. Keep the objective of a
> single source and respect the target of good changelogs everywhere where
> it's needed.
> 

Ok, that's now in the code. In fact I used 2 files, one for external
info changelog which contains what I need and is my current log format.
That allows to cretae the %changelog section and the NEWS file.

Also I use the tools cvs2cl and svn2cl to generate the ChangeLog files
from the CMS. + an pbauthors file which creates the AUTHORS file and is
used by the svn2cl tool.

While trying to adapt my LinuxCOE pbconf tree to that, I encountered
another issue, I think I mentioned earlier, but which is now hurting
because it prevents me from creating the packages in the VM:

configure: error: can not automatically integrate with Apache HTTP
server

My VMs try to keep the minimum needed to just allow the build of a
software, not its installation. So I still think that those checks
(which are useful) should be rather done at install time, rather than at
build time.

Bruno.
Any Idea ?
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bryan Gartner said on Fri, Oct 26, 2007 at 07:53:12AM -0600:

> I was actually thinking of creative/reworked autoconf checks instead
> (that you could turn off in packaging context).  Since make install
> puts bits on your system, it would be frustrating to then have it tell
> you later that you still needed dependencies.  And the current scheme
> would already have turned off the apache integration, since it couldn't
> find apache*ctl.
> 
> FWIW, you already have the ability to do:
> 
> 	  --without-PACKAGE       do not use PACKAGE (same as
> 	  --with-PACKAGE=no)
> 	  --without-APACHE        skip auto-integratation with Apache webserver
> 	  --without-SUDOERS       skip auto-integration with sudo
> 
> so maybe I should just augment that list.

Humm does using --without-APACHE  avoids the creation of the conf file and u
its installation ? If yes, it's not what I want.

What about generating BOLD warnings, instead of aborting ending with 
"You build is correct but won't work on that system" ?
Remember that it's only for tar.gz, as for packages, depedencies take
care of that, and will enforce it.

After all it's not different when you use --nodeps with rpm to install a
package anyway.

Moreover, I forgot to mention it, but your checks are not working on all 
distros, as the most recent do not use mkisofs anymore but genisoimage, 
....  So you should check now for one or the other.

> Actually, the packaging/<module>debian/changelog files already have the
> info you are looking for, I think.  And since we have yet to officially
> produce .debs, they stand at 4.1-1, but the latter value should be
> rev'd each time.

I've not looked at why they were not produced, as I was already happy to
find a bunch of packages made automagically ;-) Will work on that ...
well honestly don't know exactly when as it's becoming hectic here :-)

Have a nice WE !
Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bryan Gartner said on Fri, Oct 26, 2007 at 06:44:48AM -0600:

> Hmm, you have pointed out one complete oversight on my part (that
> is not maintaining the ChangeLog ... for a very long time).  Whoops!

As a socker player, I can say 1-0 ;-)

> > What I mean is that for mondo e.g. I have:
> > 
> > # $Id: ChangeLog 1679 2007-10-11 23:24:45Z bruno $
> 
> I can certainly add the # $Id: $, yet for CVS, this really only reflects the
> revision of that particular file.

Well that was really a detail. Useful, but not mandatory.

> Actually, AFAICT, the format is completely standard:
> 
> 	http://www.gnu.org/software/guile/changelogs/guile-changelogs_3.html

Humm, wasn't aware of thet one. But after reading I find it dated if I
may express my opinion.

When used in conjunction of a good CMS (CVS being average for me, SVN
good, and maybe git, arch, ... excellent as I never tried them) I think
some of those rules are useless (especially documenting the file names
changed, which is the role of the SCM). 

In particular, I think the link between the version and the list of
modification is the most capital information. And it's completely
missing here, and could be unrelated to dates events.

And also currently it's rather considered poorly to add e-mail addresses
in those files, as it would be a help for spammers :-( (I changed mine
in particular for that).

I took example from the result of locate ChangeLog on my HDD (for
non-gnu projects as those tend to respect the format of the project ;-):

Kino:
=====
[...]
Changes in version 1.1.1:
        Changes by Dan Dennedy:
        - commands.cc: bugfix segfault on crash recovery with gtk+
          version < 2.11

Changes in version 1.1.0:
        Changes by Dan Dennedy:

less:
=====
[...]
Version 1.53 Apr 11, 2006
- support for Openoffice documents (Florian Cramer, Vincent Lefevre)
- support for RAR archives (suggested by Cindy Leonhardt)
[...]
Version 1.44 Dec 14, 2004
-------------------------
- fixes in configure script (Slaven Rezic)

netcat:
=======
[*] Sun Jan 11 2004 - netcat v0.7.1

  o Fixed bug in UDP connection code, using pktinfo support. [BUG
#796765]
  o Fixed a segfault in the DNS lookup routines, caused by invalid DNS
    configurations.
[...]
[*] Thu Aug 21 2003 - netcat v0.7.0

  o Added command line switch `-c' (close), which shuts down the
connection
    on EOF from stdin without waiting for any answer from remote host.

Honestly taken randomly.

So I'd tend to conclude that only official GNU projects follow the rule,
other use varied format.
Again, my problem is that I need to extract info from the changelog to
generate the %changelog entries for spec file (and I do the same for deb
changelog as well). But I really need the version info for the spec. So
it needs to be somewhere (could be in another place. Again I don't want
to duplicate info, just keeping it complete and coherent in one single
place, and generate the rest from that single instance.

Time for a ChangeLog RFC ;-)

> So, not really sure where to go on this one (other than poke myself to
> update this as changes are made).  That said, the debian/changelog file
> which should be maintained (and was created) by the dch command does
> have a slightly different format.  Color me confused on how to proceed.

What I can propose is a tool in pb which would:

Take a pb ChangeLog file (under pbconf/pkg) and process it to create a
ChangeLog file in the tar file, the %changelog section in the spec and
the changelog file for deb. THat's pretty easy. Keep the objective of a
single source and respect the target of good changelogs everywhere where
it's needed.

Your ideas ?

Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bryan G. <Bry...@HP...>]

Bruno,

On Fri, Oct 26, 2007 at 01:35:27PM +0000, Cornec, Bruno (Linux Consultant) wrote:
> Hello Bryan and others,
> 
> Bryan Gartner said on Fri, Oct 26, 2007 at 06:25:02AM -0600:
> 
> > On Thu, Oct 25, 2007 at 10:34:06PM +0000, Cornec, Bruno (Linux Consultant) wrote:
> > > The problem is that this should be rather done a run time than at build
> > > time, because the machine on which you're building the tool may be
> > > different from the machine you want to install it on.
> >
> > And if you are installing from CVS/tar, this is a "runtime" check, which
> > I'd prefer not to lose the ability to do.  So, I vote to retain the
> > checks there.
> 
> OK, so I'd like to propose an alternative that may help both of us:
> 
> Could we do the check after the make integrate phase (e.g. a make check
> new order) that would have to be called in the context of tar.gz install
> (by Creating a cheked dummy file in the tree, that could be verified
> before running anything). If there go on, if not, emit a warning rather
> than running with CGI-Carp or something like that.

I was actually thinking of creative/reworked autoconf checks instead
(that you could turn off in packaging context).  Since make install
puts bits on your system, it would be frustrating to then have it tell
you later that you still needed dependencies.  And the current scheme
would already have turned off the apache integration, since it couldn't
find apache*ctl.

FWIW, you already have the ability to do:

	  --without-PACKAGE       do not use PACKAGE (same as
	  --with-PACKAGE=no)
	  --without-APACHE        skip auto-integratation with Apache webserver
	  --without-SUDOERS       skip auto-integration with sudo

so maybe I should just augment that list.

> > > FYI, I have generated a lot of RPMS today, available at
> > > ftp://ftp.project-builder.org , except that the main one,
> > > linuxcoe-sd-base is missing :-(
> >
> > Nice job!
> 
> Nearly ;-) And I still have problems for the .deb, partly because of the
> lack of changelog files.

Actually, the packaging/<module>debian/changelog files already have the
info you are looking for, I think.  And since we have yet to officially
produce .debs, they stand at 4.1-1, but the latter value should be
rev'd each time.

bryang

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Hello Bryan and others,

Bryan Gartner said on Fri, Oct 26, 2007 at 06:25:02AM -0600:

> On Thu, Oct 25, 2007 at 10:34:06PM +0000, Cornec, Bruno (Linux Consultant) wrote:
> > The problem is that this should be rather done a run time than at build
> > time, because the machine on which you're building the tool may be
> > different from the machine you want to install it on.
> 
> And if you are installing from CVS/tar, this is a "runtime" check, which
> I'd prefer not to lose the ability to do.  So, I vote to retain the
> checks there.

OK, so I'd like to propose an alternative that may help both of us:

Could we do the check after the make integrate phase (e.g. a make check
new order) that would have to be called in the context of tar.gz install
(by Creating a cheked dummy file in the tree, that could be verified
before running anything). If there go on, if not, emit a warning rather
than running with CGI-Carp or something like that.

Of course that file could be touched by the spec/rules files to avoid
problems.

WDYT ?

> > FYI, I have generated a lot of RPMS today, available at
> > ftp://ftp.project-builder.org , except that the main one,
> > linuxcoe-sd-base is missing :-(
> 
> Nice job!

Nearly ;-) And I still have problems for the .deb, partly because of the
lack of changelog files.

Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bryan G. <Bry...@HP...>]

Bruno,

On Wed, Oct 24, 2007 at 05:13:20PM +0000, Cornec, Bruno (Linux Consultant) wrote:
> I still have one remaining issue.
> 
> The current ChangeLog files (that pb uses to create %changelog entries
> for .spec file) doesn't content anythink concerning the version it
> applies to.

Hmm, you have pointed out one complete oversight on my part (that
is not maintaining the ChangeLog ... for a very long time).  Whoops!

> What I mean is that for mondo e.g. I have:
> 
> # $Id: ChangeLog 1679 2007-10-11 23:24:45Z bruno $

I can certainly add the # $Id: $, yet for CVS, this really only reflects the
revision of that particular file.

> MONDO CHANGES
> 
> 2.2.5 (2007-10-11)
> - Build process adapted to use pb http://trac.project-builder.org (Bruno
>   Cornec)
> - Fix #137 and #3 issue with large exclude list (> 1000 chars) (Bruno Cornec)
> [...]
> 
> 2.2.4 (2007-07-06)
> - Size of DVD is 4482 (or more surely 4480 to avoid problems - used everywhere) (Bruno Cornec)
> - New Hardware migration guide with P2V (Eric Montaut/Gallig/Renaud/Bruno Cornec)
> [...] etc
> 
> 
> Which allows me to easily process that.
> 
> I do not really care about format in itself (I will have to provide a
> converter), but I care about info such as version and date.

Actually, AFAICT, the format is completely standard:

	http://www.gnu.org/software/guile/changelogs/guile-changelogs_3.html

and in fact this was the reference for the original templates used.
 
> Currently we have:
> 
> 2005-09-27      Bryan Gartner   <bry...@hp...>
> 
>         * incorporated v4 of the LinuxCOE System Designer
> 
> 2005-08-01      Bryan Gartner   <bry...@hp...>
> 
>         * automated the integration of the LinuxCOE System Designer
>           to a system, and partitioned the package into modules
>           for each distribution
> 
> Would it be at least possible to add the version concerned ?
> (Which goes back to my question on version management from the other day
> BTW).

So, not really sure where to go on this one (other than poke myself to
update this as changes are made).  That said, the debian/changelog file
which should be maintained (and was created) by the dch command does
have a slightly different format.  Color me confused on how to proceed.

bryang

Re: [Linuxcoe-devel] Update on packaging

[Bryan G. <Bry...@HP...>]

Bruno,

On Thu, Oct 25, 2007 at 10:34:06PM +0000, Cornec, Bruno (Linux Consultant) wrote:
> > I still have one remaining issue.
> 
> And here is the question of the day ;-)
> 
> Currently the autoconf.ac script makes checks on tools such as mkisofs,
> perl modules, ...
> 
> The problem is that this should be rather done a run time than at build
> time, because the machine on which you're building the tool may be
> different from the machine you want to install it on.

And if you are installing from CVS/tar, this is a "runtime" check, which
I'd prefer not to lose the ability to do.  So, I vote to retain the
checks there.

> Typically in my VMs, I do not have mkisofs, nor perl-CGI for the moment.
> So should I add them to make the build process happy (even if I don't
> think it's the right thing to do, I'll do it if that's the concensus),
> or should I work on a patch to move those checks at run time or/and in
> dependencies either with .spec requires, or the same for deb.
> 
> What is your take on this one ?

I am quite open to clever autoconf-ish ways to turn off those checks for
your use case, though.

> FYI, I have generated a lot of RPMS today, available at
> ftp://ftp.project-builder.org , except that the main one,
> linuxcoe-sd-base is missing :-(

Nice job!

bryang

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

> I still have one remaining issue.

And here is the question of the day ;-)

Currently the autoconf.ac script makes checks on tools such as mkisofs,
perl modules, ...

The problem is that this should be rather done a run time than at build
time, because the machine on which you're building the tool may be
different from the machine you want to install it on.

Typically in my VMs, I do not have mkisofs, nor perl-CGI for the moment.
So should I add them to make the build process happy (even if I don't
think it's the right thing to do, I'll do it if that's the concensus),
or should I work on a patch to move those checks at run time or/and in
dependencies either with .spec requires, or the same for deb.

What is your take on this one ?

FYI, I have generated a lot of RPMS today, available at
ftp://ftp.project-builder.org , except that the main one,
linuxcoe-sd-base is missing :-(

Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bruno Cornec said on Tue, Oct 16, 2007 at 11:58:18PM +0200:

> I'm right now working on having a single spec file for all my 21 rpm
> based VMs. Once it's done, adapting your debian build files to pb will
> be easy, and as you're on Debian based systems, you'll then be able to
> benefit from what I'm doing.
> 

I still have one remaining issue.

The current ChangeLog files (that pb uses to create %changelog entries
for .spec file) doesn't content anythink concerning the version it
applies to.

What I mean is that for mondo e.g. I have:

# $Id: ChangeLog 1679 2007-10-11 23:24:45Z bruno $

MONDO CHANGES

2.2.5 (2007-10-11)
- Build process adapted to use pb http://trac.project-builder.org (Bruno
  Cornec) 
- Fix #137 and #3 issue with large exclude list (> 1000 chars) (Bruno Cornec)
[...]

2.2.4 (2007-07-06)
- Size of DVD is 4482 (or more surely 4480 to avoid problems - used everywhere) (Bruno Cornec)
- New Hardware migration guide with P2V (Eric Montaut/Gallig/Renaud/Bruno Cornec)
[...] etc


Which allows me to easily process that.

I do not really care about format in itself (I will have to provide a
converter), but I care about info such as version and date.

Currently we have:

2005-09-27      Bryan Gartner   <bry...@hp...>

        * incorporated v4 of the LinuxCOE System Designer

2005-08-01      Bryan Gartner   <bry...@hp...>

        * automated the integration of the LinuxCOE System Designer
          to a system, and partitioned the package into modules
          for each distribution

Would it be at least possible to add the version concerned ?
(Which goes back to my question on version management from the other day
BTW).

Thanks for your feedbacks.

Bruno.

PS: I have an option in pb to ignore logs that I'll use meanwhile, but
that's not good practice IMHO.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bruno Cornec said on Mon, Oct 22, 2007 at 04:13:17PM +0200:

> `/cvsroot/linuxcoe/pbconf/linuxcoe-sd-data-opensuse'
> (/cvsroot/linuxcoe/pbconf/linuxcoe-sd-data-opensuse/#cvs.lock): No such
> file or directory
> cvs commit: lock failed - giving up
> cvs [commit aborted]: lock failed - giving up
>=20
> Do someone on this list has the possibility to unlock this, or is it a
> SF.net request to make ?

Ok, it seems I'm not CVS friendly. I removed all my local CVS dir under
that subdir, re did cvs add and this time cvs ci was successful.

Bruno.
--=20
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA I=
ET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.n=
et
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/lin=
ux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.o=
rg

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bruno Cornec said on Mon, Oct 22, 2007 at 03:40:45PM +0200:

> linuxcoe-sd-data-opensuse-4.1-1.mdv2008.0

I seem to not be able to commit my changes for that last package :-(
(All previous overlay are in):

cvs commit: failed to create lock directory for
`/cvsroot/linuxcoe/pbconf/linuxcoe-sd-data-opensuse'
(/cvsroot/linuxcoe/pbconf/linuxcoe-sd-data-opensuse/#cvs.lock): No such
file or directory
cvs commit: lock failed - giving up
cvs [commit aborted]: lock failed - giving up

Do someone on this list has the possibility to unlock this, or is it a
SF.net request to make ?

Bruno.
--=20
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA I=
ET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.n=
et
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/lin=
ux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.o=
rg

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bruno Cornec said on Fri, Oct 19, 2007 at 06:55:18PM +0200:

> In the mean time, I know I can provide the other RPM for the other
> remaining distro, and test all of that next week.

Some more news:

# rpm -aq | grep linuxcoe
linuxcoe-sd-data-centos-4.1-1.mdv2008.0
linuxcoe-sd-data-debian-4.1-1.mdv2008.0
linuxcoe-sd-docs-4.1-1.mdv2008.0
linuxcoe-sd-data-fedora-4.1-1.mdv2008.0
linuxcoe-sd-data-ubuntu-4.1-1.mdv2008.0
linuxcoe-sd-data-scientific-4.1-1.mdv2008.0
linuxcoe-sd-base-4.1-1.mdv2008.0
linuxcoe-sd-data-opensuse-4.1-1.mdv2008.0

All of those appear in the combo box.
Still needs to check with how to instll the image part.

Bruno.
--=20
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA I=
ET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.n=
et
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/lin=
ux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.o=
rg

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bruno Cornec said on Fri, Oct 19, 2007 at 05:22:28PM +0200:

> It's possible that I try to produce a bunch of RPMs during the WE for at
> least -base and -docs for you to test next week. I aslo have a -centos
> build ready, which I may test later on today if time permits (not sure,
> I'd like to finish my TechCon proposal as well).

Good day today, I have an abstract (still needs another one for pb ;-)
And I also tested my linuxcoe-sd-data-centos package:

[root@dploy svn]# urpmi
/usr/src/svn/LinuxCOE/build/RPMS/noarch/linuxcoe-sd-data-centos-4.1-1.mdv20=
08.0.noarch.rpm

installing linuxcoe-sd-data-centos-4.1-1.mdv2008.0.noarch.rpm from
/usr/src/svn/LinuxCOE/build/RPMS/noarch
Preparing...  ############################################################
      1/1: linuxcoe-sd-data-centos
              ############################################################

And I have the CentOS choices in the ComboBox of links, and was able to
go up to the last page where I got:

Cannot locate defimage for CentOS-4.4-i386 which I think is the correct
error as I do not have the images anymore (I asked it enough ;-)

So now Lee, I'll welcome your proposal of giving instructions ;-)
In he meean time, I know I can provide the other RPM for the other
remaining distro, and test all of that next week.
(BTW, everything is under CVS, except some info for pb building, but
that conf file will be put under CVS when pb reach 0.8.6)

Have a nice week-end.
Bruno.
--=20
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA I=
ET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.n=
et
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/lin=
ux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.o=
rg

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Mayes, Lee said on Fri, Oct 19, 2007 at 02:28:32PM +0000:

> While we certainly could script something to create the virgin images, I'm leaning more towards just documenting the process, as it varies quite wildly between distros, and even some between versions of the same distro.  Since we'll have tarballs of the images available publicly, that should do for all but the most paranoid, and as long as we give them the recipe to create their own, I think that's sufficient.
> 
> I'll be happy to write up the processes.  Thoughts?

That would make a good design doc for the script BTW ;-)
And having the script seems mandatory for dploy.org ;-))

Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Louis Bouchard said on Fri, Oct 19, 2007 at 03:15:52PM +0200:

> This is why the generation script is important in my mind and needs to
> be addressed asap. I'm off monday to finalize the .spec files and
> hopefully start on this one.

Humm, FYI I now have everything needed for .spec files based on your
original work Louis. So do not pass time anymore on it.

It's possible that I try to produce a bunch of RPMs during the WE for at
least -base and -docs for you to test next week. I aslo have a -centos
build ready, which I may test later on today if time permits (not sure,
I'd like to finish my TechCon proposal as well).

Should also have a look on the deb, as with what Bryan provides, we
shouldn't be far from the result.

BTW Bryan to come back to a question you had, PB is perfectly able to
*not* filter any file: just do not use macros in it ;-) The only
requirement is the structure of the pbconf dir, and the names of subdir.

Bruno.
--=20
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA I=
ET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.n=
et
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/lin=
ux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.o=
rg

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bryan Gartner said on Fri, Oct 19, 2007 at 07:03:49AM -0600:

> I was in the process of doing just that the other day when I
> did an errant import and got a duplication of an entire existing
> module.  And with the way that CVS works (eg. the attic), and the
> policies of sf.net, I had to place a support call to get the cruft removed
> from that new module.  I just got confirmation today that it had
> been completed.  So I have removed the images from -CentOS as of now.
> Once others chime in on the following question, I'll proceed with
> the rest.

Ahh, the fun of dealing with sf.net :-(
Guess why I preferred to host myself my repository for my projects here
?
More over at that time they were not even proposing SVN, and (sorry to
be crude) I find CVS so featureless, sometimes over complicated :-(

> To be honest, I am starting to debate whether we want to have those
> images checked in at all.  I can see making them available for
> download, but do we really need to have them under version control
> on our sf.net project? 

In theory as they are the result of a production process they should not
be versioned. However, we do not have that process in place yet don't we
? So I think we still need to provide them in one way or another. Again
sf.net doesn't help for ftp availability :-( 

What I could do is ask here for an MSA to add to my mondorescue server
and host what is needed here. 

Probably not the reliability of sf.net BTW, so should be mirrored
elsewhere (Lee i'll come back to you on this topic in your other mail ...
later sorry !)

I vote for a ftp access of those files. 

Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bryan G. <Bry...@HP...>]

Lee,

On Fri, Oct 19, 2007 at 02:28:32PM +0000, Mayes, Lee wrote:
> While we certainly could script something to create the virgin images, I'm leaning more towards just documenting the process, as it varies quite wildly between distros, and even some between versions of the same distro.  Since we'll have tarballs of the images available publicly, that should do for all but the most paranoid, and as long as we give them the recipe to create their own, I think that's sufficient.

I pretty much agree, and had written up the basics some months ago,
in the SystemDesigner/images/README file.  It is certainly not complete
(eg. missing usb image info) and can certainly use more details.
However, at least one customer has used this to generate RHEL update
images (since we never distributed those).

> I'll be happy to write up the processes.  Thoughts?

Feel free to correct/fix this up as necessary.  Plus if  you have some
time, all the SystemDesigner/*/README files are probably up for a good
review/cleanup as well.

bryang

Re: [Linuxcoe-devel] Update on packaging

[Mayes, L. <lee...@hp...>]

While we certainly could script something to create the virgin images, I'm =
leaning more towards just documenting the process, as it varies quite wildl=
y between distros, and even some between versions of the same distro.  Sinc=
e we'll have tarballs of the images available publicly, that should do for =
all but the most paranoid, and as long as we give them the recipe to create=
 their own, I think that's sufficient.

I'll be happy to write up the processes.  Thoughts?

Lee

-----Original Message-----
From: lin...@li... [mailto:linuxcoe-devel-b=
ou...@li...] On Behalf Of Bouchard, Louis
Sent: Friday, October 19, 2007 9:16 AM
To: Gartner, Bryan W
Cc: Cornec, Bruno (Linux Consultant); lin...@li...
Subject: Re: [Linuxcoe-devel] Update on packaging

Hi,

>
> To be honest, I am starting to debate whether we want to have those
> images checked in at all.  I can see making them available for
> download, but do we really need to have them under version control on
> our sf.net project?
>

I don't see a reason to check in binary data that we do not produce/compile=
 ourselves.  We should checkin what is used to produce those images (i.e. s=
cripts) but the the end product itself.

This is why the generation script is important in my mind and needs to be a=
ddressed asap. I'm off monday to finalize the .spec files and hopefully sta=
rt on this one.

Stay tuned,

...Louis

> bryang
> Louis Bouchard, Linux Support Engineer EMEA Linux Competency Center,
> Linux Ambassador, HP
>
> HP Services                       1 Ave du Canada
> HP France                         Z.A. de Courtaboeuf
> lou...@hp...             91 947 Les Ulis
> http://www.hp.com/go/linux        France
> http://www.hp.com/fr

Re: [Linuxcoe-devel] Update on packaging

[Bryan G. <Bry...@HP...>]

FWIW,

> I was in the process of doing just that the other day when I
> did an errant import and got a duplication of an entire existing
> module.  And with the way that CVS works (eg. the attic), and the
> policies of sf.net, I had to place a support call to get the cruft removed
> from that new module.  I just got confirmation today that it had
> been completed.  So I have removed the images from -CentOS as of now.
> Once others chime in on the following question, I'll proceed with
> the rest.

And this has now been completed for:

	CentOS
	Debian
	Fedora
	OpenSUSE
	Scientific
	Ubuntu

I will do some further cleanup of the these modules later.  And I
will attack the others soon.

Remember, these images are still available at:

	http://www.instalinux.com/snapshots/images

bryang

Re: [Linuxcoe-devel] Update on packaging

[Louis B. <lou...@hp...>]

Hi,

>=20
> To be honest, I am starting to debate whether we want to have those
> images checked in at all.  I can see making them available for
> download, but do we really need to have them under version control
> on our sf.net project?
>=20

I don't see a reason to check in binary data that we do not
produce/compile ourselves.  We should checkin what is used to produce
those images (i.e. scripts) but the the end product itself.

This is why the generation script is important in my mind and needs to
be addressed asap. I'm off monday to finalize the .spec files and
hopefully start on this one.

Stay tuned,

...Louis

> bryang
> Louis Bouchard, Linux Support Engineer
> EMEA Linux Competency Center,
> Linux Ambassador, HP
>=20
> HP Services                       1 Ave du Canada
> HP France                         Z.A. de Courtaboeuf
> lou...@hp...             91 947 Les Ulis
> http://www.hp.com/go/linux        France
> http://www.hp.com/fr

Re: [Linuxcoe-devel] Update on packaging

[Bryan G. <Bry...@HP...>]

Bruno,

> Thanks to Bryan helps, I was able today to have my first 2 working RPMs
> for Mandriva 2008.0 (the distro used on my deployment server) for
> linuxcoe-sd-base and linuxcoe-sd-docs. And the docs are accessible
> through the Web interface so I guess it's working fine as far as I can
> see.

Actually you did all the *really* hard work.

> So I had a look at cento just now. Would you allow me to move the images
> part currently under the SystemDesigner-CentOS subdir to
> SystemDesigner-CentOS-images ?
> 
> The reason for me is that I do an export from the repository (cvs
> export) to get the content and I still then get the images as part as
> the process, which slow down things for testings :-(

I was in the process of doing just that the other day when I
did an errant import and got a duplication of an entire existing
module.  And with the way that CVS works (eg. the attic), and the
policies of sf.net, I had to place a support call to get the cruft removed
from that new module.  I just got confirmation today that it had
been completed.  So I have removed the images from -CentOS as of now.
Once others chime in on the following question, I'll proceed with
the rest.

To be honest, I am starting to debate whether we want to have those
images checked in at all.  I can see making them available for
download, but do we really need to have them under version control
on our sf.net project? 

bryang

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Hello,

Thanks to Bryan helps, I was able today to have my first 2 working RPMs
for Mandriva 2008.0 (the distro used on my deployment server) for
linuxcoe-sd-base and linuxcoe-sd-docs. And the docs are accessible
through the Web interface so I guess it's working fine as far as I can
see.

Bruno Cornec said on Thu, Oct 18, 2007 at 01:20:59PM +0200:

> > All I have yet to finish are the nonFOSS distro's.  So feel free to
> > give it a try with base+one other.

So I had a look at cento just now. Would you allow me to move the images
part currently under the SystemDesigner-CentOS subdir to
SystemDesigner-CentOS-images ?

The reason for me is that I do an export from the repository (cvs
export) to get the content and I still then get the images as part as
the process, which slow down things for testings :-(

Or give a better advice,
TIA,
Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bruno C. <Bru...@hp...>]

Bryan Gartner said on Wed, Oct 17, 2007 at 07:52:01PM -0600:

> > The remaining question I have is for the name of the directories.
> > Currently by default they are named /var/log/linuxcoe-sd-base, ...
> > /var/www/linuxcoe-sd-base. I wonder whether it would be cleaner to call
> > those just /var/www/linuxcoe-sd (or even /var/www/linuxcoe) for more
> > clarity on the admin side, and as all packages will add content below.
> > May be solved later on BTW.
> > 
> Actually, my debian packages (which now include base+docs+Distro=FOSS
> ones) all install to the same prefix which is currently
> /usr/share/linuxcoe-sd/4.1 (yes I know you object to the 4.1 portion),
> but the point remains the same (and that issue is still bubbling to
> the top of my todo list).

Same here:

[root@dploy ~]# rpm -ql linuxcoe-sd-base | grep 4.1
/var/cache/linuxcoe-sd/4.1
/var/lib/linuxcoe-sd/4.1
/var/lib/linuxcoe-sd/4.1/profiles
/var/log/linuxcoe-sd/4.1

So I guess this remains to be chosen and solved.
Maybe another variable would be useful for that, that we could use to
create subdirs under /etc, /var/lib, /var/log, ... and in the upstream
version you could still use linuxcoe-sd/4.1, when we could then use in pkg
only linuxcoe-sd (or whatever name we agree upon). Is that what you intended ?

> And I have a vm (distinct from the vm that I build these .debs on) that
> has all of these installed and working. 

Out of curiosity, what VM system are you using ?
Do you use LinuxCOE to deploy them ;-) ?

> FWIW, all of the boot images
> are currently still in CVS in their original location, yet their
> respecitve Makefile.am just doesn't deliver them.  One can still
> obtain the images from:
> 
> 	http://instalinux.com/snapshots/images
> 
> And I'll note that in the README and respective documents when this
> process is complete.

Ok. I'm also far behing in term of docs :-(

However, for non-OSS distro, there is probably a need to still provide
the content somewhere, internaly,as well as the tool to build it from
repository.

> What I was thinking is pretty general, basically have the user supply
> the prefix to a "known" good repository, and then recursively look
> to find the correct kind of files,  then do the "magic" as noted
> in the images/README file.  And for extra credit, remember that the
> repository does not need to reside locally, but could simply be
> a well-known internet mirror ;)  With something like this, whether
> you mirror, use mrepo, or just reference somebody else's repository,
> you would be covered.

Well the design is probably more easy than the code here ;-)
The prefix to a "known" good repository should probably be a URL such as
file://... or http://... or ftp://... but the recursive aspect may then
be difficult to deal with remotely.

I think that 90%+ of the people, will just use an existing/or internal
mirror of the original structure.

So I guess that for, say centos, we will find :

[root@dploy ~]# ls /pub/centos/
2@    3.4@  3.8/  4.1/  4.5/    dostools/       RPM-GPG-KEY-beta RPM-GPG-KEY-CentOS-4@
3@    3.5@  3.9/  4.2/  5/      graphics/       RPM-GPG-KEY-CentOS-2@ RPM-GPG-KEY-CentOS-5
3.1@  3.6@  4@    4.3/  5.0/    HEADER.html     RPM-GPG-KEY-CentOS-3 TIME
3.3@  3.7@  4.0/  4.4/  build/  HEADER.images/  RPM-GPG-KEY-centos4 timestamp.txt

I don't think it makes a big contraint. So having the such a prefix allow
for the script to fetch with wget what is needed and create the local
structure needed for LinuxCOE. Is that what you were proposing ?

(In that case, locally it could be file:///pub/centos and remotely
ftp://deploy/pub/centos)

> > On that one, if you want, I can copy your current content under pbconf,
> > make the adaptations, and you could then test it, once you have a
> > working pb on your side ;-)
> 
> All I have yet to finish are the nonFOSS distro's.  So feel free to
> give it a try with base+one other.

Ok, I made a first copy today of the deb files and have made a rough
adaptation. Not even tested. I'd really like to finish first the rpm
one, as I think making deb after that is just a piece of cake, as you
have the conf ifles, and I'll then have the logic for pb.

> FYI, I think I have a lead on the relaxing automake1.4 dependency and
> am doing some testing now.

Good :-)

Bruno.
-- 
Linux Profession Lead EMEA  / Open Source Evangelist \        HP C&I EMEA IET
http://www.mondorescue.org / HP/Intel Solution Center \  http://hpintelco.net
Des infos sur Linux?  http://www.HyPer-Linux.org      http://www.hp.com/linux
La musique ancienne?  http://www.musique-ancienne.org http://www.medieval.org

Re: [Linuxcoe-devel] Update on packaging

[Bryan G. <Bry...@HP...>]

Bruno,

On Wed, Oct 17, 2007 at 11:42:07PM +0000, Cornec, Bruno (Linux Consultant) wrote:
> Hello,
> 
> Bryan Gartner said on Wed, Oct 17, 2007 at 11:01:06AM -0600:
> > Sorry for my proposal/diversion, I have recrafted the following modules to
> > use the previous namespace:
> >
> >       linuxcoe-sd-base
> >       linuxcoe-sd-docs
> 
> Oh, great ! so now my system works directly, and I do not have any
> warning/error message anymore:

Yay!

> # urpmi /usr/src/svn/LinuxCOE/build/RPMS/noarch/linuxcoe-sd-docs-devel-1.mdv2008.0.noarch.rpm /usr/src/svn/LinuxCOE/build/RPMS/noarch/linuxcoe-sd-base-devel-1.mdv2008.0.noarch.rpm
> 
> installing linuxcoe-sd-base-devel-1.mdv2008.0.noarch.rpm linuxcoe-sd-docs-devel-1.mdv2008.0.noarch.rpm from /usr/src/svn/LinuxCOE/build/RPMS/noarch
> Preparing...  #################################################################################
>       1/2: linuxcoe-sd-base #################################################################################
> Reloading httpd: [  OK  ]
>       2/2: linuxcoe-sd-docs #################################################################################
> 
> I'm already happy with that ;-)

You are almost caught up with me now ;)

> More tests tomorrow to see if it really works. (I need to configure the
> apache server now).
> 
> The remaining question I have is for the name of the directories.
> Currently by default they are named /var/log/linuxcoe-sd-base, ...
> /var/www/linuxcoe-sd-base. I wonder whether it would be cleaner to call
> those just /var/www/linuxcoe-sd (or even /var/www/linuxcoe) for more
> clarity on the admin side, and as all packages will add content below.
> May be solved later on BTW.
> 
> >       linuxcoe-sd-data-centos
> >           (and note that this module no longer delivers images)

Actually, my debian packages (which now include base+docs+Distro=FOSS
ones) all install to the same prefix which is currently
/usr/share/linuxcoe-sd/4.1 (yes I know you object to the 4.1 portion),
but the point remains the same (and that issue is still bubbling to
the top of my todo list).

And I have a vm (distinct from the vm that I build these .debs on) that
has all of these installed and working. FWIW, all of the boot images
are currently still in CVS in their original location, yet their
respecitve Makefile.am just doesn't deliver them.  One can still
obtain the images from:

	http://instalinux.com/snapshots/images

And I'll note that in the README and respective documents when this
process is complete.

> I just looked today at the fedora one to see what was needed and what
> had to be generated. If you want I can take a closer look at that based
> on the existing script for ideas, and write a perl script that creates
> what is required for LinuxCOE based on a mirrored centos setup.

What I was thinking is pretty general, basically have the user supply
the prefix to a "known" good repository, and then recursively look
to find the correct kind of files,  then do the "magic" as noted
in the images/README file.  And for extra credit, remember that the
repository does not need to reside locally, but could simply be
a well-known internet mirror ;)  With something like this, whether
you mirror, use mrepo, or just reference somebody else's repository,
you would be covered.

> > and have updated the packaging/<module>/debian artifacts to correctly
> > package them as .debs.  I will work through the remaining modules
> > this week.  After this round of modifications, I will modify:
> >
> >       linuxcoe-sd-data-*
> >           name stds, skip image delivery
> >         linuxcoe-sd-*
> >           pull back version number in delivered directory hierarchy
> >         linuxcoe-sd-*.deb
> >           migrate to .deb via PB
> 
> On that one, if you want, I can copy your current content under pbconf,
> make the adaptations, and you could then test it, once you have a
> working pb on your side ;-)

All I have yet to finish are the nonFOSS distro's.  So feel free to
give it a try with base+one other.

> I'm still not done with all my rpm VMs setup, so have not yet a .deb for pb
> itself. Will come soon I hope.

FYI, I think I have a lead on the relaxing automake1.4 dependency and
am doing some testing now.

bryang