Re: [Linuxcoe-users] Using Mirrors

[Cusick, J. C. N. 5. <joh...@na...>]

Lee,

Success, up to a point with your directions below. I've built the deb using EPM (straight out of the sourceforge box - haven't read the EPM docs yet). The LinuxCOE stuff was created on the client as follows

/opt/LinuxCOE/bin/ 
/opt/LinuxCOE/bin/LinuxCOE-final

and 

/etc/opt/LinuxCOE/INSTALLED
/etc/opt/LinuxCOE/replay
/etc/opt/LinuxCOE/waystation (my waystation is listed)
/etc/opt/LinuxCOE/apt/ (empty)
/etc/opt/apt.sources/COE_BASE (with my individual repos - similar to yours below)

I noticed that none of the three bin scripts from (in my case) linuxcoe-sw-sources-4.0-0-linux-2.6-all.deb were there, though. Should they be? After reading reading the perl scripts it looks to me like the should, for example the configure_apt script. POST-DEB is setup as you have it below, modded for my system, of course.

The system still loaded completely from outside mirrors.

John C.



-----Original Message-----
From: Mayes, Lee [mailto:lee...@hp...] 
Sent: Tuesday, December 14, 2010 5:38
To: Cusick, John CTR NAWCWD, 541400D; Gartner, Bryan W
Cc: lin...@li...
Subject: RE: [Linuxcoe-users] Using Mirrors

Hi John,

What you want to do is certainly do-able and is how we run our shop as well.  While there is nothing proprietary about our post processing environment it's probably not our best documented feature even though we live and die by it on our internal System Designer instance.   In the older days Deb/Ubuntu did the right thing automatically, whatever value you preseeded with was what ended up in sources.list but that appears to no longer be the case.

The LinuxCOE post-processing environment runs in 4 stages, we call them PRE, MID, POST and the bundle installer proper.  You'll need to make a couple of changes to enable it.

In your $PREFIX/linuxcoe.rc file,  comment out (or set to 0) the NO_COE directive.   This tells the image generation scripts that you desire LinuxCOE post processing and they will append the bootstrap that calls it to your %final script.  If folks are currently using your system designer instance and you want to test/debug this functionality alongside but without touching your existing deployments, create the file $PREFIX/COE containing the single line 'NO_COE 0' (without quotes) and append ?defs=COE to your URLs, for example http://your_server/systemdesigner-cgi-bin/coe_bootimage?defs=COE .  when you use ?defs= on the URLs, it parses the standard linuxcoe.rc first and then another file overriding any values you've redefined.  We use this not only for testing but to give some internal customers a custom view.  

The post processing 'bootstrap' code it will append to your final script can be seen in the $PREFIX/data/*_final_skel files.  These skel files are written to layer any post processing needs onto a system in 3 stages.  On rpm based systems it runs a straight rpm -i @METHOD@/@WAYSTATION@/whatever.rpm kind of thing, on deb based system it will wget && dpkg -I the files.  The sole intention of the first 3 stages is to configure apt/yum/up2date/etc. to point at internal waystations and to deliver the LinuxCOE bundle installer.  If there you have ADDONS defined and chose custom bundles during image creation, a 4 step is called that would install this software using APT || YUM depending on distro now that it's configured to point at your waystation.

If you're simply wanting to ensure your servers end up pointed at your standard mirrors, you'll only need to install the deb that configures APT on the clients to point back at you.  The post processing stuff is on sf here:

http://linuxcoe.cvs.sourceforge.net/viewvc/linuxcoe/postproc/

http://linuxcoe.cvs.sourceforge.net/viewvc/linuxcoe/postproc/deb/apt-config/linux-2.6-all/linuxcoe-sw-sources-4.0-linux-2.6-all.deb?revision=1.1.1.1 - oops, I accidentally checked in the .deb vs. just source for it. :^)

I started using EPM to do my packaging a long time ago when I supported HPUX and Linux since I'm lazy and I only had to learn one packaging meta-language.  That's the 'format' of this check-in.  You can apt-get install epm, its home is http://www.epmhome.org/ .   You'll want to build linuxcoe-sw.deb and export somewhere on your waystation.  Then create the file $PREFIX/base/POST-DEB, here's our's for an example:

hpcoe@g2t0002:/usr/local/linuxcoe-sd/4.2/etc/base> cat POST-DEB
@ME...@li.../LinuxCOE/COE/3.1/packages/DEBS/linuxcoe-sw-sources-4.0.all.deb
hpcoe@g2t0002:/usr/local/linuxcoe-sd/4.2/etc/base>

The typical @METHOD@/@WAYSTATION@/etc. will get sub/replaced.  This will be pulled in with a wget/dpkg -i.   

The last piece to configure tells the linuxcoe-sw-sources.deb what to replace the distro's default apt configuration with.  That's done in $PREFIX/depots/, here's our Maverick example:

hpcoe@g2t0002:/usr/local/linuxcoe-sd/4.2/etc/depots> cat apt-Ubuntu-Maverick 

# core distro
deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick main restricted
deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick main restricted

# additional goodies
# deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick universe multiverse
# deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick universe multiverse

# updates
deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-updates main restricted
deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-updates main restricted

# additional updates (should enable these if doing the corresponding ones above)
# deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-updates universe multiverse
# deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-updates universe multiverse

# security
deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-security main restricted
deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-security main restricted

# additional security (should enable these if doing the corresponding ones above)
# deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-security universe multiverse
# deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-security universe multiverse

# backports
#deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-backports main restricted
#deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-backports main restricted

# additional backports (should enable these if doing the corresponding ones above)
# deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-backports universe multiverse
# deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-backports universe multiverse

# proposed
#deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-proposed main restricted
#deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-proposed main restricted

# additional proposed (should enable these if doing the corresponding ones above)
# deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-proposed universe multiverse
# deb-src  @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-proposed universe multiverse

hpcoe@g2t0002:/usr/local/linuxcoe-sd/4.2/etc/depots>

I realize this is a lot to digest and kind of rambling but give a whirl and see what you end up with.  All of the post processing leaves its logfiles at /var/log/LinuxCOE-* on the installing system.    Bryan and I will be happy to help.

Best Regards,

Lee