Menu
▾
▴
Re: [Linuxcoe-users] Using Mirrors
|
From: Mayes, L. <lee...@hp...> - 2010-12-14 13:45:55
|
Hi John, What you want to do is certainly do-able and is how we run our shop as well. While there is nothing proprietary about our post processing environment it's probably not our best documented feature even though we live and die by it on our internal System Designer instance. In the older days Deb/Ubuntu did the right thing automatically, whatever value you preseeded with was what ended up in sources.list but that appears to no longer be the case. The LinuxCOE post-processing environment runs in 4 stages, we call them PRE, MID, POST and the bundle installer proper. You'll need to make a couple of changes to enable it. In your $PREFIX/linuxcoe.rc file, comment out (or set to 0) the NO_COE directive. This tells the image generation scripts that you desire LinuxCOE post processing and they will append the bootstrap that calls it to your %final script. If folks are currently using your system designer instance and you want to test/debug this functionality alongside but without touching your existing deployments, create the file $PREFIX/COE containing the single line 'NO_COE 0' (without quotes) and append ?defs=COE to your URLs, for example http://your_server/systemdesigner-cgi-bin/coe_bootimage?defs=COE . when you use ?defs= on the URLs, it parses the standard linuxcoe.rc first and then another file overriding any values you've redefined. We use this not only for testing but to give some internal customers a custom view. The post processing 'bootstrap' code it will append to your final script can be seen in the $PREFIX/data/*_final_skel files. These skel files are written to layer any post processing needs onto a system in 3 stages. On rpm based systems it runs a straight rpm -i @METHOD@/@WAYSTATION@/whatever.rpm kind of thing, on deb based system it will wget && dpkg -I the files. The sole intention of the first 3 stages is to configure apt/yum/up2date/etc. to point at internal waystations and to deliver the LinuxCOE bundle installer. If there you have ADDONS defined and chose custom bundles during image creation, a 4 step is called that would install this software using APT || YUM depending on distro now that it's configured to point at your waystation. If you're simply wanting to ensure your servers end up pointed at your standard mirrors, you'll only need to install the deb that configures APT on the clients to point back at you. The post processing stuff is on sf here: http://linuxcoe.cvs.sourceforge.net/viewvc/linuxcoe/postproc/ http://linuxcoe.cvs.sourceforge.net/viewvc/linuxcoe/postproc/deb/apt-config/linux-2.6-all/linuxcoe-sw-sources-4.0-linux-2.6-all.deb?revision=1.1.1.1 - oops, I accidentally checked in the .deb vs. just source for it. :^) I started using EPM to do my packaging a long time ago when I supported HPUX and Linux since I'm lazy and I only had to learn one packaging meta-language. That's the 'format' of this check-in. You can apt-get install epm, its home is http://www.epmhome.org/ . You'll want to build linuxcoe-sw.deb and export somewhere on your waystation. Then create the file $PREFIX/base/POST-DEB, here's our's for an example: hpcoe@g2t0002:/usr/local/linuxcoe-sd/4.2/etc/base> cat POST-DEB @ME...@li.../LinuxCOE/COE/3.1/packages/DEBS/linuxcoe-sw-sources-4.0.all.deb hpcoe@g2t0002:/usr/local/linuxcoe-sd/4.2/etc/base> The typical @METHOD@/@WAYSTATION@/etc. will get sub/replaced. This will be pulled in with a wget/dpkg -i. The last piece to configure tells the linuxcoe-sw-sources.deb what to replace the distro's default apt configuration with. That's done in $PREFIX/depots/, here's our Maverick example: hpcoe@g2t0002:/usr/local/linuxcoe-sd/4.2/etc/depots> cat apt-Ubuntu-Maverick # core distro deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick main restricted deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick main restricted # additional goodies # deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick universe multiverse # deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick universe multiverse # updates deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-updates main restricted deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-updates main restricted # additional updates (should enable these if doing the corresponding ones above) # deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-updates universe multiverse # deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-updates universe multiverse # security deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-security main restricted deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-security main restricted # additional security (should enable these if doing the corresponding ones above) # deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-security universe multiverse # deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-security universe multiverse # backports #deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-backports main restricted #deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-backports main restricted # additional backports (should enable these if doing the corresponding ones above) # deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-backports universe multiverse # deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-backports universe multiverse # proposed #deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-proposed main restricted #deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-proposed main restricted # additional proposed (should enable these if doing the corresponding ones above) # deb @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-proposed universe multiverse # deb-src @METHOD@@WAYSTATION@/LinuxCOE/Ubuntu maverick-proposed universe multiverse hpcoe@g2t0002:/usr/local/linuxcoe-sd/4.2/etc/depots> I realize this is a lot to digest and kind of rambling but give a whirl and see what you end up with. All of the post processing leaves its logfiles at /var/log/LinuxCOE-* on the installing system. Bryan and I will be happy to help. Best Regards, Lee -----Original Message----- From: Cusick, John CTR NAWCWD, 541400D [mailto:joh...@na...] Sent: Monday, December 13, 2010 6:58 PM To: Mayes, Lee; Gartner, Bryan W Cc: lin...@li... Subject: RE: [Linuxcoe-users] Using Mirrors Lee and Bryan, Actually, what I'm trying to do is have the systems install from my waystation and not access the Ubuntu (or Debian) public mirrors at all... ever. Management wants a single point of installation and updates with control over what gets installed and updated, particularly the kernel. They are lenient on choice of distributions, however, which is why I chose LinuxCOE as a starting point. That and the web-based installation disk creation which is perfect in an environment where only 1 of the Sys Admins is technically responsible for all the Linux systems here, but all sys admins may be required to install, and secure, the Linux Desktops. Eventually (shortly) this will be required for all the rpm-based and deb-based distros we support here. I've set up two different LinuxCOE systems, one for debs and one for rpms, including appropriate mirrors (centos, fedora, ubuntu, etc., depending on user prefs/requirements), and since I'm less experienced with debian way than Redahat/Fedora/CentOS I figured I'd start with the deb system setup first since I would find it more interesting :-) Also, due to some restrictions on network access locally, some of the chosen US mirrors are blocked or tightly choked. For example, the basic desktop install I tested last week took over 2 hours across the 'net. At home the same basic install took about 45 minutes complete to the working desktop, and my ISP at home supplies a far smaller bandwidth than I have available here. Not so good when I'm trying to grab some bragging rights with the local MS Guys :-) I figured that since we must have the mirrors, the LinuxCOE system would be a perfect adjunct for non-linux-oriented sys admins to put together installations with a minimum time and knowledge requirement while also meeting our platform and security update requirements. That, and the PRE, MID and POST stuff looks very interesting for downloading locally required security packages, documentation, scripts, etc. to speed up the final hands-on portion of the installation. My perl skills are inadequate at best, although I did spend some time with the source code last week which helped, but right now I'm not really sure of the most effective way to do all this. I tested what I interpreted the answer to be (below), but as you pointed out, Lee, it isn't working as setup out of the box. (LinuxCOE 4.3, by the way - installation and basic setup from source was a breeze, and Thank You all for that). Is it possible to setup a post-processing environment outside a H-P Facility? From what I saw in the code and read in various docs online at H-P, it looks like there may be some proprietary software involved. Or do I need to rebuild the appropriate apt debian packages (assuming they are delivered as part of the waystation package) so that they point to my server mirror only? Regards, John C. -----Original Message----- From: Mayes, Lee [mailto:lee...@hp...] Sent: Monday, December 13, 2010 12:24 To: Cusick, John CTR NAWCWD, 541400D; Gartner, Bryan W Cc: lin...@li... Subject: RE: [Linuxcoe-users] Using Mirrors The depots directory is scraped and applied only if running the full LinuxOE post-processing environment. You'll also need to set up the stuff in linuxcoe-sd/base/ stuff to reference our post processing RPM. BryanG, do we do that out of the box? John, If repointing the installed boxes to your waystation is all you need you might just sed sources.lists.d/* to populate with your values in the final script. Lee -----Original Message----- From: Cusick, John CTR NAWCWD, 541400D [mailto:joh...@na...] Sent: Monday, December 13, 2010 3:08 PM To: Gartner, Bryan W Cc: lin...@li... Subject: Re: [Linuxcoe-users] Using Mirrors Bryan, Wow! That was a very quick reply. Ah... so I think I got it! In other words, I use the linuxcoe-sd/depot directory and, for example, the apt-Ubuntu-Maverick file would have the following entries, (ubuntu-lds being the name of my server): deb @ME...@ub.../ubuntu/ maverick main multiverse restricted universe deb @ME...@ub.../ubuntu/ maverick-updates main multiverse restricted universe whereas in the linuxcoe-sd/osvend.d the Ubuntu file has the following entry: OSVEND ubuntu-lds.my.domain.mil Ubuntu Maverick x86_64 HTTP /ubuntu/ At image creation time, in my case using http, @METHOD@ will be translated to http, Correct? I'm a little new to mirroring debian-based distros and I've not always been sure of the correct syntax, although I am now making the assumption that it should match the standard sources.list file (not to mention that I'm not really strong with perl, either :-). That may have been where my initial tests had failed to do what I expected as the linuxcoe-sd/depot/ files were not 'right' compared to a typical sources.list file. Thanks again. Regards, John Cusick -----Original Message----- From: Bryan Gartner [mailto:bry...@hp...] Sent: Monday, December 13, 2010 11:34 To: Cusick, John CTR NAWCWD, 541400D Cc: lin...@li... Subject: Re: [Linuxcoe-users] Using Mirrors John, On Mon, Dec 13, 2010 at 06:55:05PM +0000, Cusick, John CTR NAWCWD, 541400D wrote: > I've setup LinuxCoe locally and first tests have been basically successful. Good deal, feel free to follow-up with other questions you may have. > I do have one question regarding waystations, however. > > I was under the impression that the entire workstation would be loaded from the server entered as the OSVEND, but that does not seem to be the case. Although the install starts from there, the packages are downloaded from outside-the-local-network mirrors instead of the locally setup mirror. Actually, think of OSVEND as a simple point mechanism to any valid distribution mirror. > Is there a way from within LinuxCOE to force the local mirror as the only download point? Specifically, I have an Ubuntu waystation/mirror setup and although it correctly used the waystation/mirror during the initial load phase, once the software install started it accessed us.archive.ubuntu.com. I would like it to use the local mirror only. You can augment/replace any tuple (distro/version/arch) combination with as many mirrors as you'd like, whether remote or local to your environment. Simply follow the syntax examples in the file of interest. > In case this is not the best place to ask questions, is there a better list or forum to use? This is entirely fine, ask away :) bryang |
