Re: [Helidelinux-devel] Package manager

[Bennett T. <be...@ra...>]

2004-03-15T21:31:14 Hui Zhou:
> I have no experience with package managers.

I have quite a bit; I'll try and share some:-). But note a Great
Big Disclaimer: different folks, with different experience with
different package managers used to do different jobs, come out with
different opinions; Debian's dpkg, the BSD "ports" facility, the
package managers for Gentoo and Sourcerer, and I'm sure any number
of other software package management strategies all have their own
following.

> My first linux system is redhat, [...]

Not a bad exemplar, in my opinion; I think rpm is the most capable
software package manager currently in existence. I also think it's
grown a bit overripe, long in the tooth. That's why I'm implementing
my own software package manager.

> [...] installed and never used because I don't know how it was
> managed.

I can strongly recommend you at least skim, if not read, the book
Maximum RPM, available from the rpm web site at
<URL:http://www.rpm.org/>. Even if you end up going in a different
direction, that book is accessible documentation of all the major
capabilities of one of the best software package managers in
existence today.

> Then I tried linux from scratch,  and now that's the only
> thing I know how to work with.

LFS is a _lovely_ place to learn how to build Linux systems, but
plain LFS has no software package management at all.

> I am thinking the automated lfsbuilder may finally develop into a
> package manager also.

Could be. If you try and grow it into a package management system, I
think you'll lose the lovely density you've got, with so few lines
of config describing the complete build. But I may be wrong!

> What is generally expected for a package manager?

"generally" can't be answered, different people have different
expectations. I'll go into this more below.

> The minimum I believe is just install and uninstall.

That's definitely not sufficient; a software package manager must
also offer inventory (what packages are installed), dependancy
management of some sort (what packages does this package depend on,
at a minimum), integrity checking (are the files in this package
intact on the system?), and pre/post install/remove scripts.

Modern software package managers also undertake to provide build
reproduceability, with automated package build from sources and some
provision for build dependancy management.

The first software package management system I know of was the
System V Packaging Tools. They completely ignored the package
building automation side of things, they had week integrity
checking, and they had quite poor performance, but nonetheless they
work well enough to make Sun's Jumpstart a sound tool for enterprise
use, and I've used them to build enterprise system configuration
management, with automatic software inventory and system rebuilding
and automated update deployment (at a previous job, work for hire, I
don't have the code, it's not open source).

rpm was built to tackle a harder problem; Red Hat supports a rich
distro, with packages drawn from diverse upstream sources, across
multiple architectures. When I got started with Red Hat, they were
supporting i386 and Alpha. They update pretty aggressively, it's not
hard to stay fairly up to date with Red Hat, and they automated both
system builds and system updates. rpm grew to support their quite
demanding needs.

The only other software package management system I've looked at
closely that tackles the same problem space so well is Debian's
dpkg, and it has (or at least had, when last I looked at it), two
features that put me off, left me favouring Red Hat. First, the
sources dpkg works from aren't the virgin upstream sources; instead,
they're a repackaging of the upstream tarball, and there's no place
where the URL of the upstream source is automatically and reliably
available. And second, you couldn't build a binary package from a
source package as a non-root user.

To look at the set of features a software packaging tool must
support to compete with rpm, consider some tasks that it must
assist in:

- Easily wrapping new packages

- Easily updating existing packages to track new versions of the
  up-stream software

- Easily wrapping packages that apply patches to the upstream
  software, either to fix bugs quicker than the upstream integrates
  the fixes, or to adjust the package to build the way you want it
  on your dev platform

- Assist in automating complex package installations; it should be
  possible to build something like "apt", that can be pointed at a
  package repository, and automatically install all prerequisites

- Automate installing systems --- think Sun's Jumpstart, Red Hat's
  kickstart. A fully automatic system build to a "profile" (defined
  as a package list) is vital for enterprise computing. Updating the
  build to track updates to packages must be easy.

- Automate delivery of patches and updates; the update process must
  be so robust and safe that it can be automated over tens of
  thousands of machines with a near-zero failure rate and clear
  error reporting.

- Automate inventory of system configuration; with simple scripting
  you should be able to populate an RDBMS with every version of
  every package installed on every system in a large enterprise

When you go through those cases, you'll find that many, perhaps most
of the features of rpm get their motivation. I don't begrudge rpm
its rich feature set, if I don't abandon the project bpm will grow a
comparable one. I believe (I may be proven wrong) that a complete
redesign can make it possible to deliver the features on a much,
much simpler implementation; rpm "just grew", it's requirements were
discovered as its implementation evolved. This is understandable in
a new prototype, but the end result is not likely to be a clean
implementation. A complete reimplementation from scratch can do
much, much better. Sendmail -> {qmail, postfix} is a clear example;
the two modern MTAs are vastly faster, more robust, more secure, and
simpler to configure and manage than sendmail. Sendmail evolved as
the requirements of an internet MTA were discovered; in fact, it
pioneered so many features that it's the driver for the required
feature set.

-Bennett