Menu ▾ ▴

Home

Mark Dixon

Flavours - Environment Modules add-on library

Flavours is an add-on library for the popular Environment Modules package, used to make optional software components available to users on Unix platforms.

What is it for?

Flavours is an attempt to resolve the "permutation problem". Modules is a great tool to use, but the number of modules needed can be excessive if multiple builds of the same software is required. This can be very confusing for the user.

For example, you may want to install a Fortran/C/C++ library, but your system may have multiple compilers with incompatible runtimes and perhaps you have some users requiring 64-bit and others 32-bit. For a library written to take advantage of parallel programming techniques, you might also have multiple MPI libraries to build against.

This is frequently the case on general purpose HPC (High Performance Computing) machines. The usual way to resolve this is to have a different module for each combination of dependencies. For example, there might be a number of modules installed for the fftw package:

$ module avail fftw
----------------------- /apps/Modules/libraries ------------------------
fftw/2.1.5/gcc      fftw/2.1.5/intel       fftw/2.1.5intel32bit
fftw/2.1.5/intel12  fftw/2.1.5/gcc-openmpi fftw/2.1.5pgi
fftw/2.1.5/gcc32bit fftw/3.2.2/pgi         fftw/3.2.2/gcc

This can quickly get out of hand.

Flavours abstracts all this away, and turns it into something considerably clearer:

$ module avail fftw
----------------------- /apps/Modules/libraries ------------------------
fftw/2.1.5       fftw/3.2.2

This is useful for more than just compilers and MPI libraries. Flavours can be used to model any permutations you like.

How does it work?

Flavours works by allowing modulefiles to declare interdependencies, via suped-up versions of the existing "prereq" and "conflict" Module commands.

  • Each permutation of prerequisites (architecture and MPI, etc.) is known as a flavour and are not displayed by the "module avail" command.
  • The appropriate flavour is automatically selected by the module command, based on what prerequisite modules are currently loaded.
  • If a prerequisite module is changed (e.g. switched from GNU to Intel compiler), the correct flavour for all dependents is automatically updated.
  • You can add a prerequisite on a single module name or define a class of modules (those that provide a particular set of functionality - e.g. compilers, MPIs).

Each flavour translates to a unique location on disk where Flavours expects to find an appropriate version of the software.

This reduces the management problem to installing compatible builds of the software in the correct locations.

What other benefits are there?

If you are using compilers and libraries installed as flavours, it can become incredibly easy to try out a different compiler or MPI.

This is great news as different implementations tend to have different strengths and weaknesses appropriate in different situations, and so trying them out should be encouraged. For example:

  • One compiler might be slow but tell you about non standards-compliant code you might want to rewrite.
  • Another compiler might generate incredibly fast code - but that code may keep happily running even after it should have segfaulted.
  • A third compiler might perform the essential task of telling you off for using swear words as variable names.
MongoDB Logo MongoDB