Menu ▾ ▴

Home

Attila Krasznahorkay Jr.

Idea, motivation, history of the SFrame package

SFrame is supposed to be a general HEP analysis package based on ROOT trees. An analysis in HEP is often performed in cycles. Each cycle usually corresponds to a reduction, or more generally to a new treatment of the data (e.g. calculation of new quantities for each event). SFrame follows this cycle-based analysis approach by splitting an analysis in several cycles. Each cycle takes a number of ROOT trees (e.g. from different sources, like data or different MC generators) in a certain format as input and produces ROOT trees in a different output format. In addition control histograms are produced. SFrame allows the combination of multiple input ROOT trees potentially coming from different physics processes taking into account varying luminosity values to which these trees correspond. In addition, SFrame can properly handle cuts applied on MC generator level of the input trees. The output trees of a certain step are supposed to be the input trees of the next step.

SFrame is a framework in which users can implement their analysis after they have produced the ROOT Trees for the various data sources. All analysis-specific steps are still under the control of the user. In particular the implementation of all analysis cycles is in the hand of the user: (S)he only has to provide an ExecuteEvent(...) method for each cycle in which the selection/calculation steps and the histogram filling is done. All the framework functionality is provided by SFrame: I/O of trees, I/O of histograms, loop over events, weighting, etc... As steering parameters, the user has to provide some meta-data describing the cycle (e.g. integrated luminosity the background should be weighted to), the input ROOT trees (type of physics process, integrated luminosity, cuts applied on generator level) and the output format. These meta-data are provided in an XML format.

The work on SFrame started in the context of the effort of the CERN ATLAS trigger group in analyses in Supersymmetry. Initially SFrame was intended to work on the ROOT trees provided by the Atlas SusyView package. However, SFrame developed towards a SUSY indepdendent package that can be used for any HEP analysis based on ROOT trees. Institutes involved in the development are CERN, University Hamburg and New York University. Meanwhile it is used by several groups (DESY, CERN, Manchester, University Hamburg, NYU, Bonn University).

Technicalities

SFrame is a C++ package that is independent of the ATLAS software. It only depends on ROOT and works with all ROOT versions starting with 5.14/00. The code is usually being developed against the newest production release of ROOT. Currently this is version v5-34-03 of ROOT. SFrame is currently supported/used on both Linux and MacOS X platforms. (Support for Windows has been dropped at one point.)

The repository

The SFrame code is now in a new SourceForge SVN repository under here. (Available more conveniently from the bar at the top of the page...)

The old SourceForge repository should remain accessible for a while longer under https://sframe.svn.sourceforge.net/svnroot/sframe/, but it won't be accessible for ever.

A note to ATLAS users: Some common, ATLAS-specific SFrame packages can be found under /atlasgroups/Misc/sframe. (Protected page, only accessible to ATLAS members.)

The structure of the code

The SFrame package builds 4 separate libraries, and an executable. Short descriptions of the libraries follows:

  • SFrameCore: Built from the sources under SFrame/core. It holds the main classes of the framework for controlling the execution of an analysis cycle.
  • SFramePlugIns: Built from the sources under SFrame/plug-ins. It holds extra classes that can be useful in physics analyses.
  • SFrameCintex: Built from the sources under SFrame/cintex. It can be used to successfully mix Reflex and CINT dictionaries at runtime in an analysis job.
  • SFrameUser: Built from the sources under SFrame/user. This library is a skeleton for physics analysis code built on top of SFrame. It holds two example analysis cycles, which demonstrate the features of SFrame. It also holds example configurations of these cycles. Note, that this library is only serving as an example, the users should in principle start by creating their own SFrame analysis package. (See section [UserCodeCreation] for details.)

The main SFrame executable is called sframe_main, and is built from SFrame/core/app/sframe_main.cxx. The executable expects the name of exactly one XML file describing the analysis cycle as a command line parameter. It is only linked against SFrameCore, as the executable only uses code from this library. Any additional libraries which are needed for the analysis (additional ROOT libraries, the user's own library holding his/her cycles) are loaded dynamically, and have to be specified in the configuration XML file.

Obtaining, compiling and using the package

SFrame is supported on two platforms:

  • Linux systems with a standard ROOT installation: The testing/development is being done mainly on SLC5 using the AFS installation (/afs/cern.ch/sw/lcg/external/root/) on ROOT.
  • MacOS X systems with either "standard" or Fink ROOT installation: The testing is currently done on 10.8 (Mountain Lion) with a Fink and a standard ROOT installation.

If you're on MacOS X and using Fink to install ROOT, you only have to make sure that the Fink directories (from /sw) are correctly configured in your environment. (Usually by sourcing /sw/bin/init.sh in your login script.) When using a "standard" installation of ROOT (either on Linux or MacOS X), you have to set the following environmental variables:

ROOTSYS = /location/of/root
PATH = $ROOTSYS/bin:$PATH
LD_LIBRARY_PATH = $ROOTSYS/lib:$LD_LIBRARY_PATH

In many situations this can be exchanged by just sourcing /location/of/root/bin/thisroot.[c]sh. (But not always!)

Since SFrame-02-01-00 SFrame also indirectly depends on Python. (Through the ROOT bindings.) Because of this, you also have to make sure that your environment is set up to use the same version of python that your version of ROOT was compiled against. For instance ROOT 5.20 and newer versions are compiled against Python 2.5, while the default Python version on SLC5 is 2.4...

The latest versions of ROOT now don't compile XRootD by themselves, but rely on XRootD having been built externally. This means that if your ROOT version was compiled against some specific XRootD version, you have to make sure that this version of XRootD is also in your runtime environment.

All in all, on an AMD64 SLC5 machine you could make the following settings to get the latest version of ROOT set up within CERN:

export ROOTSYS=/afs/cern.ch/sw/lcg/app/releases/ROOT/5.34.03/x86_64-slc5-gcc43-opt/root
export PYTHONDIR=/afs/cern.ch/sw/lcg/external/Python/2.6.5/x86_64-slc5-gcc43-opt
export XROOTDDIR=/afs/cern.ch/sw/lcg/external/xrootd/3.2.2/x86_64-slc5-gcc43-opt
export LD_LIBRARY_PATH=$ROOTSYS/lib:$PYTHONDIR/lib:$LD_LIBRARY_PATH
export PATH=$ROOTSYS/bin:$PYTHONDIR/bin:$PATH
export PYTHONPATH=$ROOTSYS/lib:$PYTHONDIR/lib:$PYTHONPATH
source $ROOTSYS/bin/setxrd.sh $XROOTDDIR

Wherever you do the setup, most importantly $ROOTSYS/bin/root-config has to be in your $PATH. Don't forget also that on SLC5 the default GCC compiler is GCC 4.1. Since CERN only officially supports GCC 4.3 for the SLC5 projects, you also have to set up GCC 4.3 by hand. To do this on lxplus, the simplest thing that you can do is:

source /afs/cern.ch/sw/lcg/contrib/gcc/4.3/x86_64-slc5-gcc43-opt/setup.[c]sh

To check out the code, select the directory where you want to put/compile it. Check out the latest version of the code at the time of writing, with:

svn co svn://svn.code.sf.net/p/sframe/code/SFrame/tags/SFrame-03-06-11 SFrame

Note: When starting to work with SFrame, it's always a good idea to start using the newest tagged version from the main branch in SVN. Bugfixes and small improvements are usually added more often than this page is updated...

Note: Since SFrame-03-00-00 the code from the "old" SFrame-PROOF branch was migrated back into the main development line (trunk). The old code was moved to a separate branch, called SFrame-classic-branch.

Go to the SFrame directory, and execute:

source setup.[c]sh
make

This will build all libraries and the executable of the package. The libraries are put under SFrame/lib/ and the executable is put under SFrame/bin/. (They are both put in your environment by the setup.[c]sh script.)

Running the examples

There are two example cycles in the SFrame/user/ directory. They are called FirstCycle and SecondCycle. There are two XML files configuring these cycles under SFrame/user/config/. They are meant to work out-of-the-box on lxplus.

To run the first example, go to SFrame/user/config/, and execute:

sframe_main FirstCycle_config.xml

The executable will read in the FirstCycle_config.xml file, and execute the FirstCycle cycle. The cycle will create two output ROOT files named: FirstCycle.MC.Zee_1.root and FirstCycle.MC.Zee_2.root.

The second example reads in one of the files created by the first example (FirstCycle.MC.Zee_2.root) and creates some histograms from the data stored in the file. Thus demonstrating how the idea of multiple cycles works. To run the example, execute the following in the same directory:

sframe_main SecondCycle_config.xml

Notice, that the second cycle runs over less events for MC.Zee_2 than the first cycle did. This is because FirstCycle executed some basic event selection, and only the events that passed the selection were written into FirstCycle.MC.Zee_2.root.

Implementing a user analysis package

This section explains how to use SFrame to create a full analysis. All the basic ideas and basic features of the code are explained. If you are new to SFrame, or have not tried to implement a complicated analysis in the SFrame framework before, this should be a useful read. This section is on page [UserCodeCreation].

Advanced features of the code

There are many features in the code that one doesn't have to use in simple analyses, but which can be quite useful in a large analysis code. These features are collected and explained on the page [AdvancedFeatures].

Using the additional scripts

The SFrame package provides a number of scripts that are meant to aid various steps of an analysis. The function and usage of these scripts is explained on the page [SFrameScripts].

More information on SFrame with PROOF

SFrame is capable of taking advantage of the PROOF functionality of ROOT. There are a number of concepts that the users should be well aware of when using a PROOF cluster (or even PROOF-Lite). This information is collected on the page [SFramePROOF].

SFrameARA

SFrameARA is a relatively thin layer above SFrame that binds it to the ATLAS offline software. It's purpose is to enable the user to use AthenaROOTAccess with SFrame. Detailed information about it is given on the page: [SFrameARA].

Running SFrame with Ganga

It is now possible to run SFrame on the GRID by using the SFrameApp plugin for the GRID job management application Ganga. The usage of this Ganga plugin is explained on the page [SFrameGanga].

Running SFrame jobs on Panda

The Panda user interface provides a very flexible way of running basically any kind of applications/scripts on the GRID. The following page explains how to use Panda to run SFrame (or even SFrameARA) jobs on the GRID with the help
of Panda: [SFramePanda].

Developers

The following people have contributed to the SFrame package:

  • Stefan Ask - U. of Manchester
  • David Berge - CERN
  • Nicolas Berger - CERN
  • Till Eifert - U. of Geneva
  • Andreas Hoecker - CERN

The current developers are:

Project Admins:

Comments, questions, suggestions, contributions

If you have comments, questions or suggestions or you would like to see additional functionalities of SFrame -- and you have an account at CERN --, please send an email to one of the developers mentioned above or post to (atlas-sframe-users@cern.ch). That list is only for people with CERN computer accounts though, so other people should use the feedback systems provided by SourceForge. It would actually be interesting to see some non-particle physicists use the software.