DXTBX design

DXTBX design proposal 2013/APR/23

Suggestions for the interface designs for image sets / sweeps to deal with the problems set out yesterday in the DIALS phone conference namely:

• sweeps of images (traditional MX rotation data sets)
• sets of images (XFEL data sets)
• larger containers with multiple image sweeps / sets in
• construction of all of the above from an arbitrary list of files

Basic Principle

Define a basic type ImageSet: this will reflect a set of images recorded with a consistent source and detector geometry, for example a sequence of XFEL images. Define derived type ImageSweep: this is an ImageSet with an additional rotational dimension, such that there is a continuous relationship between the image pixel values in three dimensions (i.e. two on the image and a third due to rotation).

Have a factory type which will return a list of ImageSet's based on input files:

Factory(files):
    result = []
    for i_set in CreateImageSets(files):
        if SatisfiesSweepCriteria(i_set):
            result.append(ImageSweep(i_set))
        else:
            result.append(i_set)

This means that the result from the factory will always be a list of Sets some of which may be Sweeps.

Some assumptions:

• if given a list of filenames, these filenames are sorted in the way they should be interpreted e.g. alphanumerical order

• sets are continuous within this list i.e.

  |A|B|C| NOT |A|B|A|C|

Bridge to Legacy Code

DetectorBase is used all over the place. At the moment these are available from Format objects. ImageSet needs to have a method detectorbase(j) where j can be some index of the set: numerical (based on slice or frame ID) by filename or if empty or None the current frame. Add Python iterators to the ImageSet so we can have for image in ImageSet...

Phases
1: This uses the existing methods to generate the detector base instances
2: Generate a detector base on-the-fly from the models etc (e.g. compute beam centre)
3: Use interfaces properly in calling code (certainly aim for this with new code)

Multi-Image Files

These need their own equivalent of a Format class - multi-format perhaps, which coould inherit from format but would give things useful for a multidimensional data set. This would only require slight modification of the principles behind the factory to work, and could share sweep / set interfaces. N.B. for the HDF5 interface assume all images in a given set or sweep are in the same HDF5 group (similar to principle with filenames above.)

N.B. all sweeps / sets will share the same FileReader and hence share the same HDF5 handle.

Stepping Through Files

For many cases it would be useful to be able to step through a file with e.g. next() method for the sweep / set or to have for frame in set: so adding Python iterators here would be helpful. Potentially this is not thread safe: for a given sweep this is almost certainly not thread safe, but for the use of several sweeps pointing at one filereader we may also need to check thread safety. The state for the iterator though should be kept in the sweep object.