Title: Continuous Scan Implementation
SEP: 6
State: DRAFT
Date: 2013-07-29
Drivers: Zbigniew Reszela <zreszela@cells.es>
URL: https://sourceforge.net/p/sardana/wiki/SEP6
License: http://www.jclark.com/xml/copying.txt
Abstract:
Generic Scan Framework requires extension for a new type of scans: continuous scans.
Various types of synchronization between moveable and acquisition elements exists:
software (so called best effort) and hardware (either position or time driven).
The challenge of this SEP is to achieve the maximum generalization and transparency
level between all types of continuous scans (and probably step scans as well). This
design and implementation will require enhancement of the already existing elements of
Sardana: controllers, moveables, experimental channels and measurement group and
probably definition of new elements e.g. triggers. A proof of concept work was already
done at ALBA, and it could be used as a base for the further design and development.
In the present situation, step scans can be executed using the software synchronization mode. The software-synchronized continuous scans are already implemented in Sardana as well and are described here. A reduced API abstracts some specifics of the totally hardware-synchronized systems, however requires some generalization.
A set of scan macros with the "ct" suffix allow to execute continuous scans. The Experimental Channel pseudo-API is based on the following extra attributes: "NrOfTriggers", "AcquisitionTime", "TriggerMode" and "Data". Control over start and stop of the experimental channels is achieved via "SendToCtrl" method of the controllers. The following string arguments should be understandable by the controller: "pre-start <axis>", "start <axis>", "pre-stop <axis>", "stop <axis>". Configuration of the hardware-triggering objects is done via "TriggerDevice" environment variable. It accepts a list of the Tango device names, where the first one is the master trigger. The pseudo-API of the triggering-object is based on the following attributes: InitialDelayTime, HighTime, LowTime (all in seconds), SampPerChan (used to configure number of triggers) and IdleState (Low or High). This interface is also based on two commands: "Start" and "Stop". Current implementation extracts the acquired data at the end of the scan.
A proof of concept of the "online data visualization and storage" was implemented in the "ASCANCT" branch of the former Sardana svn repository, what later was renamed to the "sep6" branch in the current git repository. Previously required be the Experimental Channel pseudo-API "AcquisitionTime" attribute was replaced by the "IntegrationTime" attribute of the Measurement Group. Its value gets passed to the controller via a sequence of Load[One,All] calls. "TriggerMode" attribute was substituted by the Measurement Group configuration parameter "Trigger". Calls to the "SendToCtrl" method were substituted by calls to the [Pre]Start[One,All] controller methods. During scan execution, poolacquisition action extracts the already available data (using sequence of [Pre]Read[One,All] calls) and pushes Tango CHANGE_EVENTs of "Data" attributes. The Measurement Group representative on the Macroserver side is subscribed to these events. The callback execution feeds this data to the DataHandler, which decides if a complete record is ready. If so, it gets passed to the registered recorders.
Both of this approaches supports the following hook places: "pre-scan", "pre-configuration", "post-configuration", "pre-start", "pre-cleanup" and "post-cleanup"
Scans are divided, according to the synchronization mode, into: step, continuous and hybrid. This SEP does not try to achieve a total transparency between them.
A step scan can be executed with one of the standard macros e.g. ascan, d2scan, mesh. While a continuous scan can be executed with their equivalents terminated with 'c' suffix.
The continuous scans can use hardware or software synchronization mode. A transparency between these two should be achieved, maintaining the same macro parameters e.g.
ascanc <motor> <start_pos> <final_pos> <nr_interv> <integ_time>
In case of totally software synchronized continuous scans, fixed (nr_interv + 1) number of scan points can not be ensured.
The synchronization mode is configured on the Measurement Group level, for every Experimental Channels. Three configuration parameters: "Synchronization", "Trigger/Gate Channel" and "Timer/Monitor" must be selected prior to the scan execution.
Synchronization
One of the following options must be selected for every Experimental Channel present in the Measurement Group:
Trigger/Gate Channel
In case of gate or trigger synchronization, this parameter specify which of the Trigger/Gate Channel is used by a particular Experimental Channel. If the software synchronization is selected, the Trigger/Gate parameter is ignored for the continuous scan.
Timer/Monitor
The Timer/Monitor parameter indicates which of the experimental channels controls the acquisition process duration. If the integration time used by the scan has a positive value, the acquisition process ends when the integration time passes (timer). The negative values of the integration time, indicates that the acquisition process ends when the number of counts indicated by the integration time is reached (monitor). The monitor feature is out of the scope of this SEP.
In case of software or trigger synchronization Timer/Monitor is responsible for the internal gating (if supported by hardware) or software gating (using the operating system timer and forcing the acquisition to stop). If the gate synchronization is selected, Time/Monitor parameter is simply ignored - external gating is used.
Depending on the selected configuration, the Measurement Group could contain only software synchronized channels, only hardware synchronized channels (using gates and/or triggers) or mix of both of them.
In case of pure software synchronization, the acquisition process, equivalent to the one from step scan, is repeated as often as possible during the motion period. Because to this best effort approach, the amount of the scan records can not be determined "a priori".
In case of pure hardware synchronization, the process of configuration and arming is performed at the beginning and reconfiguration and stop at the end of the scan. During the scan execution, channels which support data queries while acquiring are progressively asked for new data. The channels which do not support this feature are asked for all the data at the end of the scan.
In case of mixed synchronization (software and hardware), data are organized in scan records based on the time-stamp. Each data produced by the software synchronized channels is stamped with the operating system time of the machine as close to the hardware as possible (in the worst case of the machine hosting the Pool server). Ideally, the hardware synchronized Experimental Channels provide time-stamped data as well. If no hardware time-stamping mechanism is available, theoretically calculated, from starting point, active period, passive period and a trigger/gate index, time-stamps are used. The start of the trigger/gate generation process is time-stamped with the system time, and each of the acquired data is posteriorly stamped with the relative theoretical time of the trigger/gate. In case of the position driven trigger/gate, the motion path parameters are also used in the calculation process.
Measurement Group could contain external, to the Sardana system, channels. Similar to the software synchronized channels, as often as possible repeats of their instant readouts would be incorporated in the scan records - based on the time-stamp.
Detailed explanation about different synchronization mechanism can be found in 1.
Devices implementing multiple hardware functionalities like: motion control, trigger/gate generation, data acquisition, etc. are represented by just one Sardana controller. The main functionality of the hardware is used for determining the controller type and the rest of them are the supported extensions/interfaces. In case of hybrid hardware, with no main functionality, a pure HardwareController type could be used.
The controller elements are assigned to the axes.
More than one Sardana element could be assigned to one controller axis e.g. a motor and a trigger/gate would be occupying only one controller axis but would have two different names.
In a different scenario, a controller with multiple interfaces, could restrict a group of axes for one type of elements and different axes for the others. Multipurpose controllers must define which axes are available for which functionalities.
(Optional: One Sardana element, assigned to its axis, could be used for example as a trigger/gate as well as a motor. This way the same name would be shown in a list of motors, as well as in a list of trigger/gates.)
An example of defining the controller and its elements:
defctrl MyMotorController motctrl01 #MyMotorController is of type MotorController
defelem mot1 motctrl01 1 # Motor is the default type
defelem counter1 motctrl 1 CounterTimer
defelem trigger1 motctrl 1 TriggerGate
Trigger/Gate is a new controller type. It is used to abstract a trigger/gate generating device. Trigger or gate characteristics could be described in either of two domains: time or distance. Distance domain is used to relate triggers/gates with changing of a moveable axis. Similar to motor interface, translation from hardware distance unit to user distance unit is necessary.
Trigger configuration is separated into two parts:
Configuration of triggers/gates is possible in two modes: equidistant and arbitrary.
Parameters characterizing equidistant trigger/gate generation:
Arbitrary trigger/gate generation is configured with a list of each trigger/gate characteristics.
Trigger/Gate Controller (plug-in)
In the hardware synchronized acquisition, the Experimental Channels must know how many acquisitions will be performed (the step scan could also take a profit of this enhancement, allowing to perform multiple acquisitions per scan point). During the acquisition process channels may need to report more than one acquired data per readout. For that reason ReadOne method is allowed to return a spectrum of values, and the Value attribute may need to change its shape.
When using trigger synchronization mode, three different modes of controlling acquisition are possible: pre-trigger, mid-trigger and post-trigger. The pre-trigger acquisition mode, will start the acquisition right after the trigger is received. The post-trigger mode will recover the acquisition which had occurred just before the trigger was received.
The mid-trigger will recover half of the acquisition just before the trigger and the second half will start right after the trigger. This SEP will work only on the pre-trigger acquisition mode.
Experimental Channel controller (plug-in)
The hardware triggered/gated Experimental Channels are divided into two groups: the ones which allow querying while the acquisition is running, and the ones which not. This feature is defined on the controller level. The Controllers which support it, are sequentially asked for the new data. The controllers which do not support this feature are asked for all the data at the end of the acquisition process.
In order to mitigate the dead-time, the software triggered experimental channels are involved in the constant acquisition process composed from: configuration, acquisition and readout. This process stops when all the hardware synchronized channels stop acquiring.
The external attributes are treated similarly but their process just comprises the readout.
(Optional: to optimize time, the software synchronized channels and external attributes could read for the first time at the pre-start position)
Every value acquired or read during the continuous scan execution is stamped with an absolute time. In case of the Experimental Channels, the time-stamp correspond to the acquisition start and in case of the external attribute is provided by the external system. If the external system does not provide a time-stamp, Sardana attaches one corresponding to the readout return. The experimental channels synchronized by hardware (gate or trigger) provide the core part of each scan record. The software synchronized channels and external attributes provide values which are not organized in records. The Data Handler is responsible for composing the full records. All the software synchronized channels and external attributes acquire their initial values at the starting positions. The zero order hold ("constant interpolation") method is used to estimate a missing part of the records. Different interpolation methods could be available to the user end executed as a post-scan processing, however implementation of them is out of the scope of this SEP. The real data has to be easily distinguishable from the interpolated one, so each recorder could store/visualize them in its own way.
The acquisition process over the time:
The merged scan records:
#Pt No mot01 ct1 ct2 emet temp pressure emet_raw temp_raw pressure_raw
0 0 20 3 7.0 68.1 2.01e-5 7.5 - -
1 10 25 2 7.5 68.1 2.01e-5 - 68.3 -
2 20 60 5 7.5 68.3 2.01e-5 - - 2.02e-5
3 30 71 7 7.5 68.3 2.02e-5 5.5 68.5 -
4 40 63 5 5.5 68.5 2.02e-5 - - -
5 50 37 3 5.5 68.5 2.02e-5 - - 1.99e-5
6 60 20 5 5.5 68.5 1.99e-5 3.1 - -
7 70 15 7 3.1 68.5 1.99e-5 - 69.2 -
8 80 13 12 3.1 69.2 1.99e-5 1.2 - 2.00e-5
9 90 5 16 1.2 69.2 2.00e-5 - - -
The data collected by the MG data collection mechanism needs to be transferred to the Data Handler. Proper organization of scan records, using time-stamp information if necessary, and storing them is the responsibility of the Data Handler and its servants - recorders. Ideally, different configuration of the Sardana systems use different implementations of the communication channel. Let's imagine a Sardana system composed from one Pool and one MS running in different servers distributed over the network. Another example would be a Pool and a MS running in the same Sardana server. A proper Inter-process Communication method must be used for various possible setups.
This SEP will deal only with the linear motion. Any combination of Sardana motors and pseudomotors could be used as a scan moveable. The following attributes: acceleration time, velocity and deceleration time are configured, so all the motors reach and leave the constant velocity region at the same time.
pre-start position - is calculated for each motor separately:
start position - (velocity * acceleration time) / 2 (scanning in positive direction)
start position + (velocity * acceleration time) / 2 (scanning in negative direction)
acceleration time - is common to all the motors and is determine by the slower accelerating motor involved in the scan. If motors have the acceleration time limits configured, the limit value is used for the comparison, otherwise, the current value is used.
velocity - is calculated for each motor separately from the following parameters: the scan range = abs(end position - start position) and the scan time. The scan time calculation depends on the selected synchronization mode. If pure software synchronization is used scan time = nr of intervals * integration time. In case of the hardware or mixed synchronization the re-arming time must be taken into account, and it is determined by the channel which re-arms the longest. So the scan time = nr of intervals * (integration time + re-arming time). If the result of calculation is lower or higher than the velocity limit, this one is used, properly notifying user about the incident.
deceleration time - is common to all the motors and is determine by the slower accelerating motor involved in the scan. If motors have the deceleration time limits configured, the limit value is used for the comparison, otherwise, the current value is used.
post-end position - is calculated for each motor separately:
end position + (velocity * deceleration time) / 2 + (velocity * integration time) (scanning in positive direction)
end position - (velocity * deceleration time) / 2 - (velocity * integration time) (scanning in negative direction
TODO: if the transparency between step and continuous scan in terms of the number of scan points wants to be achieved, to the post-end position should be added the distance necessary for the end position acquisition (velocity * integration time)
Some scans require execution of multiple sub-scans e.g. mesh. In this case a sequence of sub-scans will be executed in a loop, substituting the "Move to end position" action with a "Move to pre-start position" (of the next sub-scan).
More about the motion control could be found in 1.
Let's assume that we have to control the following devices (their characteristics are invented for the purpose of the example):
The acceleration and deceleration of mot01 and mot02 are 0.3 s and 0.4 s respectively.
Summary of the timing characteristics:
Device | required trigger | re-arming | generated trigger |
---|---|---|---|
ADC | > 5 ms | 3 ms | - |
MCA | > 8 ms | 5 ms | - |
C/T | > 25 ns | 50 ns | > 25 ns |
Motor | > 50 us | 50 us | > 50 us |
Multi | > 10 us | 50 us | > 10 us |
Electrometer | - | - | - |
A Sardana system comprises the following controllers:
Door_1 [3]: lsctrl
Name Type Class
-------------- ----------------- -----------------------------
adc_ctrl CTExpChannel AdlinkAICoTiCtrl
tg_ctrl TriggerGate ZebraBoxController
mot_ctrl Motor IcepapController
mca_ctrl OneDExpChannel XiaOneDCtrl
ct_ctrl CTExpChannel Ni660XCoTiCtrl
emet_ctrl CTExpChannel AlbaEmCoTiCtrl
Following motors are available:
Door_1 [4]: lsm
Name Type Controller Axis
------------ ------------- -------------- ------
mot_01 Motor mot_ctrl 1
mot_02 Motor mot_ctrl 2
Following experimental channels are available:
Door_1 [5]: lsexp
Name Type Controller Axis
---------- -------------- -------------- ------
adc_timer CTExpChannel adc_ctrl 1
adc_00 CTExpChannel adc_ctrl 2
adc_01 CTExpChannel adc_ctrl 3
ct_timer CTExpChannel ct_ctrl 1
ct_00 CTExpChannel ct_ctrl 2
ct_01 CTExpChannel ct_ctrl 3
ct_02 CTExpChannel ct_ctrl 4
mca_timer OneDExpChannel mca_ctrl 1
mca_00 OneDExpChannel mca_ctrl 2
tg_01_pos TriggerGate tg_ctrl 1
emet_timer CTExpChannel emet_ctrl 1
emet_00 CTExpChannel emet_ctrl 2
mot_01_pos CTExpChannel mot_ctrl 1
Following trigger/gates are available:
Door_1 [4]: lstg
Name Type Controller Axis
------------ ------------- -------------- ------
tg_01 TriggerGate tg_ctrl 1
tg_02 TriggerGate tg_ctrl 2
ct_00 CTExpChannel ct_ctrl 2
ct_01_tg CTExpChannel ct_ctrl 3
ct_02_tg CTExpChannel ct_ctrl 4
mot_01_tg Motor motor_ctrl 1
The measurement group configuration:
ExpChannel | Timer | Synchronization | TGChannel |
---|---|---|---|
adc_timer | adc_timer | trigger | mot_01_tg |
adc_00 | adc_timer | trigger | mot_01_tg |
adc_01 | adc_timer | trigger | mot_01_tg |
ct_timer | ct_timer | trigger | mot_01_tg |
ct_00 | ct_timer | trigger | mot_01_tg |
mca_timer | mca_timer | trigger | mot_01_tg |
mca_00 | mca_timer | trigger | mot_01_tg |
emet_timer | emet_timer | software | - |
emet_00 | emet_timer | software | - |
tempertaure | - | - | - |
pressure | - | - | - |
The execution parameters:
Door_1 [4]: ascan mot01 0 100 10 0.1
The measurement group configuration:
ExpChannel | Timer | Synchronization | TGChannel |
---|---|---|---|
adc_timer | adc_timer | trigger | mot_01_tg |
adc_00 | adc_timer | trigger | mot_01_tg |
adc_01 | adc_timer | trigger | mot_01_tg |
ct_timer | ct_timer | gate | mot_01_tg |
ct_00 | ct_timer | gate | mot_01_tg |
emet_timer | emet_timer | software | - |
emet_00 | emet_timer | software | - |
tempertaure | - | - | - |
The execution parameters:
Door_1 [4]: a2scan mot01 0 100 mot02 -5 5 10 0.2
The measurement group configuration:
ExpChannel | Timer | Synchronization | TGChannel |
---|---|---|---|
ct_timer | ct_timer | trigger | tg_01 |
ct_ch00 | ct_timer | trigger | tg_01 |
mca_timer | mca_timer | gate | tg_02 |
mca_00 | mca_timer | gate | tg_02 |
emet_timer | emet_timer | software | - |
emet_00 | emet_timer | software | - |
tempertaure | - | - | - |
The execution parameters:
Door_1 [4]: ascan mot02 0 100 10 0.2
How to deal with the experimental channels coming from many Pools?