We are looking at SCR data from a task (120 trials) where each trial consists of:
a) 2sec (+/- 30 %) fixation cross
b) presentation of a sentence containing information about a bad situation establishing uncertainty
c) self-paced jitter to move on from this (cut off if participants take more than 7 seconds)
d) 2.5sec (+/-30%) jittered anticipation (visually distinct from c)
e) 0.5sec fixation cross
f) outcome of the situation revealed and displayed for 2sec (+/- 30%)
g) 2sec (+/- 30 %) fixation before the following trial
Initially we have been running GLMs marking events at (b) and (f) and looked at mean values of the reconstructed responses for different conditions. These values are often mostly negative. From previous advice on this forum to look at the mean responses we then used the FIR tool to extract time 32 time bins of 0.5 seconds following events at (b) and (f) across conditions. These graphs start with the peak values in the first time bins, there is no rise to peak. (Using the extract segments and segment means tools to plot did not show a response at all becauses the scale was based on the raw SCR values so the response was too small on the scale.)
There seems to me to be 2 explanations for this (please let me know if there are others and what to try if so):
1) these graphs are showing the underlying sudomotor activity which would begin almost immediately after the stimuli are presented
2) participants are starting to respond before the stimuli are presented, possibly due to the fixation crosses predicting stimuli onset and same structure of each trial. In line with this, running FIR marking the stimuli 1 / 2 secs before they are actually presented shows shapes similar to the SCRFs in your papers.
If (1) is the case so the timing of the model is correct, there must be another explanation for the negative parameter estimates. So far my list of things to try are: adding other aspects of the trials to the model e.g. instructions to take a break, looking more manually for artefacts which remain despite filtering and checking whether extending the periods before and after the last events (currently closely trimmed) makes a difference, is there anything else I should try?
If (2) is the case, what is the most valid way of modelling these events? Is it biasing the model to use FIR (blind to conditions) to detect the timing before the stimuli are presented which gives a response closest to the SCRF timing? One option would be to model the fixation cross onsets (a) and (e) but (e) seems to be too short based on the FIR graphs. Or is it valid to change the parameters of the SCRF to better match our faster responses?
Finally, if DCM is advised for any of these issues I had an problem when trying to run it before - the estimates for the fixed events only became identical for every trial after trial 2 for each participant (the pattern was the same for all participants but the values differed). Any idea of what went wrong here?
Really sorry for such a long post and thank you so much in advance for any advice you are able to give. Please let me know which files / more details you need to answer these questions, I haven't added any now just because there are so many aspects.
Many thanks,
Jo
Last edit: Jo Cutler 2018-07-27
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
you are looking into possibility (2): the response starts before, or within the 1 s after, the event. It is quite possible that subjects respond to the fixation cross - I've seen that before.
You should be able to see this using extract segments. Can you increase the scale on the plots, or plot manually? Happy to assist if something does not work with this feature. You could also extract segments (or run the FIR model) wrt event (1) to disentagle whether participants are responding to the fixation cross, or showing an anticipatory response shortly before event (b). In the first case, modelling events (a) and (b) separately could work.
I don't know what went wrong with your DCM - could you send an example model file?
Dominik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks so much for getting back to me so quickly. I'm experimenting with the FIR and extract segements now so will let you know if I find anything which seems odd. In terms of deciding what to model, is it better to be on the safe side and model everything e.g. (a) and (b) even if participants may be only responding to one event or is this costly for the model? It seems there might be individual differences between my participants in whether they show an anticipatory repsonse or not - should a DCM be able to handle this? If so, is it best to have two events separately for (a) and (b) or just have a single longer flexible events covering both phases?
For the previous DCM which showed odd statistics, I've attached an example model file. The issue seems specific to the fixed responses - models I've run with only flexible responses show unique values for each trial. Please let me know if there is anything else you need.
I think what happens in the DCM is that there is no response at the specified time, but the little wiggles in the data mean that changing the amplitude parameter a bit has a neglible impact on the fit. In this scenario, the estimated parameter can be equal to the prior (which is the amplitude of an average trial).
Regarding your GLM, there should not be a problem to model (a) and (b) as they are sufficiently far apart and there is some jitter in the timing. The problem occurs if participants don't respond to (a) but instead show an anticipatory response to (b) which is difficult to time exactlly. But this is a feature of the experiment, not of any particular analysis method.
I hope this helps
Dominik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for your ongoing support. That makes sense re. the DCM falling back on the prior as the model I sent was based on timings from before we discovered the anticipatory responses.
To work out the best way to model the data, I've run DCMs with different timings to create, for example, a single flexible event across the anticipation and onset of a stimulus vs. two events - one for each of these parts.
Is it valid to then use the trial-by-trial model fit statistics (blind to any experimental manipulation) to select a model? Or should this be done at the overall participant level (I couldn't find fit statistics in the model files for this)? If trial-by-trial fit stats are valid then I just wanted to check that the AIC and BIC values in the model files are measures of evidence for the model so larger values = better fit?
Many thanks,
Jo
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
the problem with fit statistics on the participant level is that they don't tell you how good your inference is. For example, in Bach et al. 2013 you can see that the GLM fit (expressed as model evidence, i.e. accounting for increased model complexity) becomes better with more flexible response functions, but the inference on the underlying psychological process becomes worse. The same was observed in Staib et al. 2015.
In other words, the more complex model is more plausible than the simpler model to explain the data - but its parameters are less closely related to the psychological process. The former is what you are interested in when you build biophysical models. The latter is what you ultimately want if you do psychology or neuroscience. Well, it would be great to do both at the same time but that's a big challenge.
Pragmatically, I would recommed using a contrast that you are not interested in and where you would plausibly expect an effect, and optimising the t-statistic for this contrast across participants. This is what we have done in Tzovara et al. 2018 Plos CB (just coming out).
However, if you want to use participant-level fit, I would recommend just using the residual sum of squares (RSS = sum(y - yhat)^2), y and yhat are contained in the model file. You can also convert this into AIC if you want. The AIC values in the model file refer to individual trials, and thereby to overlapping data segments.
Hope this helps
Dominik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you so much for your previous message and apologies for only just acknowledging it. I've now run multiple models and chosen one based on the RSS (which seems like a valid procedure as the same model came out best for the vast majority of participants and across 4 different tasks with similar trial structures but different content).
I'm now examining the DCM amplitude estimates and have a few questions:
1) Are the units of these amplitudes in the native GSR recording units if the data were not normalised during the DCM?
2) Nearly half of the amplitudes are between 0 and 0.01 which isn't surprising as we wouldn't expect a response to many of the events. However, a few events have amplitudes several orders of magnitude higher e.g. 8 or 11. Is there a biologically plausible limit to the amplitudes which I should limit responses to?
3) I'm thinking about excluding 14 of my 186 participants as non-responders as over half of their average (median to avoid the very high estimates) responses to events are less than 0.01 and none are above 0.02 across the 4 tasks - does this seem reasonable?
4) Even if I apply a threshold above which the data are considered biologially impossible and excluded, the data are very skewed. I know I need to normalise responses at some point (is simple z-scoring for each event, by participant best?) but is there a further transformation to the data which you would recommend before applying mixed models?
Finally, now I have used DCM to identify the best combination of event timings to model, if I use these to construct GLM designs (the ISIs are short and jittered) would you recommend specifying the duration as the full length of the event to best match the DCM?
Apologies for so many questions and thank you so much in advance for any advice you are able to give.
Jo
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1) Yes. Note that if you have a flexible latency model, amplitude of the sudomotor burst does not directly relate to SCR amplitude (only together with burst duration/dispersion).
2) I'd check the dispersion here. Amplitude and dispersion are in some cases not clearly separable. If it is very small on these trials, it may be a good idea to set the minimum higher (e.g. 0.3) in pspm_init, or to constrain it to 0.3 (new option in the repository version of the software).
3) Not sure why you would exclude non-responsers, but then I don't know your hypotheses.
4) If you average within conditions, and if there are sufficiently many trials, the averages will be approximately normally distributed. Otherwise, if you do LMEs, then I'd suggest using generalised LMEs and selecting a link function based on model evidence (AIC) as there are several parallel conventions on data transformation (e.g. sqrt, log) , and none seems independently justified in the literature.
Hope this helps
Dominik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you for your reply. The dispersion of the trials with very high amplitudes were very small. I have changed the minimum to 0.3 and this does reduce (approximately half) the very high amplitudes, although they are still much greater than the average.
As most of my events of interest are flexible, what is the best way to put the amplitude and dispersion together? I can't see a way to extract the peaks of reconstructed responses from a DCM as there is with the GLM - have I missed something? Or is it valid to multiply the amplitude by the dispersion to create a single dependent variable to use in mixed models / put the dispersion into the model when predicting amplitude?
Depending on the answer to this question of the best way to combine the amplitude and dispersion, at what point should I normalise the scores for each participant? E.g. multiply amplitude by dispersion then normalise the products / normalise both dispersion and amplitude separately before entering into the model.
Many thanks again,
Jo
Last edit: Jo Cutler 2018-10-22
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
it is quite possible that the amplitudes are actually much higher on some trials.
If you want to combine SN burst dispersion and amplitude to work out the SCR amplitude, the formula is
SCR(t) = a exp((t-m)^2/2s^2) x SCRF(t)
where t is time, x the convolution operator, a is the SN amplitude, m the onset, s the dispersion, and SCRF the canonical skin conductance response function (found in pspm_bf_scrf_f). So you can either work out the maths or do it numerically.
Best
Dominik
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Dear all
Thank you for the software and previous support.
We are looking at SCR data from a task (120 trials) where each trial consists of:
a) 2sec (+/- 30 %) fixation cross
b) presentation of a sentence containing information about a bad situation establishing uncertainty
c) self-paced jitter to move on from this (cut off if participants take more than 7 seconds)
d) 2.5sec (+/-30%) jittered anticipation (visually distinct from c)
e) 0.5sec fixation cross
f) outcome of the situation revealed and displayed for 2sec (+/- 30%)
g) 2sec (+/- 30 %) fixation before the following trial
Initially we have been running GLMs marking events at (b) and (f) and looked at mean values of the reconstructed responses for different conditions. These values are often mostly negative. From previous advice on this forum to look at the mean responses we then used the FIR tool to extract time 32 time bins of 0.5 seconds following events at (b) and (f) across conditions. These graphs start with the peak values in the first time bins, there is no rise to peak. (Using the extract segments and segment means tools to plot did not show a response at all becauses the scale was based on the raw SCR values so the response was too small on the scale.)
There seems to me to be 2 explanations for this (please let me know if there are others and what to try if so):
1) these graphs are showing the underlying sudomotor activity which would begin almost immediately after the stimuli are presented
2) participants are starting to respond before the stimuli are presented, possibly due to the fixation crosses predicting stimuli onset and same structure of each trial. In line with this, running FIR marking the stimuli 1 / 2 secs before they are actually presented shows shapes similar to the SCRFs in your papers.
If (1) is the case so the timing of the model is correct, there must be another explanation for the negative parameter estimates. So far my list of things to try are: adding other aspects of the trials to the model e.g. instructions to take a break, looking more manually for artefacts which remain despite filtering and checking whether extending the periods before and after the last events (currently closely trimmed) makes a difference, is there anything else I should try?
If (2) is the case, what is the most valid way of modelling these events? Is it biasing the model to use FIR (blind to conditions) to detect the timing before the stimuli are presented which gives a response closest to the SCRF timing? One option would be to model the fixation cross onsets (a) and (e) but (e) seems to be too short based on the FIR graphs. Or is it valid to change the parameters of the SCRF to better match our faster responses?
Finally, if DCM is advised for any of these issues I had an problem when trying to run it before - the estimates for the fixed events only became identical for every trial after trial 2 for each participant (the pattern was the same for all participants but the values differed). Any idea of what went wrong here?
Really sorry for such a long post and thank you so much in advance for any advice you are able to give. Please let me know which files / more details you need to answer these questions, I haven't added any now just because there are so many aspects.
Many thanks,
Jo
Last edit: Jo Cutler 2018-07-27
Hi Jo
you are looking into possibility (2): the response starts before, or within the 1 s after, the event. It is quite possible that subjects respond to the fixation cross - I've seen that before.
You should be able to see this using extract segments. Can you increase the scale on the plots, or plot manually? Happy to assist if something does not work with this feature. You could also extract segments (or run the FIR model) wrt event (1) to disentagle whether participants are responding to the fixation cross, or showing an anticipatory response shortly before event (b). In the first case, modelling events (a) and (b) separately could work.
I don't know what went wrong with your DCM - could you send an example model file?
Dominik
Hi Dominik
Thanks so much for getting back to me so quickly. I'm experimenting with the FIR and extract segements now so will let you know if I find anything which seems odd. In terms of deciding what to model, is it better to be on the safe side and model everything e.g. (a) and (b) even if participants may be only responding to one event or is this costly for the model? It seems there might be individual differences between my participants in whether they show an anticipatory repsonse or not - should a DCM be able to handle this? If so, is it best to have two events separately for (a) and (b) or just have a single longer flexible events covering both phases?
For the previous DCM which showed odd statistics, I've attached an example model file. The issue seems specific to the fixed responses - models I've run with only flexible responses show unique values for each trial. Please let me know if there is anything else you need.
Many thanks again,
Jo
Hi Jo
I think what happens in the DCM is that there is no response at the specified time, but the little wiggles in the data mean that changing the amplitude parameter a bit has a neglible impact on the fit. In this scenario, the estimated parameter can be equal to the prior (which is the amplitude of an average trial).
Regarding your GLM, there should not be a problem to model (a) and (b) as they are sufficiently far apart and there is some jitter in the timing. The problem occurs if participants don't respond to (a) but instead show an anticipatory response to (b) which is difficult to time exactlly. But this is a feature of the experiment, not of any particular analysis method.
I hope this helps
Dominik
Hi Dominik
Thanks for your ongoing support. That makes sense re. the DCM falling back on the prior as the model I sent was based on timings from before we discovered the anticipatory responses.
To work out the best way to model the data, I've run DCMs with different timings to create, for example, a single flexible event across the anticipation and onset of a stimulus vs. two events - one for each of these parts.
Is it valid to then use the trial-by-trial model fit statistics (blind to any experimental manipulation) to select a model? Or should this be done at the overall participant level (I couldn't find fit statistics in the model files for this)? If trial-by-trial fit stats are valid then I just wanted to check that the AIC and BIC values in the model files are measures of evidence for the model so larger values = better fit?
Many thanks,
Jo
Hi Jo
the problem with fit statistics on the participant level is that they don't tell you how good your inference is. For example, in Bach et al. 2013 you can see that the GLM fit (expressed as model evidence, i.e. accounting for increased model complexity) becomes better with more flexible response functions, but the inference on the underlying psychological process becomes worse. The same was observed in Staib et al. 2015.
In other words, the more complex model is more plausible than the simpler model to explain the data - but its parameters are less closely related to the psychological process. The former is what you are interested in when you build biophysical models. The latter is what you ultimately want if you do psychology or neuroscience. Well, it would be great to do both at the same time but that's a big challenge.
Pragmatically, I would recommed using a contrast that you are not interested in and where you would plausibly expect an effect, and optimising the t-statistic for this contrast across participants. This is what we have done in Tzovara et al. 2018 Plos CB (just coming out).
However, if you want to use participant-level fit, I would recommend just using the residual sum of squares (RSS = sum(y - yhat)^2), y and yhat are contained in the model file. You can also convert this into AIC if you want. The AIC values in the model file refer to individual trials, and thereby to overlapping data segments.
Hope this helps
Dominik
Dear Dominik
Thank you so much for your previous message and apologies for only just acknowledging it. I've now run multiple models and chosen one based on the RSS (which seems like a valid procedure as the same model came out best for the vast majority of participants and across 4 different tasks with similar trial structures but different content).
I'm now examining the DCM amplitude estimates and have a few questions:
1) Are the units of these amplitudes in the native GSR recording units if the data were not normalised during the DCM?
2) Nearly half of the amplitudes are between 0 and 0.01 which isn't surprising as we wouldn't expect a response to many of the events. However, a few events have amplitudes several orders of magnitude higher e.g. 8 or 11. Is there a biologically plausible limit to the amplitudes which I should limit responses to?
3) I'm thinking about excluding 14 of my 186 participants as non-responders as over half of their average (median to avoid the very high estimates) responses to events are less than 0.01 and none are above 0.02 across the 4 tasks - does this seem reasonable?
4) Even if I apply a threshold above which the data are considered biologially impossible and excluded, the data are very skewed. I know I need to normalise responses at some point (is simple z-scoring for each event, by participant best?) but is there a further transformation to the data which you would recommend before applying mixed models?
Finally, now I have used DCM to identify the best combination of event timings to model, if I use these to construct GLM designs (the ISIs are short and jittered) would you recommend specifying the duration as the full length of the event to best match the DCM?
Apologies for so many questions and thank you so much in advance for any advice you are able to give.
Jo
Dear Jo
1) Yes. Note that if you have a flexible latency model, amplitude of the sudomotor burst does not directly relate to SCR amplitude (only together with burst duration/dispersion).
2) I'd check the dispersion here. Amplitude and dispersion are in some cases not clearly separable. If it is very small on these trials, it may be a good idea to set the minimum higher (e.g. 0.3) in pspm_init, or to constrain it to 0.3 (new option in the repository version of the software).
3) Not sure why you would exclude non-responsers, but then I don't know your hypotheses.
4) If you average within conditions, and if there are sufficiently many trials, the averages will be approximately normally distributed. Otherwise, if you do LMEs, then I'd suggest using generalised LMEs and selecting a link function based on model evidence (AIC) as there are several parallel conventions on data transformation (e.g. sqrt, log) , and none seems independently justified in the literature.
Hope this helps
Dominik
Dear Dominik
Thank you for your reply. The dispersion of the trials with very high amplitudes were very small. I have changed the minimum to 0.3 and this does reduce (approximately half) the very high amplitudes, although they are still much greater than the average.
As most of my events of interest are flexible, what is the best way to put the amplitude and dispersion together? I can't see a way to extract the peaks of reconstructed responses from a DCM as there is with the GLM - have I missed something? Or is it valid to multiply the amplitude by the dispersion to create a single dependent variable to use in mixed models / put the dispersion into the model when predicting amplitude?
Depending on the answer to this question of the best way to combine the amplitude and dispersion, at what point should I normalise the scores for each participant? E.g. multiply amplitude by dispersion then normalise the products / normalise both dispersion and amplitude separately before entering into the model.
Many thanks again,
Jo
Last edit: Jo Cutler 2018-10-22
Dear Jo
it is quite possible that the amplitudes are actually much higher on some trials.
If you want to combine SN burst dispersion and amplitude to work out the SCR amplitude, the formula is
SCR(t) = a exp((t-m)^2/2s^2) x SCRF(t)
where t is time, x the convolution operator, a is the SN amplitude, m the onset, s the dispersion, and SCRF the canonical skin conductance response function (found in pspm_bf_scrf_f). So you can either work out the maths or do it numerically.
Best
Dominik