Re: Some way to differentiate (1st order derivative)?

[<pl...@pi...>]

On Tue, 23 Oct 2007 18:05:29 +0200, <ln...@dr...> wrote:

> Concerning the spline differentiating question: I am aware that splines
> can cause unwanted oscillations,

Yes one of the problems with splines is that the contraints placed on  
continuity and the fact that it must pass through all data points can lead  
to some fairly unexpected and sometimes extreme excursions from the zone  
where the real data lie.

Sometimes it's just clearly the wrong way to respresent certain data.  
Sometimes it's less obvious but still wrong, this is more dangerous. Since  
the constraints on the continuity of the differencial are *completely  
artificially* imposed as being the very definition of the spline fit , I  
believe it is utterly wrong to place any scientific meaning on the  
derivative of a spline fit.

I would invite you to reflect on what that means for your results and  
maybe have a quick look at the maths behind spline fitting to check what I  
say. The maths is pretty simple actually but it's important to know what  
you are doing in applying such a fit.

Many people apply even least squares fits to completely inappropriate data  
because they do not realise the basic assumptions of the maths behind it,  
that is that y errors are >> x errors. If that's not true you get the  
wrong fit!

> but in the past I was succesfully using
> acsplines with weights carefully chosen to approximate the curve I would
> draw by hand through the experimental data.

Well I certainly hope you're not using that sort of bad science for  
anything more than personal curiosity.
It does not sound like valid , reproducable scientific analysis. :)

gnuplot claims not to do data processing (other than fitting user defined  
functions) although "smoothing" data with splines clearly is just that.

I think you should be looking at applying some recognised data processing  
techniques to your data rather than hand tweeking splines.

I hope my comments have highlighted some of the traps , in particular with  
regards to the derivative that you are interested in.

regards, Peter.