gnuplot-beta

Does "fit" really need to use the axis structures?

[Ethan A M. <sf...@us...>]

In response to a request on the newsgroup, I've been looking into relaxing
the limit of 5 independent parameters in "fit".

So far as I can see there are 2 reasons for this limit.

1)  df_readline() limits the number of fields in a 'using' clause to
    MAXDATACOLS, and this is currently defined as 7.

2)  the fit parameters can be range-limited in the "fit" command,
    and these ranges are stored in the axis data structures for the
    x, y, t, u, and v axes.

(1) It is easy to change MAXDATACOLS, and I have confirmed that by
itself this doesn't break anything.

(2) is harder to relax if it really is necessary to use the axis data
structures.  But is it really necessary?

It looks to me that the only reason the axis data structures are involved
is that routine parse_range(), which used to be macro PARSE_NAMED_RANGE,
takes an axis index as input parameter and overwrites the 
autoscale/min/max fields in that axis structure.
During the fit operation, these ranges are checked in the axis structure.

Is there some other reason I am missing that requires access to the
axis data structures?  Can we just provide a local array containing
autoscale/min/max for each fit parameter and not access the axis
structures at all?

	Ethan

Re: Does "fit" really need to use the axis structures?

[Bastian M. <bma...@we...>]

Am 12.06.2013 23:29, schrieb Ethan A Merritt:
> In response to a request on the newsgroup, I've been looking into relaxing
> the limit of 5 independent parameters in "fit".
>
> So far as I can see there are 2 reasons for this limit.
>
> 1)  df_readline() limits the number of fields in a 'using' clause to
>      MAXDATACOLS, and this is currently defined as 7.
>
> 2)  the fit parameters can be range-limited in the "fit" command,
>      and these ranges are stored in the axis data structures for the
>      x, y, t, u, and v axes.
>
> (1) It is easy to change MAXDATACOLS, and I have confirmed that by
> itself this doesn't break anything.
>
> (2) is harder to relax if it really is necessary to use the axis data
> structures.  But is it really necessary?
>
> It looks to me that the only reason the axis data structures are involved
> is that routine parse_range(), which used to be macro PARSE_NAMED_RANGE,
> takes an axis index as input parameter and overwrites the
> autoscale/min/max fields in that axis structure.
> During the fit operation, these ranges are checked in the axis structure.
>
> Is there some other reason I am missing that requires access to the
> axis data structures?  Can we just provide a local array containing
> autoscale/min/max for each fit parameter and not access the axis
> structures at all?
>
> 	Ethan

I am not familiar enough with the axis structures to comment on this. 
The fitting code itself certainly does not have a restriction to the 
number of indep. variables.  Hans-Bernhard, any comments?

Support for up to five independent variables was implemented by Jim Van 
Zandt.  Interestingly, the discussion about the axis array was brought 
up already in the original discussion:
http://thread.gmane.org/gmane.comp.graphics.gnuplot.devel/8319

There was also was a proposal by SF user Hanno, where he discusses a 
patch he was working on with support for more than 2 indep. variables. 
Sadly, no code is attached.
http://sourceforge.net/p/gnuplot/feature-requests/191/

   Bastian

Re: Does "fit" really need to use the axis structures?

[Hans-Bernhard B. <HBB...@t-...>]

On 14.06.2013 08:46, Bastian Märkisch wrote:
> Am 12.06.2013 23:29, schrieb Ethan A Merritt:

>> It looks to me that the only reason the axis data structures are involved
>> is that routine parse_range(), which used to be macro PARSE_NAMED_RANGE,
>> takes an axis index as input parameter and overwrites the
>> autoscale/min/max fields in that axis structure.

In a nutshell: data file reading refers to axis data structures (--> 
global df_axis[]), e.g. for time/date data handling and pseudo files '+' 
and '++'.

Fit has to read data.  So fit has to set up at least some semblance of 
axis data structures.  And currently, those have to be in the main 
axis_array[] because that's the only one handled by interfaces using an 
AXIS_INDEX argument (including df_axis[]).  But that axis_array[] can't 
be extended all that easily, if only because its entries are presented 
to the user by way of names ("x1", "z", "cb", ...), not numbers.

> The fitting code itself certainly does not have a restriction to the
> number of indep. variables.  Hans-Bernhard, any comments?

I wouldn't know.  It wasn't me who lifted the original restriction of 2 
independent variables, which used to be the same as with all other 
commands in gnuplot: 2.  I didn't study James Van Zandt's changes in detail.

The restriction is ultimately in how ranges are handled in gnuplot. 
Ever since the big "axis array" change in year 2000, a range has been a 
internal property of an axis, and thus completely handled by code in 
axis.c/axis.h.  This worked well as long as fit tried to model its 
behaviour after (s)plot, including its number of independent variables. 
Variable ranges for 'fit', e.g., are parsed by axis.c::parse_range().

But if we want 'fit' to support more independent variables than any 
other command in gnuplot (which it already does), or even support an 
unlimited number of them, that whole concept of

	one indepentent variable --> one range <--> one axis

will have to be given up and replaced by something else.  And that 
change would affect not just 'fit', but also 'plot', 'splot' and a good 
deal of other commands.  Ranges would get a data structure of their own, 
and axis would point to (or contain) such a range struct.

Re: Does "fit" really need to use the axis structures?

[sfeam (E. Merritt) <eam...@gm...>]

On Thursday, 13 June 2013, Bastian Märkisch wrote:
> Am 12.06.2013 23:29, schrieb Ethan A Merritt:
> > In response to a request on the newsgroup, I've been looking into relaxing
> > the limit of 5 independent parameters in "fit".
> >
> > So far as I can see there are 2 reasons for this limit.
> >
> > 1)  df_readline() limits the number of fields in a 'using' clause to
> >      MAXDATACOLS, and this is currently defined as 7.
> >
> > 2)  the fit parameters can be range-limited in the "fit" command,
> >      and these ranges are stored in the axis data structures for the
> >      x, y, t, u, and v axes.
> >
> > (1) It is easy to change MAXDATACOLS, and I have confirmed that by
> > itself this doesn't break anything.
> >
> > (2) is harder to relax if it really is necessary to use the axis data
> > structures.  But is it really necessary?
> >
> > It looks to me that the only reason the axis data structures are involved
> > is that routine parse_range(), which used to be macro PARSE_NAMED_RANGE,
> > takes an axis index as input parameter and overwrites the
> > autoscale/min/max fields in that axis structure.
> > During the fit operation, these ranges are checked in the axis structure.
> >
> > Is there some other reason I am missing that requires access to the
> > axis data structures?  Can we just provide a local array containing
> > autoscale/min/max for each fit parameter and not access the axis
> > structures at all?
> >
> > 	Ethan
> 
> I am not familiar enough with the axis structures to comment on this. 
> The fitting code itself certainly does not have a restriction to the 
> number of indep. variables.  Hans-Bernhard, any comments?
> 
> Support for up to five independent variables was implemented by Jim Van 
> Zandt.  Interestingly, the discussion about the axis array was brought 
> up already in the original discussion:
> http://thread.gmane.org/gmane.comp.graphics.gnuplot.devel/8319
> 
> There was also was a proposal by SF user Hanno, where he discusses a 
> patch he was working on with support for more than 2 indep. variables. 
> Sadly, no code is attached.
> http://sourceforge.net/p/gnuplot/feature-requests/191/
> 
>    Bastian

Thanks for the feedback on understanding the current fitting code.
As I said, this is the first time I've looked at it.   

I have put a patch on SourceForge increases the number of independent
variables from 5 to 12.   In principle it could be increased much more than
that, and indeed I tested setting it as high as 99.
 
   <https://sourceforge.net/p/gnuplot/patches/625/>

First it decouples the fit variables (other than x and y) from the normal
axis structures.  That removes one cap on the maximum number of
independent variables.  The only change to current behaviour
(IMHO a benefit) is that setting a range on t/u/v does not affect
subsequent fit commands.

Having removed that limiting cap, the next one hit is MAXDATACOLS.
I've audited the code and am 99% convinced that it is OK to increase
this as needed.  The patch sets it to 14 (==MAX_VAR_NUM+2)

Finally, "fit" requires giving names ("dummy names") to the independent
variables.  The number of dummy variables is capped by MAX_VAR_NUM,
which is currently 12.    I raised that to 99 for testing, but left it at 12
in the patch.  To make it easy to set these, I've changed the code in CVS
so that "fit" will use the names (if any) previously requested by a
"set dummy x1,x2,x3,x4,...." command.  This already worked in 4.6/4.7
but only for the first two dummies (default "x" and "y").  Now it works
for as many as you like.

Hanno Hofstadt's test script was very useful in testing, so I have added
it to the CVS demo collection for both 4.6 and 4.7. 

Bastian:  I didn't try to reconcile this patch set with your patch #621 
that adds per-dimension error terms.   There may or may not be any
conflict.  At any rate this one makes it easy to read in larger numbers of
data columns, which will help if there are two input columns per dimension
needed.

	Ethan