Re: gnuplot slow on BIG files

[Robert H. <en...@no...>]

On Wed, 8 Jun 2005, Ethan Merritt wrote:

> If you are willing, could you run one more check?
> This entire section of code, with or without NO_FORTRAN_NUMS, is
> inside a larger block which starts with the comment:
> 
> #ifdef OSK
>             /* apparently %n does not work. This implementation
>              * is just as good as the non-OSK one, but close
>              * to a release (at last) we make it os-9 specific
>              */
>             int count;
>             char *p = strpbrk(s, "dqDQ");
>             if (p != NULL)
>                 *p = 'e';
> 
>             count = sscanf(s, "%lf", &df_column[df_no_cols].datum);
> #else
>    [Previously analysed code is here in the #else]
> 
> The question is whether this comment, which dates back at least to 1999,
> is in fact correct.  Could you please compare the previous benchmarks
> to the case where the code is prefixed by:   #define OSK 1   

This is much much worse. (Taking nearly 5 minutes on my benchmark)

Here's why:

In the standard code path, sscanf is used to get the next float out of the
input. Then, if the *NEXT CHARACTER* is a d, D, q, or Q, that character is
replaced with an "e" and the sscanf is repeated.

In the NO_FORTRANS_NUMS code path, atof is used to get the next
float. This is much faster than sscanf.

In the OSK code path, strpbrk is used to scan *THE ENTIRE INPUT LINE* for
the first occurence of d, D, q, or Q. This is *REPEATED* for every value
on the line. sscanf is used to read the values which is slow.

Rob








This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.