Re: gnuplot slow on BIG files

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Br=F6ker wrote:
>> Neither should it be a memory problem (something that would really=20
>> slow down Dimitris' machine perahps).  The block of memory it consumes=
=20
>> will not be much bigger than is eventually stored in memory.
>=20
>=20
> For the case of plot "datafile" matrix every 500:500, it might...

OK.  Point taken.  That could probably be fixed by doing the decimation f=
irst=20
and then changing the "every" internally to 1.

>=20
>> But, what could be a problem--in light of the discussion about FORTRAN=
=20
>> doubles--is that instead of "floats" maybe this routine should be=20
>> storing doubles. =20
>=20
>=20
> Yes, it should.  More to the point, it should store in type coordval.
> That's what it's defined for.

OK.  Who want's to change that?  Or should we hold off until everything e=
lse is=20
worked out?

>=20
>  > Certainly a plotting program doesn't need the
>=20
>> resolution of doubles.=20
>=20
>=20
> Oh, but it does --- if only because some people want to be able to zoom=
=20
> in somewhat deeply.  1/DBL_FLT is less than the number of pixels on the=
=20
> screen, and only a zoom factor of 1000 away from introducing aliasing=20
> artefacts.

You're right.  I forgot about zooming.

>=20
>> Unless one writes a more sophisticated parser.  Given how much testing=
=20
>> is already done here, that would almost be as good an option.  (But=20
>> perhaps not preferable.)
>=20
>=20
> A parser wouldn't do --- we'ld need a dynamic parser *generator*, i.e.=20
> an engine that, given a set of 'using', 'every', 'index' and whatnot,=20
> will generate a parser on-the-fly and run that on the code.
[...]
> Not really.  This is a test for "the first <n> using specifiers are jus=
t=20
> 1..<n>, with (<n> <=3D 5)".  I don't think there's much of a way of=20
> expressing that any more efficiently.  It might make sense to move this
> test to a different place, though.

I think you are saying the same thing here, and sort of what I was gettin=
g at.=20
Couldn't these tests be done before looping through the code to figure ou=
t <n>=20
and then test the loop counter against <n>.  Not sure...



>=20
>> NEXT ISSUE:
>=20
> [...]
>=20
>>         /* HBB 20001221: avoid breaking parsing of time/date
>>          * strings like 01Dec2000 that would be caused by
>>          * overwriting the 'D' with an 'e'... */
>=20
>=20
>> Isn't this a bug? =20
[...]
> No.  This would kill both the scanf() format string allowed to be passe=
d=20
> in, if that contained some random 'q' characters, and, perhaps more=20
> importantly, Ethan's datastrings stuff.

Oy.  Guess that is so.

Dan