gnuplot-py-users

[Gnuplot-py-users] Gnuplot.py float precision (again)

[Xaver W. <xav...@we...>]

Hi guys,

I'm sorry to bring this up again, but I'm about to publish my code and I'd 
really love to get this straight/fixed. In case you don't remember, I'm 
trying to plot high precision numbers (float64, that is), but gnuplot.py 
converts them to float32 arrays. 
Michael Haggerty told me to 'workaround' by using numpy double arrays, but it 
seems that's not working here. I think the problem might be here: 
(/usr/share/pyshared/Gnuplot/utils.py, line 20ff)

>def float_array(m):
>    """Return the argument as a numpy array of type at least 'Float32'.
>
>    Leave 'Float64' unchanged, but upcast all other types to
>    'Float32'.  Allow also for the possibility that the argument is a
>    python native type that can be converted to a numpy array using
>    'numpy.asarray()', but in that case don't worry about
>    downcasting to single-precision float.
>
>    """
>
>    try:
>        # Try Float32 (this will refuse to downcast)
>        return numpy.asarray(m, numpy.float32)
>    except TypeError:
>        # That failure might have been because the input array was
>        # of a wider data type than float32; try to convert to the
>        # largest floating-point type available:
>        # NOTE TBD: I'm not sure float_ is the best data-type for this...
>        try:
>            return numpy.asarray(m, numpy.float_)
>        except TypeError:
>            # TBD: Need better handling of this error!
>            print "Fatal: array dimensions not equal!"
>            return None

If I understand this correctly, the line
> return numpy.asarray(m, numpy.float32)
is supposed to raise a TypeError if you give a numpy.float64 array.
However, my python shell doesn't:

>(00:13:52)xaver@siduxbox:~$python
>Python 2.5.4 (r254:67916, Feb 18 2009, 03:00:47)
>[GCC 4.3.3] on linux2
>Type "help", "copyright", "credits" or "license" for more information.
>>>> import numpy
>>>> a = numpy.array( [2.000000001,3.0,4.0] )
>>>> a.dtype
>dtype('float64')
>>>> numpy.asarray(a, numpy.float32)
>array([ 2.,  3.,  4.], dtype=float32)

Am I doing sth wrong? Can anyone confirm that?

Thanks in advance,
Xaver

Re: [Gnuplot-py-users] Gnuplot.py float precision (again)

[Michael H. <mh...@al...>]

Xaver Wurzenberger wrote:
> Hi guys,
> 
> I'm sorry to bring this up again, but I'm about to publish my code and I'd 
> really love to get this straight/fixed. In case you don't remember, I'm 
> trying to plot high precision numbers (float64, that is), but gnuplot.py 
> converts them to float32 arrays. 
> Michael Haggerty told me to 'workaround' by using numpy double arrays, but it 
> seems that's not working here. I think the problem might be here: 
> (/usr/share/pyshared/Gnuplot/utils.py, line 20ff)
> 
>> def float_array(m):
>>    """Return the argument as a numpy array of type at least 'Float32'.
>>
>>    Leave 'Float64' unchanged, but upcast all other types to
>>    'Float32'.  Allow also for the possibility that the argument is a
>>    python native type that can be converted to a numpy array using
>>    'numpy.asarray()', but in that case don't worry about
>>    downcasting to single-precision float.
>>
>>    """
>>
>>    try:
>>        # Try Float32 (this will refuse to downcast)
>>        return numpy.asarray(m, numpy.float32)
>>    except TypeError:
>>        # That failure might have been because the input array was
>>        # of a wider data type than float32; try to convert to the
>>        # largest floating-point type available:
>>        # NOTE TBD: I'm not sure float_ is the best data-type for this...
>>        try:
>>            return numpy.asarray(m, numpy.float_)
>>        except TypeError:
>>            # TBD: Need better handling of this error!
>>            print "Fatal: array dimensions not equal!"
>>            return None
> 
> If I understand this correctly, the line
>> return numpy.asarray(m, numpy.float32)
> is supposed to raise a TypeError if you give a numpy.float64 array.
> However, my python shell doesn't:
> 
>> (00:13:52)xaver@siduxbox:~$python
>> Python 2.5.4 (r254:67916, Feb 18 2009, 03:00:47)
>> [GCC 4.3.3] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import numpy
>>>>> a = numpy.array( [2.000000001,3.0,4.0] )
>>>>> a.dtype
>> dtype('float64')
>>>>> numpy.asarray(a, numpy.float32)
>> array([ 2.,  3.,  4.], dtype=float32)
> 
> Am I doing sth wrong? Can anyone confirm that?

I don't see that you are doing anything wrong.  That array-conversion
code was written long ago, originally for Numeric (not numpy).
Apparently numpy behaves differently than Numeric in this situation,
resulting in a Gnuplot.py bug.

I suggest that you try to find the right incantation for Gnuplot.py to
do this conversion without losing precision, and submit a patch to
Gnuplot.py.

Michael

Re: [Gnuplot-py-users] Gnuplot.py float precision (again)

[Benny M. <ben...@gm...>]

I suggest you look up things in numpy doc. If nothing there, use the python
way of determining things:

if m:
   if isinstance(m[0], numpy.float64):
        raise TypeError

Provide a patch to gnuplot when you make it work.

Benny

2009/3/3 Michael Haggerty <mh...@al...>

> Xaver Wurzenberger wrote:
> > Hi guys,
> >
> > I'm sorry to bring this up again, but I'm about to publish my code and
> I'd
> > really love to get this straight/fixed. In case you don't remember, I'm
> > trying to plot high precision numbers (float64, that is), but gnuplot.py
> > converts them to float32 arrays.
> > Michael Haggerty told me to 'workaround' by using numpy double arrays,
> but it
> > seems that's not working here. I think the problem might be here:
> > (/usr/share/pyshared/Gnuplot/utils.py, line 20ff)
> >
> >> def float_array(m):
> >>    """Return the argument as a numpy array of type at least 'Float32'.
> >>
> >>    Leave 'Float64' unchanged, but upcast all other types to
> >>    'Float32'.  Allow also for the possibility that the argument is a
> >>    python native type that can be converted to a numpy array using
> >>    'numpy.asarray()', but in that case don't worry about
> >>    downcasting to single-precision float.
> >>
> >>    """
> >>
> >>    try:
> >>        # Try Float32 (this will refuse to downcast)
> >>        return numpy.asarray(m, numpy.float32)
> >>    except TypeError:
> >>        # That failure might have been because the input array was
> >>        # of a wider data type than float32; try to convert to the
> >>        # largest floating-point type available:
> >>        # NOTE TBD: I'm not sure float_ is the best data-type for this...
> >>        try:
> >>            return numpy.asarray(m, numpy.float_)
> >>        except TypeError:
> >>            # TBD: Need better handling of this error!
> >>            print "Fatal: array dimensions not equal!"
> >>            return None
> >
> > If I understand this correctly, the line
> >> return numpy.asarray(m, numpy.float32)
> > is supposed to raise a TypeError if you give a numpy.float64 array.
> > However, my python shell doesn't:
> >
> >> (00:13:52)xaver@siduxbox:~$python
> >> Python 2.5.4 (r254:67916, Feb 18 2009, 03:00:47)
> >> [GCC 4.3.3] on linux2
> >> Type "help", "copyright", "credits" or "license" for more information.
> >>>>> import numpy
> >>>>> a = numpy.array( [2.000000001,3.0,4.0] )
> >>>>> a.dtype
> >> dtype('float64')
> >>>>> numpy.asarray(a, numpy.float32)
> >> array([ 2.,  3.,  4.], dtype=float32)
> >
> > Am I doing sth wrong? Can anyone confirm that?
>
> I don't see that you are doing anything wrong.  That array-conversion
> code was written long ago, originally for Numeric (not numpy).
> Apparently numpy behaves differently than Numeric in this situation,
> resulting in a Gnuplot.py bug.
>
> I suggest that you try to find the right incantation for Gnuplot.py to
> do this conversion without losing precision, and submit a patch to
> Gnuplot.py.
>
> Michael
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco,
> CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the
> Enterprise
> -Strategies to boost innovation and cut costs with open source
> participation
> -Receive a $600 discount off the registration fee with the source code:
> SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Gnuplot-py-users mailing list
> Gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-py-users
>

Re: [Gnuplot-py-users] Gnuplot.py float precision (again)

[Alan G I. <ai...@am...>]

You are correct: the try clause now succeeds
because downcasting is allowed:

     >>> x = np.random.random((5,))
     >>> x.dtype
     dtype('float64')
     >>> x32 = x.astype(np.float32)
     >>> x32.dtype
     dtype('float32')

How about returning any ndarray untouched?

     if isinstance(m, numpy.ndarray):
         return m
     else:
         return numpy.array(m,dtype=numpy.float32)

I see no need to print an error message:
NumPy will report appropriately.

Alan Isaac

Re: [Gnuplot-py-users] Gnuplot.py float precision (again)

[Xaver W. <xav...@we...>]

Hi,
basically, I agree, we probably don't need to raise an exception. However,

 isinstance(m, numpy.ndarray)

returns 'True' for any valid numpy array, which includes e. g. string arrays, 
and for a non-gnuplot compatible array the error message with your code is

gnuplot> plot "/tmp/tmpndt27x.gnuplot/fifo" notitle
              ^
         line 0: Bad data on line 1

..not very helpful. I'd propose to rather have explicitly permitted array 
types, like

     if m.dtype.name in ('float64','float32','int32',int64'):
         return m
     else:
         return numpy.array(m,dtype=numpy.float32)

whereas the 'else' clause serves mainly for having a defined error message, 
maybe intercepted by a 'try...except'.

This might also eliminate the redundant conversion from float32 to float32 (if 
python/numpy is not intelligent enough to not do that). What about int32 / 
int64 arrays? It would be just fine to not convert them as well, right? Saves 
memory, I guess...

Xaver

P.S.:
Is
diff --context oldfile.py oldfile_new.py >> mypatch.py.patch
the right way to make a patch file for the tracker? I've never done that 
before...


Am Dienstag, 3. März 2009 14:40:51 schrieb Alan G Isaac:
> You are correct: the try clause now succeeds
>
> because downcasting is allowed:
>      >>> x = np.random.random((5,))
>      >>> x.dtype
>
>      dtype('float64')
>
>      >>> x32 = x.astype(np.float32)
>      >>> x32.dtype
>
>      dtype('float32')
>
> How about returning any ndarray untouched?
>
>      if isinstance(m, numpy.ndarray):
>          return m
>      else:
>          return numpy.array(m,dtype=numpy.float32)
>
> I see no need to print an error message:
> NumPy will report appropriately.
>
> Alan Isaac
>
>
>
> ---------------------------------------------------------------------------
>--- Open Source Business Conference (OSBC), March 24-25, 2009, San
> Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing
> the Enterprise -Strategies to boost innovation and cut costs with open
> source participation -Receive a $600 discount off the registration fee with
> the source code: SFAD http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Gnuplot-py-users mailing list
> Gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-py-users