numpy-discussion

[Numpy-discussion] doctest fortran I/O synchronization

[Todd M. <jm...@st...>]

I'm trying to make a doctest to verify that the different flow patterns
of f2py interfaces work with different varieties of numarrays (normal,
byte-swapped, misaligned, dis-contiguous, type-converted).  I'm trying
to test this out under Linux with g77, and (it seems like) I'm having
trouble synchronizing the Fortran I/O with Python's C I/O.

Given foo.f:

      subroutine in_c(a,m,n)
      real*8 a(n,m)
Cf2py intent(in,c) a
Cf2py depend(a) ::  n=shape(a,0),  m=shape(a,1)
      do j=1,m
         do i=1,n
            write (6,1) a(i,j)
 1          format( $, 1F3.0, ', ')
         enddo
         print *,''
      enddo
      end

And given f2py_tests.py:

"""
>>> foo.in_f(a)
 0.,  5., 10.,
 1.,  6., 11.,
 2.,  7., 12.,
 3.,  8., 13.,
 4.,  9., 14.,
"""
import foo, numarray

def test():
    import doctest
    global a
    t = doctest.Tester(globs=globals())
    a = numarray.arange(15., shape=(3,5))
    t.runstring(__doc__, "c_array")
    return t.summarize()

I get this:

[jmiller@halloween ~/f2py_tests]$ python f2py_tests.py
 0.,  5., 10.,
 1.,  6., 11.,
 2.,  7., 12.,
 3.,  8., 13.,
 4.,  9., 14.,
*****************************************************************
Failure in example: foo.in_f(a)
from line #1 of c_array
Expected:
 0.,  5., 10.,
 1.,  6., 11.,
 2.,  7., 12.,
 3.,  8., 13.,
 4.,  9., 14.,
Got:
*****************************************************************
1 items had failures:
   1 of   1 in c_array
***Test Failed*** 1 failures.

Where it appears that the output from the first example somehow escapes
the C I/O system I presume doctest is using.  The actual test I'm
writing has multiple examples, and the fortran I/O *does* make it into
the doctest after the first example  but remains out of sync.

Does anyone have an explanation and/or fix for this problem?

-- 
Todd Miller 			jm...@st...
STSCI / ESS / SSB

[Numpy-discussion] Re: [SciPy-user] doctest fortran I/O synchronization

[Pearu P. <pe...@sc...>]

Hi Todd,

On 10 Oct 2003, Todd Miller wrote:

> I'm trying to make a doctest to verify that the different flow patterns
> of f2py interfaces work with different varieties of numarrays (normal,
> byte-swapped, misaligned, dis-contiguous, type-converted).  I'm trying
> to test this out under Linux with g77, and (it seems like) I'm having
> trouble synchronizing the Fortran I/O with Python's C I/O.
> 
> Given foo.f:
> 
>       subroutine in_c(a,m,n)
>       real*8 a(n,m)
> Cf2py intent(in,c) a
> Cf2py depend(a) ::  n=shape(a,0),  m=shape(a,1)
>       do j=1,m
>          do i=1,n
>             write (6,1) a(i,j)
>  1          format( $, 1F3.0, ', ')
>          enddo
>          print *,''
>       enddo
>       end
> 
> And given f2py_tests.py:

> """
> >>> foo.in_f(a)
<snip>

I could not run given tests as they were not complete and had typos.
For instance, foo.f defines in_c but in test string you are using in_f.
Also AFAIK, Python I/O will not catch I/O from Fortran.

Any way, when I modify in_c to in_f then the following code

  a = numarray.arange(15., shape=(3,5))
  print a
  foo.in_f(a)

outputs:

[[  0.   1.   2.   3.   4.]
 [  5.   6.   7.   8.   9.]
 [ 10.  11.  12.  13.  14.]]
 0.,  1.,  2.,  
 3.,  4.,  5.,  
 6.,  7.,  8.,  
 9., 10., 11.,  
12., 13., 14.,  

You probaly would not expect this. This is related to different storage 
order in C and Fortran, of cource, and you have disabled f2py ability to 
take into account this by using intent(c).

So, when you would not use intent(c), that is, in foo.f is a line

  Cf2py intent(in) a

then the following output occurs:

[[  0.   1.   2.   3.   4.]
 [  5.   6.   7.   8.   9.]
 [ 10.  11.  12.  13.  14.]]
 0.,  5., 10.,  
 1.,  6., 11.,  
 2.,  7., 12.,  
 3.,  8., 13.,  
 4.,  9., 14.,  

The arrays look transposed because in Fortran your row index varies 
faster, that is, a transposed array is displayed.

To sum up, don't use intent(c) and change the order of loops in in_f 
function to get matching results.

HTH,
  Pearu

Re: [Numpy-discussion] doctest fortran I/O synchronization

[David M. C. <co...@ph...>]

On Fri, Oct 10, 2003 at 10:17:54AM -0400, Todd Miller wrote:
> I'm trying to make a doctest to verify that the different flow patterns
> of f2py interfaces work with different varieties of numarrays (normal,
> byte-swapped, misaligned, dis-contiguous, type-converted).  I'm trying
> to test this out under Linux with g77, and (it seems like) I'm having
> trouble synchronizing the Fortran I/O with Python's C I/O.
> 
> Given foo.f:
> 
>       subroutine in_c(a,m,n)
>       real*8 a(n,m)
> Cf2py intent(in,c) a
> Cf2py depend(a) ::  n=shape(a,0),  m=shape(a,1)
>       do j=1,m
>          do i=1,n
>             write (6,1) a(i,j)
>  1          format( $, 1F3.0, ', ')
>          enddo
>          print *,''
>       enddo
>       end
> 
> And given f2py_tests.py:
> 
> """
> >>> foo.in_f(a)
>  0.,  5., 10.,
>  1.,  6., 11.,
>  2.,  7., 12.,
>  3.,  8., 13.,
>  4.,  9., 14.,
> """
> import foo, numarray
> 
> def test():
>     import doctest
>     global a
>     t = doctest.Tester(globs=globals())
>     a = numarray.arange(15., shape=(3,5))
>     t.runstring(__doc__, "c_array")
>     return t.summarize()
> 
> I get this:
> 
> [jmiller@halloween ~/f2py_tests]$ python f2py_tests.py
>  0.,  5., 10.,
>  1.,  6., 11.,
>  2.,  7., 12.,
>  3.,  8., 13.,
>  4.,  9., 14.,
> *****************************************************************
> Failure in example: foo.in_f(a)
> from line #1 of c_array
> Expected:
>  0.,  5., 10.,
>  1.,  6., 11.,
>  2.,  7., 12.,
>  3.,  8., 13.,
>  4.,  9., 14.,
> Got:
> *****************************************************************
> 1 items had failures:
>    1 of   1 in c_array
> ***Test Failed*** 1 failures.
> 
> Where it appears that the output from the first example somehow escapes
> the C I/O system I presume doctest is using.  The actual test I'm

doctest uses Python's I/O system: it assigns a new object to
sys.stdout. Your code uses Fortran's output, which would go the same
place a printf in C would: to the program's stdout (file descriptor 1).

You'd need to run the code in a separate process, and capture the
output. Something along the lines of this:

import commands
def test_f2py():
    """
    put your doctest here
    """
    output = commands.getoutput('python f2pytest1.py')
    print output

Or, set your test up to write output to a file instead of stdout, then
read that file (that's probably better).

> writing has multiple examples, and the fortran I/O *does* make it into
> the doctest after the first example  but remains out of sync.

It's out of sync because it's not going through Python; Python has
absolutely no clue that the Fortran code wrote anything.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] doctest fortran I/O synchronization

[Todd M. <jm...@st...>]

Thanks for the work around.  I haven't tried it yet but I've got a
feeling I'm home free...  something along these lines will definitely
work.

Regards,
Todd

On Fri, 2003-10-10 at 12:37, David M. Cooke wrote:
> On Fri, Oct 10, 2003 at 10:17:54AM -0400, Todd Miller wrote:
> > I'm trying to make a doctest to verify that the different flow patterns
> > of f2py interfaces work with different varieties of numarrays (normal,
> > byte-swapped, misaligned, dis-contiguous, type-converted).  I'm trying
> > to test this out under Linux with g77, and (it seems like) I'm having
> > trouble synchronizing the Fortran I/O with Python's C I/O.
> > 
> > Given foo.f:
> > 
> >       subroutine in_c(a,m,n)
> >       real*8 a(n,m)
> > Cf2py intent(in,c) a
> > Cf2py depend(a) ::  n=shape(a,0),  m=shape(a,1)
> >       do j=1,m
> >          do i=1,n
> >             write (6,1) a(i,j)
> >  1          format( $, 1F3.0, ', ')
> >          enddo
> >          print *,''
> >       enddo
> >       end
> > 
> > And given f2py_tests.py:
> > 
> > """
> > >>> foo.in_f(a)
> >  0.,  5., 10.,
> >  1.,  6., 11.,
> >  2.,  7., 12.,
> >  3.,  8., 13.,
> >  4.,  9., 14.,
> > """
> > import foo, numarray
> > 
> > def test():
> >     import doctest
> >     global a
> >     t = doctest.Tester(globs=globals())
> >     a = numarray.arange(15., shape=(3,5))
> >     t.runstring(__doc__, "c_array")
> >     return t.summarize()
> > 
> > I get this:
> > 
> > [jmiller@halloween ~/f2py_tests]$ python f2py_tests.py
> >  0.,  5., 10.,
> >  1.,  6., 11.,
> >  2.,  7., 12.,
> >  3.,  8., 13.,
> >  4.,  9., 14.,
> > *****************************************************************
> > Failure in example: foo.in_f(a)
> > from line #1 of c_array
> > Expected:
> >  0.,  5., 10.,
> >  1.,  6., 11.,
> >  2.,  7., 12.,
> >  3.,  8., 13.,
> >  4.,  9., 14.,
> > Got:
> > *****************************************************************
> > 1 items had failures:
> >    1 of   1 in c_array
> > ***Test Failed*** 1 failures.
> > 
> > Where it appears that the output from the first example somehow escapes
> > the C I/O system I presume doctest is using.  The actual test I'm
> 
> doctest uses Python's I/O system: it assigns a new object to
> sys.stdout. Your code uses Fortran's output, which would go the same
> place a printf in C would: to the program's stdout (file descriptor 1).
> 
> You'd need to run the code in a separate process, and capture the
> output. Something along the lines of this:
> 
> import commands
> def test_f2py():
>     """
>     put your doctest here
>     """
>     output = commands.getoutput('python f2pytest1.py')
>     print output
> 
> Or, set your test up to write output to a file instead of stdout, then
> read that file (that's probably better).
> 
> > writing has multiple examples, and the fortran I/O *does* make it into
> > the doctest after the first example  but remains out of sync.
> 
> It's out of sync because it's not going through Python; Python has
> absolutely no clue that the Fortran code wrote anything.
> 
> -- 
> |>|\/|<
> /--------------------------------------------------------------------------\
> |David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
> |co...@ph...
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> SourceForge.net hosts over 70,000 Open Source Projects.
> See the people who have HELPED US provide better services:
> Click here: http://sourceforge.net/supporters.php
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			jm...@st...
STSCI / ESS / SSB