From: Piter_ [mailto:x.piter@gmail.com]
Sent: Tuesday, October 27, 2009 14:37

Hi all.
I have a problem with loading file of following format:
first 1024 rows are tab delimited and contain from 2 to 256 elements (in different files different number of columns)
after that 5 empty lines
and at the end  some 20 text lines for description.

Although the following isn't specific to matplotlib, I submit it for the sake of others who may have similar questions about reading text data.

Because a file object may be iterated, one can use the itertools module. In particular, the islice iterator allows you to select the start and stop lines and the step. So, you can read the desired portion of the file into a list of rows, splitting each row into a list of text tokens, then use numpy.array to convert the list into a numeric array. For example,

# Begin code
import numpy as np
import itertools
from __future__ import with_statement  # no longer required in Python 2.6
 
with open('filename.dat') as f:
    a = np.array(
        [line.rstrip().split('\t') for line in itertools.islice(f, 1024)],
        dtype=np.float)
# End code

Just alter the islice arguments and dtype as necessary to suit your file.

Documentation:

·         http://docs.python.org/library/stdtypes.html#file.next

·         http://docs.python.org/library/itertools.html#itertools.islice

·         http://docs.scipy.org/doc/numpy-1.3.x/reference/generated/numpy.array.html

·         http://docs.scipy.org/doc/numpy/reference/arrays.dtypes.html