[Lse-tech] Doing raw I/O using multiple blocksizes

[Shailabh N. <na...@us...>]

Attached are patches to do raw I/O using multiple blocksizes.
These patches try to use PAGE_SIZE transfers as far as possible.

The idea is to perform I/O for a request of size <size>, from offset
<offset>
of a file in three phases :

          <initial> <pagealigned> <final>
       <---------- <size> ----------->

where <initial> and <final> are done at the default granularity
and <pagealigned> is done at PAGE_SIZE granularity. The default granularity
depends on the path used for raw I/O. If /dev/raw is used, it is generally
set
to hardsector_size (unmounted filesystems). For the O_DIRECT path, it used
to
be 1024(=BUFFERED_BLOCK_SIZE) though I now see 4K size being used in
2.4.13-pre2aa1 (have to investigate further but doesn't affect the point
being
made as a < PAGE_SIZE blocksize is used for mounted file ).

The main benefit of doing the breakup is to allow a major part of the
request
to be carried out at a large granularity and reduce the number of
invocations
to submit_bh() and other lower layer functions.

The patches attached achieve the breakup and work for most requests.
odirvar-2.4.13pre2aa1.patch has the changes for the O_DIRECT path and
rawvary-1-2.4.10.patch has them for the /dev/raw/ path. The latter also
applies
cleanly on 2.4.13-pre2aa1 (kernel version chosen on account of O_DIRECT
being
dropped from post-2.4.10 kernels).

The patches will always work if the major user parameters : <buf>, <offset>
and
<size> are all page-aligned. They will also work if <offset> and <buf> are
misaligned by the same margin e.g. have the same value modulo PAGE_SIZE.

If <buf> is pagealigned but <offset> is not, the patches cause an
incomplete
read. This is what happens :
I/O for <initial>  is done at default granularity and completes
successfully.
While attempting to perform the I/O for <pagealigned>, the portion of <buf>
that needs to be mapped into the kiobuf is NO LONGER aligned with
the new blocksize (i.e. PAGE_SIZE) and hence brw_kiovec() fails in its
alignment checks.

The crux of the matter lies in what alignment conditions are to be imposed
on
the <size>,<offset> and <buf> parameters that come from the application's
I/O
request. If <offset> is not pagealigned, there will be an <initial> portion
of
the I/O that needs to be performed at default granularity. Once that is
done,
<buf> is out of alignment with pagesize.

The solution seems to be impose page alignment restrictions on <offset> so
that
<initial> is always non-existent.

Any other ways to get around this somewhat unsatisfactory restriction ?
Other comments on the patches ?


(See attached file: rawvary-1-2.4.10.patch)(also applies cleanly on
2.4.13-pre2aa1)

(See attached file: odirvar-2.4.13pre2aa1.patch)


Shailabh Nagar
Enterprise Linux Group, IBM TJ Watson Research Center
(914) 945 2851, T/L 862 2851