Re: [Firebird-devel] FB 2.1 64bit and memory usage ??

[Fred P. <fps...@ya...>]

On 07/23/2007 12:23 PM, Leyne, Sean wrote:
> Disk sectors have been 512 bytes for the last 20 years and likely for
> many more.  Accordingly, our disk accesses will always be on sector
> boundaries.

Hi,

A few months ago, I learned that the disk drive industry
association, IDEMA, approved their new Long Block Data (LBD)
sector standard. The LBD spec. increases the standard sector
size from 512 bytes to 4096 bytes.

The idea is that the larger sector should allow more efficient
error correction, resulting in increased data densities,
improved reliability (up to 10x reduction in error rates), and
increased performance.

Seagate and Western Digital are supposed to have
"transitional" drives available this year.  These drives will
support either the old 512 byte sector or the new 4096 byte
sector (consisting of 8 contiguous 512 byte chunks). Sector
size will be software selectable (512 or 4096).

Other drive makers don't want to support both standards, so
they may wait until the "transitional" period is near its end
before offering LBD sector drives. The IDEMA said that most
drives will support 4k sectors by 2010 and adoption of the new
standard will be gradual.


My concern: the new sector standard may impact direct I/O
alignment constraints for I/O buffers, transfer sizes and file
offsets, depending on the implementation of direct I/O used.

Sadly, direct I/O isn't part of POSIX (or any other standard
that comes to mind). Some implementations use an O_DIRECT
open() flag to enable direct I/O on a file and some use other
methods (e.g. Solaris' directio() library function). Each
implementation may impose its own set of constraints on
subsequent read/write/lseek parameters. I think I have a few
notes on some of the specifics.  If it will help, I can try to
locate them.

I know various db "vendors" support direct I/O as a db option
on Linux, using the current O_DIRECT implementation, but they
had to be careful in their code to avoid certain O_DIRECT
kernel bugs and race conditions.  For example, it seems mixing
O_DIRECT and non-O_DIRECT reads and writes on the same file
simultaneously may result in stale data reads or writes. The
workaround: don't do that!

I suspect that the FB's direct I/O option on linux is going to
need a lot of testing.  ;-)

Fred P.