Re: [Firebird-devel] FB 2.1 64bit and memory usage ??

[Vlad H. <hv...@us...>]

> >> My concern: the new sector standard may impact direct I/O
> >> alignment constraints for I/O buffers, transfer sizes and file
> >> offsets, depending on the implementation of direct I/O used.
> > 
> >     We not allow database page size less than 4KB since FB 2.1
> > Old databases may still use small pages however
> 
> Hi Vlad,
> 
> Good to know. Currently, Linux 2.6's O_DIRECT requires 512
> byte alignment. So, no problem there!

    Yes
 
> However, some O_DIRECT implementations (e.g. Linux 2.4) want
> buffer alignment to be on a *filesystem* block size boundary
> and, possibly, the read/write transfer size and file offset to
> be a multiple of this value. So, an fs block size might be 4k,
> 8k, or more, depending on how the filesystem was built.  E.g.,
> AIX typically wants 4k alignment, except for JFS+big_file
> filesystems, which want 128k alignment.

    Very sad

> Don't you just love non-standard feature implementations? :)

    :)
 
> IMHO, rather than try to make FB support every OS/filesystem
> combination in the first direct I/O implementation, it seems
> reasonable to start with a popular subset and gain some
> experience with this new tuning feature before expanding
> support. Phased implementations can be a good thing.
> 
> That said, some questions remain: how smart should FB be about
> detecting, reporting and/or recovering from a user's attempt
> to use direct I/O when it isn't supported or encounters a
> read/write error downstream? Should FB silently fallback to
> non-direct I/O and retry the I/O op?  Or something else...?

    Silent fallback to non-direct I/O. If direct IO is not supported it
must be detected when FB tried to switch it on, not when real IO
is performed. 
 
> It might be nice if FB could handle this is a consistent way
> across all platforms.

    Agree

> > ...
> >     posix_fadvise free from this drawback and not restrict us to use 
> > aligned memory buffers, right ?
> 
> Correct. Also, posix_fadvise() might be useful for
> tuning I/O elsewhere in FB (e.g., read-ahead?), but that's
> another topic. :)

    I thinking about it too ;)
...
> >     As for testing - are you taker ? :)
> 
> :) Maybe. That will depend on my schedule at the time. I may
> be able to do some basic qualitative testing on Linux 2.6.

    Tell me when (if) you ready and if you'll need any help (with FB 
sources) from my side.

Regards,
Vlad