Re: [Firebird-devel] RFC: Look-ahead disk space allocation and no useof file system cache

[Leyne, S. <Se...@br...>]

Vlad,=20

> 1. Look-ahead disk space allocation
>=20
>     To avoid problems with page buffers allocated in=20
> in-memory cache but not on disk i propose to grab enough disk=20
> space before any attempt to use pages not allocated on disk.=20
>=20
>     Only one thread (process) can extend file to avoid conflicts.=20
>=20
>     On Windows use SetFileValidData API call on those=20
> platforms where it is present (>=3D WinXP) to avoid very slow=20
> synchronous filling of a new place with zeros.
>=20
>     To make process more efficient i offer to allocate space=20
> for several pages at a time. Low and high number of batch of=20
> allocated pages limited by hard-coded constants (i propose=20
> 128KB and 128MB but this is can be discussed of course).
> Engine take 1/16 part of already allocated space and adjust=20
> this number against low and high limits above (1/16 also can=20
> be discussed).

Rather than this elaborate approach, could we not simply introduce a new
setting "minimum DB growth" which would represent the number of pages
which a database will grow when required?

A value range between 1 (naturally) and 16384 would constrain the value,
with 1 as the default.

There are some installs where file growth/size is critical, a low end
value of 128KB would be problematic.

(Ideally, this growth setting should be defined/stored in the DB)


> If look-ahead allocation failed engine divided=20
> requested number of pages by factor of 2 and repeat process=20
> until success or requested number more than zero. If request=20
> to allocate one page is failed too error is raised (also we=20
> can log it into firebird.log)

I don't know if this is really required, but it sure sounds nice.


> 2. Allow engine to not use file system cache
>=20
>     Currently engine works with database files by using file=20
> system cache (at least on Windows). This is good (and must)=20
> for classic server with its small page cache but can hurt=20
> performance of super server configured to use really large page cache.
> If super server will not use file system cache this allow=20
> more efficient use of memory (if there are not enough memory=20
> to fit both kind of caches) and faster access to the disk.

I don't understand the case where this is required.

If there is not enough memory for the database cache, isn't it the DBA
responsibility to reduce the database page cache settings?

Isn't the larger issue that since the SS cache is per connection, as the
number of connection increases, that the allocation SS cache can
outstrip the free physical RAM (as different from Virtual Memory) and
therefore make the entire server grind (as the OS pages memory to the
disk)?


Sean