Re: [Iometer-user] Linux - Buffer Cache / Memory Alignment

[Ming Z. <mi...@el...>]

On Tue, 2005-04-19 at 16:01 +0200, Ralf Lichtenstein wrote:
> > > @ming
> > > the numbers i measured are (partial) correct (also see picture
> > attached).
> > > partial because the disk subsystem has 16GB Cache (good to remember=
)...
> > so
> > 16GB? hehe, huge.
>=20
> yep.
>=20
> >=20
> > > using 500GB of pure diskspace for measuring utilizes (parts) of the
> > cache.
> > > sequential IOs (if really sequential - means only 1 worker ;-) may =
pass
> > > through the cache. About 70 mb/s (readonly, sequ., large blocksize,=
 1
> > rank,
> > > no aal) are possible numbers for an ESS800. but i still can't expla=
in
> > the
> > > high numbers especially for 6 outst.IO - maybe it is not really
> > sequential
> > > (and so there is some cache influence)?
> > >=20
> > in fact, i think the result is possible if you system is such a big b=
ox.
> >=20
> > if you have only 1 outstanding io, that means=20
> > iometer invoke 1 request, --> scsi layer --> scsi hba --> cache
> > (probably miss/hit) --> disk media then go all way back. sequentially=
,
> > then iometer get response, then invoke next one.
> >=20
> > so this is a fully sequential one. and during some time, the cache an=
d
> > disk subsystem is idle because iometer and host are processing the
> > requests.
> >=20
> > if you increase the outstanding io to 6, then each offset of each
> > request are still sequential, but there are sent in parallel. so one
> > possible scenario will be your array hands of the response for 3rd
> > request to scsi hba via dma, while it is handling 4th request, and th=
e
> > host cpu is processing the finished 2nd one. so these components work
> > concurrently, get overlapped. then buy this way, you have higher numb=
er.
> > and with request volume increase, one component becomes bottleneck wh=
ich
> > might not only flat the throughput but also reduce a bit.
>=20
> so my understanding of the source code is correct - puhh.
>=20
> > 100MB/s is still not a big number. what is you HBA type, a ultra 160 =
one
> > or 320 one? or FC card? u might told me before but i forgot. :P
>=20
> QLogic QLA2312 FC HBA, 2GBit Full Duplex
so this one at this stage should not be a bottleneck.

>=20
> > and how many disk you used for the test and what kind of spec they ar=
e.
> > and what kind of raid type.=20
> >=20
> > all these components can have impact. :P
>=20
> (7 x 74GB SSA disk drive, 10krpm, RAID5, Stripesize 64k...)
>=20
raid 5, assume 6 disks are data disk and 1 are parity if you have no
spare. so with this stripe size, there is a sweet point that can make
use of all spindles.

> i know - i think ;-)    just told you because of your "scsi disk compar=
ison"
> - ESS800 is mainly used for io intensive/"parallel" applications (volum=
es
> spread over >1 LUN, LVM --> ~15000 IO/s =C3=A0 4KB). so sequential thro=
ughput is
> not soooo high...
>=20
agree. those array is for database intensive workload that is random
instead of sequential workload. so the raid algorithm is tuned for that
direction as well.

thx for the info.


>=20
> thank you :-)
> ralf
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20