> -----Original Message-----
> From: FUJITA Tomonori [mailto:fujita.tomonori@...]
> Sent: Tuesday, January 30, 2007 6:32 PM
> To: Ross S. W. Walker
> Cc: fujita.tomonori@...;
> iscsitarget-devel@...
> Subject: Re: [Iscsitarget-devel] New Block-io Module
>
> From: "Ross S. W. Walker" <rwalker@...>
> Subject: Re: [Iscsitarget-devel] New Block-io Module
> Date: Tue, 30 Jan 2007 12:16:32 -0500
>
> > > > > If we set up all bios first and then submit them, the
> > > logic would be
> > > > > more simple?
> > > >
> > > > It would be more complex to setup all bios first and would
> > > add latency.
> > > > To do so we would need an array of bio pointers up to the
> > > request size.
> > >
> > > I'm not sure about this.
> > >
> > > In most cases, you have only one bio with a decent HBA.
> >
> > Not true, BIO_MAX_PAGES = 32 which = 128K,
>
> AFAIK, BIO_MAX_PAGES is 256.
Ok, my bad I just looked at the headers again and you are correct it is
256. I don't know why I thought it was 32, something else I was looking
up at the time probably. So bio request side primarily limited by device
capability.
> > you can of course have
> > request sizes > then 128K from initiator, most high end HBA's limit
> > their max_sector size to 128K, some are adjustable, but
> Linux limits the
>
> Decent HBAs has no such limit.
Then my code against those HBAs should provide substantial performance
gains for large request sizes as it will be able to get the whole
request up to 1MB request size into the bio.
> > bio size anyways. The current implementation will work both
> ways though.
> > If Linux increases BIO_MAX_PAGES to 64 and HBA's
> max_sectors are upped
> > to match, our code will just start sending 256K sized bios,
> we can still
> > get 1M, 2M, 4M... request sizes from initiator though so we
> will still
> > need multiple bios.
> >
> > > > We would need additional counters for that array and additional
> > > > logic,
> > >
> > > The bio structure has something useful for this purpose. See
> > > blk_rq_map_user() post 2.6.19.
> >
> > Not really needed, tio already has data in page vectors,
> this just takes
> > an unstructured buffer and creates page vector based bio
> using zero-copy
> > methodology.
> >
> > The current implementation has gone through several
> variations on this
> > and the current version has been tested as the quickest
> implementation
> > so far.
>
> I don't buy it until I see the performance comparison.
I can provide a spreadsheet of this between file-io, old block-io and
new block-io. The old code was posted on the list not too long ago. The
old code performed very poorly with drbd, it submitted and waited on
each bio, and crashed with MD, it's use of max_sectors to determine bio
request size breaks on variably sectored devices.
> > I don't think the logic can get any simpler then, build
> bio, submit to
> > queue, build bio, submit to queue, when request is
> complete, kick the
> > queue.
>
> Not sure.
At this low level... Find me an algorithm or function that can do it
better (and not just abstract the same thing) and I'll use it. Of course
I want it to run the best it can...
> > > > This leads into another issue I am experiencing. The
> > > majority of Linux
> > > > Enterprise users are running a stable distribution
> based on earlier
> > > > kernels, 2.6.9 the most prevailent. These are going to
> be the target
> > > > users of IET as most home users will not have a need for
> > > iSCSI, yet we
> > > > do not maintain our code for these users.
> > >
> > > In the first place, enterprise users don't install out-of-mainline
> > > kernel modules, I think. iSCSI target feature should be
> provided by
> > > default (like Solaris).
> >
> > Only if they are not provided upstream and the feature is
> needed would
> > enterprise users (myself) build from scratch.
> >
> > Only the bleeding edge distros carry IET in their repositories right
> > now.
> >
> > > There is no chance to have IET in mainline (for
> reasonable reasons) so
> > > that RHEL, CentOS, feadora, Debian and etc will not include
> > > IET. That's the main reason why I've been working on another iSCSI
> > > implementation, tgt.
> >
> > I don't think I followed the argument in the first
> sentence, but if I
> > did then the second would make sense.
>
> IET will not be merged into upstream. Thus the majority of
> distributions will not include IET.
Does IET have an upstream? I thought Ardis was discontinued?
I know Suse is starting to include it in their distro.
It is a candidate for RHEL5, so eventually it will be carried by a lot.
Just not today.
-Ross
______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.
|