Menu
▾
▴
Re: [Iometer-devel] Questions about outstanding I/O Queue of Linuxvs.TCQ/NCQ.
|
From: Harder, D. W. <Dav...@gd...> - 2006-10-27 21:55:51
|
That's why I specified small-block <random> I/O. Neither the RAID controller nor the OS will be able to coalesce writes/reads that are in physically seperate parts of the disk (i.e. not sequential). Also, I wasn't exactly referring to a RAID controller card with a driver (although still applicable). I was more referring to an external RAID. (Something that has a media-type interface between the host and the RAID as well as between the RAID and the target media. Such a device looks like one or more media devices to the host even though the devices are virtualized by the controller.) Also, the OS (Windows/Linux) will only coalesce I/O if you let it. When you disable buffering (O_DIRECT or equivalent), the OS does not coalesce the I/O. -----Original Message----- From: iom...@li... [mailto:iom...@li...] On Behalf Of Liang Yang Sent: Friday, October 27, 2006 3:54 PM To: Harder, David W.; iom...@li...; iom...@li... Subject: Re: [Iometer-devel] Questions about outstanding I/O Queue of Linuxvs.TCQ/NCQ. Hi Dave, RAID controller card driver under Windows will typically do I/O coalescing for small packets, e.g. all those 512B packets could be merged into a 32K packet. With this coalescing enabled, do you think the outstanding I/O queue depth for small packets will remain the same as that for large packets? Or do you think it is better to have two outstanding I/O queue, one is for small packets and the other is for large packets? Thanks, Liang ----- Original Message ----- From: "Harder, David W." <Dav...@gd...> To: <yan...@ho...>; <iom...@li...>; <iom...@li...> Sent: Friday, October 27, 2006 12:49 PM Subject: RE: [Iometer-devel] Questions about outstanding I/O Queue of Linux vs.TCQ/NCQ. Depends on the RAID, your RAID configuration, and your IOMeter profile. For example, if you have 16 drives in a RAID-0 array and you have a capable RAID controller (that can handle 256+ I/Os), then a small-block random profile in IOMeter would typically result in roughly 16 I/Os to each drive. If you have a large-block sequential profile with the same hardware configuration, the number of I/Os would be somewhere in between 16 and 256 with a tendency towards 256 (depending on the RAID controller). -Dave- -----Original Message----- From: Liang Yang [mailto:yan...@ho...] Sent: Friday, October 27, 2006 2:32 PM To: Harder, David W.; iom...@li...; iom...@li... Subject: Re: [Iometer-devel] Questions about outstanding I/O Queue of Linux vs.TCQ/NCQ. Hi Dave, Thanks for your answer. For the depth of TAG queue of SAS drives, theoretically it could be 65535 as the SAS device can support up to that many of initators. But in practice, 64 is the range most of SAS drives are using. As you mentioned the RAID stuff in your answer, I just have a question for using outstanding I/Os on RAID volumes: Suppose you have a RAID volume which is built on 16 SAS drives. When you treat this RAID volume as single physcial drive and use IOMeter to do some physcial drive performance testing. You set the queue depth in IOMeter to 256. Does each drive behind this RAID volume get 256 I/O, or the RAID controller card will divide 256 I/O between 16 drives and each drive just get 16 I/Os? Best regards, Liang ----- Original Message ----- From: "Harder, David W." <Dav...@gd...> To: <mul...@ho...>; <iom...@li...>; <iom...@li...> Sent: Friday, October 27, 2006 12:04 PM Subject: RE: [Iometer-devel] Questions about outstanding I/O Queue of Linux vs.TCQ/NCQ. 1) You need one queue in order to populate the other. In other words, if your application does not tell the OS to issue multiple asynchronous I/O (or allow the OS to do this implicitly for you), then the block device will never have more than one I/O in it's queue. 2) I might be wrong, but I believe it is possible for block devices to support queue depths much greater than 32 or 64. For example, you could have a block device that is actually a SAS-SAS (or SAS-SATA) RAID 16 drives behind it. (It might even be indistinguishable from a SAS drive as far as the host is concerned.) 3) IMHO, I doubt there are any measurable negative effects of having an application I/O queue that is greater than the block device's I/O queue. (i.e. If you have a block device with a queue size of 64 and your application queues 70 I/Os, the performance will not be any worse than if your application queued 64.) -Dave- -----Original Message----- From: iom...@li... [mailto:iom...@li...] On Behalf Of Liang Yang Sent: Friday, October 27, 2006 1:35 PM To: iom...@li...; iom...@li... Subject: [Iometer-devel] Questions about outstanding I/O Queue of Linux vs.TCQ/NCQ. Hi, Under Linux, we can issue different number of outstanding I/O to the block device, e.g. the queue depth can be set from 1 to 256 from IOMeter. However, the hard disk drive itself may have its own queue (e.g. NCQ--Native tagged command queue for SATA drives and TCQ--tagged command queue for SAS/SCSI drives). A typical queue depth for NCQ is 32 and TCQ is 64. The outstanding I/O queue should be controlled by the block device driver and RAID controller card/SAS HBA firmware. The NCQ/TCQ should be controlled by the hard disk drive firmware. I have two questions here: If outstanding I/O queue has different queue depth from NCQ/TCQ, will this affect the I/O performance, e.g. the Linux outstanding I/O queue is much longer than the TCQ/NCQ in the hard disk drive? Why do we need to maintain two queues here? Does the duplicate queue cause additional overhead? Could anyone give me some explanation about this? Thanks, Liang ------------------------------------------------------------------------ - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D= 121642 _______________________________________________ Iometer-devel mailing list Iom...@li... https://lists.sourceforge.net/lists/listinfo/iometer-devel ------------------------------------------------------------------------ - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D= 121642 _______________________________________________ Iometer-devel mailing list Iom...@li... https://lists.sourceforge.net/lists/listinfo/iometer-devel |