Re: [Scst-devel] Direct install on iscsi target

[Vladislav B. <vs...@vl...>]

Vladislav Bolkhovitin, on 12/08/2009 11:19 PM wrote:
> Benjamin Yates, on 12/08/2009 09:05 PM wrote:
>> Vladislav Bolkhovitin wrote:
>>> P.S. One more piece of IET code is going to be rewritten..
>>     fwiw, iet works...
> 
> IET doesn't support MaxOutstandingR2T > 1.

Seems I should evaluate a bit more on this to remove very much possible 
misunderstandings.

Let's consider we have a 1Gbps network between initiator and target with 
10ms latency (a good near distance WAN/Internet to another building in 
the same town). We want to send backup from the initiator to a tape or 
tape library on the target. Another possible task is a sync. 
replication. We are limited to send only 1 write command at time, 
because our tape doesn't allow more. We will send 2MB of data in each 
command.

Our initiator and target negotiated typical values InitialR2T No, 
ImmediateData Yes, FirstBurstLength 65536, MaxBurstLength 262144. Other 
parameters don't matter for our task, except MaxOutstandingR2T. Let we 
can run 2 pieces of iSCSI target software on the target: one supporting 
only MaxOutstandingR2T 1 (e.g., IET) and another one supporting 
MaxOutstandingR2T >1 (e.g., iSCSI-SCST). The first target negotiated 
MaxOutstandingR2T 1, the second one - 16.

10ms means that a 1 byte packet send from the initiator reaches the 
target in 5ms time. Then 5ms in the opposite direction. 1Gbps bandwidth 
means that 64K of data transferred from the initiator to the target in 
about 0.5ms

Since each R2T request must contain less than MaxBurstLength data, we 
need ((2MB - 64K)/256K) = 8 R2T requests to send. The first 64K will be 
sent as immediate data without explicit R2T request.

1. The first target with MaxOutstandingR2T 1.

MaxOutstandingR2T 1 means that only one R2T request can be active on 
time. I.e., the next request for data transfer sent before the previous 
one completed and all the data received.

Thus, on the first target each command will be completed on time:

5 + 0.5 + (5 + 5 + 256K/64K * 0.5) * 7 + (5 + 5 + (256K - 64K)/64K * 
0.5) + 5 = 106ms, i.e. 9 IOPS, which is 18MB/s

2. The second target with MaxOutstandingR2T 16.

With MaxOutstandingR2T 16 the second target can send all R2T requests at 
once.

Thus, on the second target each command will be completed on time:

5 + .5 + 5 + 5 + 256K/64K * 0.5 * 7 + (256K-64K)/64K * 0.5 + 5 = 36ms, 
i.e. 27 IOPS, which is 54MB/s.

Thus, the second target will perform on 54/9 = 600% better that the 
first target.

So, iSCSI protocol isn't so bad for long latency links as many believe. 
It is only needed to be implemented properly.

Vlad