Re: [Scst-devel] Issue with FCST NPIV target port logins from physical FC adapter

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

In the great tradition of asking a question only to answer it... I
think I found the place in the fcoe driver where the priority is set.

In 3.2.19, priority is per-port, and only set up correctly on the default port:
http://lxr.linux.no/linux+v3.2.19/drivers/scsi/fcoe/fcoe.c#L1534

Looking at the latest source, it appears that at some point it was
fixed to use a fcoe-global priority:
http://lxr.linux.no/linux+v3.8/drivers/scsi/fcoe/fcoe.c#L1646

I suspect backporting this fix will fix the issue... will report either way.

-D Wright

On Thu, Feb 21, 2013 at 1:18 PM, Dave Wright <wr...@gm...> wrote:
> I tried clearing the INIT flag, but there was no improvement - port
> discovery is still done the same way, but I no longer think that's
> really the issue.
> I turned up debug logging on the Emulex adapter which lets me see
> basically all the login processing its doing. The packet trace I'm
> capturing is only from the FCOE "target" view - this lets me see the
> other side.
> What I noticed is that it didn't look like the FC adapter was seeing
> *any* of the packets sent from the NPIV port - it doesn't log
> reception of the PLOGI that was sent, nor the rejection LS_RJT sent in
> response to its own PLOGI request.
> The packets are clearly making it out of the target, as they are shown
> in the packet capture, but getting lost in the switch apparently.
>
> Looking at the capture in detail, I think I found the culprit:
> The 802.1q priority is not being set properly on packets sent from the
> NPIV FCOE port.
>
> Compare the two PLOGI requests below. One is sent from the "default"
> FCOE port (FCID b80004), the other from the child NPIV FCOE port (FCID
> b80005). The only significant difference I was able to see is that the
> priority from the NPIV port is 0, instead of 3. Every packet sent from
> the default port gets priority 3, anything from the NPIV port is 0.
> I'm not sure why that would matter in terms of the switch (a Nexus
> 5548) passing the packets on, but it does appear to... or at least
> that's the only explanation I can find.
> FWIW, with an FCOE initiator, the NPIV port still sends packets with
> priority 0, but they are passed on properly.
>
> Any idea where the priority is set and why it would be different for
> the NPIV port? I'm guessing somewhere deep in libfcoe, rather than in
> the scst, since this is all occurring during port discovery.
>
> -D Wright
>
> No.     Time            Source                Destination
> Protocol Length Info
>     271 16:39:29.212528 b8.00.04              b8.00.01              FC
> ELS   180    PLOGI
>
> Frame 271: 180 bytes on wire (1440 bits), 180 bytes captured (1440 bits)
> Ethernet II, Src: 0e:fc:00:b8:00:04 (0e:fc:00:b8:00:04), Dst:
> Cisco_a5:23:d6 (54:7f:ee:a5:23:d6)
>     Destination: Cisco_a5:23:d6 (54:7f:ee:a5:23:d6)
>     Source: 0e:fc:00:b8:00:04 (0e:fc:00:b8:00:04)
>     Type: 802.1Q Virtual LAN (0x8100)
> 802.1Q Virtual LAN, PRI: 3, CFI: 0, ID: 10
>     011. .... .... .... = Priority: Excellent Effort (3)
>     ...0 .... .... .... = CFI: Canonical (0)
>     .... 0000 0000 1010 = ID: 10
>     Type: Fibre Channel over Ethernet (0x8906)
> FCoE (SOFi3/EOFt) 140 bytes
>     Version: 0
>     SOF: SOFi3 (0x2e)
>     CRC: e1ff2796 [valid]
>     EOF: EOFt (0x42)
> Fibre Channel
>     [Exchange Last In: 286]
>     R_CTL: 0x22(Extended Link Services/Request)
>     Dest Addr: b8.00.01
>     CS_CTL: 0x00
>     Src Addr: b8.00.04
>     Type: Ext Link Svc (0x01)
>     F_CTL: 0x290000 Exchange Originator, Seq Initiator, Exchg First,
> Seq Last, CS_CTL, Transfer Seq Initiative
>     SEQ_ID: 0x00
>     DF_CTL: 0x00
>     SEQ_CNT: 0
>     OX_ID: 0x0009
>     RX_ID: 0xffff
>     Parameter: 0x00000000
> FC ELS
>     Cmd Code: PLOGI (0x03)
>     Common Svc Parameters
>         B2B Credit: 10
>         Common Svc Parameters: 0x8000  Cont. Incr. Offset Supported
> Normal B2B Credit Mgmt  E_D_TOV Resolution in ms  Normal SEQCNT rules
> Payload Len=116 bytes
>         0000 .... = BB_SC Number: 0
>         .... 1000 0100 0000 = Receive Size: 2112
>         Max Concurrent Seq: 255
>         Relative Offset By Info Cat: 31
>         E_D_TOV: 2000
>         N_Port Port_Name: 20:00:00:26:9e:cd:2a:3c (QuantaCo)
>         Fabric/Node Name: 10:00:00:26:9e:cd:2a:3c (QuantaCo)
>     Class 1 Svc Parameters
>         Service Options: 0x0000  Class Not Supported
>     Class 2 Svc Parameters
>         Service Options: 0x0000  Class Not Supported
>     Class 3 Svc Parameters
>         Service Options: 0x8800  Non-zero CS_CTL Maybe Tolerated
>         Initiator Ctl: 0x0000  Initial P_A Not Supported
>         Recipient Ctl: 0x0000  ACK0 NOT Supported  Error Policy:
> Discard Policy only  1 Category/Seq
>         Class Recv Size: 2112
>         Total Concurrent Seq: 255
>         End2End Credit: 0
>         Open Seq Per Exchg: 1
>     Class 4 Svc Parameters
>         Service Options: 0x0000  Class Not Supported
>     Vendor Version: 00000000000000000000000000000000
>
> No.     Time            Source                Destination
> Protocol Length Info
>     273 16:39:29.212829 b8.00.05              b8.00.01              FC
> ELS   180    PLOGI
>
> Frame 273: 180 bytes on wire (1440 bits), 180 bytes captured (1440 bits)
> Ethernet II, Src: 0e:fc:00:b8:00:05 (0e:fc:00:b8:00:05), Dst:
> Cisco_a5:23:d6 (54:7f:ee:a5:23:d6)
>     Destination: Cisco_a5:23:d6 (54:7f:ee:a5:23:d6)
>     Source: 0e:fc:00:b8:00:05 (0e:fc:00:b8:00:05)
>     Type: 802.1Q Virtual LAN (0x8100)
> 802.1Q Virtual LAN, PRI: 0, CFI: 0, ID: 10
>     000. .... .... .... = Priority: Best Effort (default) (0)
>     ...0 .... .... .... = CFI: Canonical (0)
>     .... 0000 0000 1010 = ID: 10
>     Type: Fibre Channel over Ethernet (0x8906)
> FCoE (SOFi3/EOFt) 140 bytes
>     Version: 0
>     SOF: SOFi3 (0x2e)
>     CRC: c0321980 [valid]
>     EOF: EOFt (0x42)
> Fibre Channel
>     [Exchange Last In: 0]
>     R_CTL: 0x22(Extended Link Services/Request)
>     Dest Addr: b8.00.01
>     CS_CTL: 0x00
>     Src Addr: b8.00.05
>     Type: Ext Link Svc (0x01)
>     F_CTL: 0x290000 Exchange Originator, Seq Initiator, Exchg First,
> Seq Last, CS_CTL, Transfer Seq Initiative
>     SEQ_ID: 0x00
>     DF_CTL: 0x00
>     SEQ_CNT: 0
>     OX_ID: 0x000a
>     RX_ID: 0xffff
>     Parameter: 0x00000000
> FC ELS
>     Cmd Code: PLOGI (0x03)
>     Common Svc Parameters
>         B2B Credit: 10
>         Common Svc Parameters: 0x8000  Cont. Incr. Offset Supported
> Normal B2B Credit Mgmt  E_D_TOV Resolution in ms  Normal SEQCNT rules
> Payload Len=116 bytes
>         0000 .... = BB_SC Number: 0
>         .... 1000 0100 0000 = Receive Size: 2112
>         Max Concurrent Seq: 255
>         Relative Offset By Info Cat: 31
>         E_D_TOV: 2000
>         N_Port Port_Name: 20:00:00:26:9e:cd:2a:3d (QuantaCo)
>         Fabric/Node Name: 10:00:00:26:9e:cd:2a:3d (QuantaCo)
>     Class 1 Svc Parameters
>         Service Options: 0x0000  Class Not Supported
>     Class 2 Svc Parameters
>         Service Options: 0x0000  Class Not Supported
>     Class 3 Svc Parameters
>         Service Options: 0x8800  Non-zero CS_CTL Maybe Tolerated
>         Initiator Ctl: 0x0000  Initial P_A Not Supported
>         Recipient Ctl: 0x0000  ACK0 NOT Supported  Error Policy:
> Discard Policy only  1 Category/Seq
>         Class Recv Size: 2112
>         Total Concurrent Seq: 255
>         End2End Credit: 0
>         Open Seq Per Exchg: 1
>     Class 4 Svc Parameters
>         Service Options: 0x0000  Class Not Supported
>     Vendor Version: 00000000000000000000000000000000
>
>
>
>
>
>
>
> On 2/21/13, Bart Van Assche <bva...@ac...> wrote:
>> On 02/20/13 00:46, Dave Wright wrote:
>>> From what I can see, it doesn't look like the target is doing anything
>>> differently - the difference appears to be on the initiator.
>>> The really odd thing from my perspective is that I'm not sure why the
>>> initiator would do anything differently, or how it would even know
>>> this is an NPIV port... nothing sent to it should look any different
>>> from the other port.
>>>
>>> One quick question that may help... is there a simple way to put the
>>> FCOE port into a "target only" role? Right now it is acting as an
>>> initiator too, which is presumably why it's even sending PLOGI
>>> requests.
>>
>> I'm not sure but what I noticed after studying the libfc source code is
>> as follows:
>> * fc_fc4_conf_lport_params() sets the FCP_SPPF_TARG_FCN flag when it is
>>    called by fc_lport_config().
>> * I haven't found any code in libfc that allows to disable the
>>    FCP_SPPF_INIT_FCN flag.
>>
>> One possible test is to disable the FCP_SPPF_INIT_FCN flag in fcst
>> before port login occurs. If that works I propose to contact the libfc
>> authors and to ask them to make it possible to disable initiator mode
>> for an FCoE port via sysfs.
>>
>> Bart.
>>