Re: [Scst-devel] iSCSI ALUA failover

[Adrian S. <Adr...@tp...>]

I enabled a debug build to try and figure this out, and what I found was that scst_tg_change_tgt_dev_state() is never being called.  The reason was due to the order in which I create the target and the device_group target_groups, but after fixing that Solaris now seems to be able to use the UA to notice the state changes and update.

The only outlier is this error on solaris when the failover is detected:

Feb 26 12:29:32 labx4200 iscsi: [ID 438296 kern.warning] WARNING: iscsi connection(110/21) protocol error - received status out of order itt:0x34f70003 statsn:0x307a expstatsn:0x3079

Which seems to cause the connection to be reset but otherwise it continues working.

Thanks,
 Adrian


-----Original Message-----
From: Vladislav Bolkhovitin [mailto:vs...@vl...]
Sent: Wednesday, 24 February 2016 3:31 PM
To: Adrian Saul; scs...@li...
Subject: Re: [Scst-devel] iSCSI ALUA failover

Hi Adrian,

I would suggest you to calm down a little bit. SCST is doing it right. I would be very surprised to know otherwise. I wrote you exact way how to test it, are there any problems to try?

Your source code read below is not complete, because you didn't notice
scst_tg_change_tgt_dev_state() function.

AENs look in Wireshark as Asynchronous Message.

Vlad

Adrian Saul wrote on 02/23/2016 06:43 PM:
> Hi Vlad,
>
> I have tried both active/standby and active/nonoptimized combinations,
> as well as various tests where I transitioned through the "transitioning" state before setting
> the end state and tested putting ports into offline state.    I also have set the
> preferred value to match the active target as well.
>
> I even tried a modified build where I changed the scst_tg_set_state()
> function to call __scst_gen_alua_state_changed_ua() following
> __scst_tg_set_state(), just to be sure it was triggering a UA
> following a state change.  As I read it the UA is only done on change
> of the preferred option, but as I was toggling that anyway it should have applied regardless.  At any rate it made no difference.
>
> As an side, what should an AEN look like from a wireshark capture?  I
> can see lots of iSCSI NOP calls but nothing that appears to indicate
> the AEN event.  If I do the manual sg_rtpg command I can see SCSI
> calls from that, but when triggering the state changes I see nothing but iSCSI NOP.
>
> I would just like to be able to present Oracle with some information
> to confirm what is happening from the target side before I chase up support on it.
>
> Thanks, Adrian
>
>
>
>
> -----Original Message----- From: Vladislav Bolkhovitin [mailto:vs...@vl...] Sent:
> Wednesday, 24 February 2016 12:21 PM To: Adrian Saul;
> scs...@li... Subject: Re: [Scst-devel] iSCSI ALUA
> failover
>
> Hi,
>
> Yes, SCST does everything as expected. For iSCSI the state change UA
> is delivered via AEN. I can guess, that potentially Solaris may not
> like this way of delivery due to some bugs (it's perfectly standard).
> You can check it by commenting out ".report_aen = iscsi_report_aen," line in the end of iscsi.c file.
>
> Are you using only "active" and "nonoptimized" states?
>
> Vlad
>
> Adrian Saul wrote on 02/23/2016 03:51 PM:
>> Hi SCST-devel,
>>
>> I am working on using SCST (currently 3.1.1.r6809) to build a system
>> for presenting Ceph LUNs as iSCSI targets.  I have a working system
>> that appeared to work fine with the Linux multipath configuration and seemed to successfully fail
>> over when changing which node was the active path.   This is all orchestrated with
>> pacemaker using some custom agents I have developed.
>>
>>
>>
>> However when using with Solaris (specifically Solaris 11.3, but 10 is
>> affected the same)  I seem to be unable to have Solaris recognise that the path states
>> have changed.   The initial discovery is correct, but if I change the path states
>> Solaris does not recognise it – the mpathadm output still shows the original path
>> configurations not the configuration as set on SCST.    If I use the sg_rtpg
>> command I can see that the path states are accurately represented
>> from that query, just not in the Solaris mpathadm output.
>>
>>
>>
>> I would just like to understand what SCST does (or expects the client to do) in
>> order for it do discover the changes to the portal groups.   From some reading I
>> see there is a mention of using AEN, however I don’t see any iSCSI
>> traffic I can identify as being that event.
>>
>>
>>
>> Also I want to check I am using the correct method of changing the
>> state – currently I simply write either  “active” or “nonoptimized”
>> to the target_groups state in sysfs depending on if the node should be the active host.  Do I need to
>> use the transitioning states or poke something else on changes?   Using a
>> transitioning or offline state also seems to be ignored by Solaris
>> until it actually errors for some other reason (iSCSI session closed).
>>
>>
>>
>> The only thing I can find around what Solaris expects is this:
>>
>>
>>
>> http://www.oracle.com/technetwork/server-storage/solaris/tpgs-support
>> -
>> 136354.html
>>
>>
>>
>> “After an implicit asymmetric access state change, a device server
>> establishes a unit attention condition for the initiator port
>> associated with every I_T nexus with the additional sense code set to ASYMMETRIC ACCESS STATE CHANGED.”
>>
>>
>>
>> Can I confirm that SCST follows this behaviour?
>>
>>
>>
>> Thanks,
>>
>> Adrian
>
>
> ----------------------------------------------------------------------
> --------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________ Scst-devel mailing
> list https://lists.sourceforge.net/lists/listinfo/scst-devel
> Confidentiality: This email and any attachments are confidential and
> may be subject to copyright, legal or some other professional
> privilege. They are intended solely for the attention and use of the
> named addressee(s). They may only be copied, distributed or disclosed
> with the consent of the copyright owner. If you have received this
> email by mistake or by breach of the confidentiality clause, please
> notify the sender immediately by return email and delete or destroy
> all copies of the email. Any confidentiality, privilege or copyright is not waived or lost because this email has been sent to you by mistake.
> ----------------------------------------------------------------------
> --------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________ Scst-devel mailing
> list https://lists.sourceforge.net/lists/listinfo/scst-devel
>

Confidentiality: This email and any attachments are confidential and may be subject to copyright, legal or some other professional privilege. They are intended solely for the attention and use of the named addressee(s). They may only be copied, distributed or disclosed with the consent of the copyright owner. If you have received this email by mistake or by breach of the confidentiality clause, please notify the sender immediately by return email and delete or destroy all copies of the email. Any confidentiality, privilege or copyright is not waived or lost because this email has been sent to you by mistake.