#1274 AMF : dependent SIs are assigned temporarily, when sponsor SI is unassigned (locked )

[Srikanth R]

Setup
Version : 4.6 FC
model : 2n
configuration : 1App,1SG,2SUs with 4comps each, 4SIs with 1 CSI each
si-si deps configured as SI1 is sponsor to SI2,3,&4.
SU1 is mapped to pl-3 and SU2 to pl-4
saAmfSGAutoRepair=1(True)
SuFailover=0(False)
component recovery policy - 3 (comp failover)

Initial state
All the AMF entities regarding the application are in unlocked states. SIs are in fully assigned state.

Issue dependent SIs are assigned temporarily, when sponsor SI is unassigned (locked )

Steps Performed

-> Before shutdown operation of sponsor SI .i.e SI1, ensure that component rejects the callback in saAmfQuiescingComplete call.

-> Perform the shutdown operation on sponsor SI and the component of SU1 rejected the callback in saAmfQuiescingCOmplete call.

Observation & Anomaly

-> As the component1 of SU1 rejected quiescing callback, the component is restarted and the components in this SU1 have received CSI remove callbacks.

-> But the components in SU2 have received dependent CSI active callbacks, which should not be received as sponsort SI1 is in unlocked and unassigned state. Immediately the CSI remove callbacks have been received.

Below is the syslog on PL-3

Mar 19 12:38:57 SYSTEST-PLD-1 osafamfnd[1895]: NO Assigning 'safSi=SI1,safApp=test2nApp' QUIESCING to 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO SU failover probation timer started (timeout: 1200000000000 ns)
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Performing failover of 'safSu=SU1,safSg=SG,safApp=test2nApp' (SU failover count: 1)
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO 'safComp=COMP1,safSu=SU1,safSg=SG,safApp=test2nApp' faulted due to 'qscingCompleteTimeout' : Recovery is 'componentFailover'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO 'safSu=SU1,safSg=SG,safApp=test2nApp' Presence State INSTANTIATED => TERMINATING
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Assigned 'safSi=SI1,safApp=test2nApp' QUIESCING to 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removed 'safSi=SI1,safApp=test2nApp' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removing 'all (3) SIs' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removing 'safSi=SI2,safApp=test2nApp' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removing 'safSi=SI3,safApp=test2nApp' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removing 'safSi=SI4,safApp=test2nApp' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removed 'safSi=SI2,safApp=test2nApp' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removed 'safSi=SI3,safApp=test2nApp' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removed 'safSi=SI4,safApp=test2nApp' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO Removed 'all SIs' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Mar 19 12:39:07 SYSTEST-PLD-1 osafamfnd[1895]: NO 'safSu=SU1,safSg=SG,safApp=test2nApp' Presence State TERMINATING => INSTANTIATED

Below is the syslog on PL-4 :

Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Removed 'safSi=SI1,safApp=test2nApp' from 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigning 'all (3) SIs' ACTIVE to 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigning 'safSi=SI2,safApp=test2nApp' ACTIVE to 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigning 'safSi=SI3,safApp=test2nApp' ACTIVE to 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigning 'safSi=SI4,safApp=test2nApp' ACTIVE to 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigned 'safSi=SI2,safApp=test2nApp' ACTIVE to 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigned 'safSi=SI3,safApp=test2nApp' ACTIVE to 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigned 'safSi=SI4,safApp=test2nApp' ACTIVE to 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigned 'all SIs' ACTIVE of 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Removed 'safSi=SI2,safApp=test2nApp' from 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Removed 'safSi=SI4,safApp=test2nApp' from 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:08 SYSTEST-PLD-2 osafamfnd[24560]: NO Removed 'safSi=SI3,safApp=test2nApp' from 'safSu=SU2,safSg=SG,safApp=test2nApp'
Mar 19 12:39:19 SYSTEST-PLD-2 osafamfnd[24560]: NO Assigning 'safSi=SI1,safApp=test2nApp' STANDBY to 'safSu=SU2,safSg=SG,safApp=test2nApp'

[Praveen]

This issue is reproducible on 4.4 GA changeset:
parent: 5042:2dec93c82bf0 4.4.GA.
Attached are AMF traces and syslog in 1274.tgz after reproducing on 4.4 GA.

[Praveen]

I have verified, #1259 solves this problem also.