#1587 amfnd: amfnd crashes when comp instantiation fails as a part of comp restart recovery

[Praveen]

Reproducible on 4.6 GA changeset, Logs and configuration attached.

Configuration: NPI, 2N model, 2SUs , 2 Cpmps in each SUcomp restart recovery. PM is enabled.
Steps to reproduce:
1)Bring attached NPI configuration up.
2)Make arrangements so that instantiation of component will fail.
3)Now kill a a npi comp.
4)After successful clean up, AMFND will instantiate comp.
5)Instantiation of component fails and amfnd crash will be observed.

bt:
#0 0x00000039ef630265 in raise () from /lib64/libc.so.6
#1 0x00000039ef631d10 in abort () from /lib64/libc.so.6
#2 0x00007f1753b9fcd9 in osafassert_fail (file=0x45ea09 "di.cc", line=579,
func=0x45f0b0 "avnd_di_susi_resp_send", __assertion=0x45eca5 "m_AVND_SU_IS_ASSIGN_PEND(su)")
at sysf_def.c:281
#3 0x0000000000431141 in avnd_di_susi_resp_send (cb=0x6745c0, su=0xd43fa0, si=0xd2ec40) at di.cc:579
#4 0x0000000000449aa1 in avnd_su_pres_st_chng_prc (cb=0x6745c0, su=0xd43fa0,
prv_st=SA_AMF_PRESENCE_INSTANTIATED, final_st=SA_AMF_PRESENCE_INSTANTIATION_FAILED) at susm.cc:1702
#5 0x00000000004490f5 in avnd_su_pres_fsm_run (cb=0x6745c0, su=0xd43fa0, comp=0xd4af40,
ev=AVND_SU_PRES_FSM_EV_COMP_INST_FAIL) at susm.cc:1455
#6 0x000000000041cb52 in avnd_comp_clc_st_chng_prc (cb=0x6745c0, comp=0xd4af40,
prv_st=SA_AMF_PRESENCE_RESTARTING, final_st=SA_AMF_PRESENCE_INSTANTIATION_FAILED) at clc.cc:1309
#7 0x000000000041b015 in avnd_comp_clc_fsm_run (cb=0x6745c0, comp=0xd4af40,
ev=AVND_COMP_CLC_PRES_FSM_EV_CLEANUP_SUCC) at clc.cc:873
#8 0x000000000041a079 in avnd_evt_clc_resp_evh (cb=0x6745c0, evt=0xd2b720) at clc.cc:416
#9 0x00000000004384ef in avnd_evt_process (evt=0xd2b720) at main.cc:660
#10 0x0000000000438345 in avnd_main_process () at main.cc:604
#11 0x00000000004376fd in main (argc=2, argv=0x7fff5b765678) at main.cc:178

amfnd log at that time:

Nov 9 11:43:48.266961 osafamfnd [26826:comp.cc:2737] IN 'safComp=AmfDemo1,safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1' Presence State INSTANTIATED => TERMINATING
Nov 9 11:43:48.266968 osafamfnd [26826:di.cc:0672] >> avnd_di_object_upd_send: Comp 'safComp=AmfDemo1,safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1'
Nov 9 11:43:48.266979 osafamfnd [26826:di.cc:0777] >> avnd_di_msg_send: Msg type '8'
Nov 9 11:43:48.266985 osafamfnd [26826:di.cc:0968] >> avnd_diq_rec_add
Nov 9 11:43:48.267012 osafamfnd [26826:di.cc:0983] << avnd_diq_rec_add
Nov 9 11:43:48.267018 osafamfnd [26826:di.cc:1044] >> avnd_diq_rec_send
Nov 9 11:43:48.267027 osafamfnd [26826:mds.cc:1169] >> avnd_mds_send: Msg type '1'
Nov 9 11:43:48.267265 osafamfnd [26826:mds.cc:1224] << avnd_mds_send: 1
Nov 9 11:43:48.267273 osafamfnd [26826:di.cc:1064] << avnd_diq_rec_send: 1
Nov 9 11:43:48.267278 osafamfnd [26826:di.cc:0809] << avnd_di_msg_send: 1
Nov 9 11:43:48.267283 osafamfnd [26826:di.cc:0691] << avnd_di_object_upd_send: 1
Nov 9 11:43:48.267289 osafamfnd [26826:clc.cc:1813] << avnd_comp_clc_inst_term_hdler
Nov 9 11:43:48.267294 osafamfnd [26826:clc.cc:0867] T1 Exited CLC FSM
Nov 9 11:43:48.267299 osafamfnd [26826:clc.cc:0869] T1 'safComp=AmfDemo1,safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1':FSM Enter presence state: 'SA_AMF_PRESENCE_INSTANTIATED(3)':FSM Exit presence state:SA_AMF_PRESENCE_TERMINATING(4)
Nov 9 11:43:48.267305 osafamfnd [26826:clc.cc:0901] >> avnd_comp_clc_st_chng_prc: Comp 'safComp=AmfDemo1,safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1', Prv_state '3', Final_state '4'
Nov 9 11:43:48.267310 osafamfnd [26826:clc.cc:1159] TR SU and Comp Non-Preinst. comp->su->flag '1024', comp->flag '1'
Nov 9 11:43:48.267316 osafamfnd [26826:susm.cc:1407] >> avnd_su_pres_fsm_run: 'safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1'
Nov 9 11:43:48.267322 osafamfnd [26826:susm.cc:1412] T1 Entering SU presence state FSM: current state: 6, event: 9, su name:safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1
Nov 9 11:43:48.267327 osafamfnd [26826:susm.cc:1424] T1 Exited SU presence state FSM: New State = 6
Nov 9 11:43:48.267332 osafamfnd [26826:susm.cc:1341] TR 'safComp=AmfDemo1,safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1' not terminated, pres.st=4
Nov 9 11:43:48.267338 osafamfnd [26826:susm.cc:1458] << avnd_su_pres_fsm_run: 1
Nov 9 11:43:48.267343 osafamfnd [26826:clc.cc:1339] << avnd_comp_clc_st_chng_prc: 1
Nov 9 11:43:48.267348 osafamfnd [26826:clc.cc:0876] << avnd_comp_clc_fsm_run: 1
Nov 9 11:43:48.267359 osafamfnd [26826:susm.cc:2971] << avnd_su_pres_inst_compinstfail_hdler: 1
Nov 9 11:43:48.267365 osafamfnd [26826:susm.cc:1424] T1 Exited SU presence state FSM: New State = 6
Nov 9 11:43:48.267370 osafamfnd [26826:susm.cc:1341] TR 'safComp=AmfDemo1,safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1' not terminated, pres.st=4
Nov 9 11:43:48.267375 osafamfnd [26826:susm.cc:1539] >> avnd_su_pres_st_chng_prc: 'safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1' INSTANTIATED => INSTANTIATION_FAILED
Nov 9 11:43:48.267381 osafamfnd [26826:susm.cc:1681] TR NPI SU :'safSu=SU1,safSg=AmfDemo,safApp=AmfDemo1'
Nov 9 11:43:48.267386 osafamfnd [26826:susm.cc:1699] TR SU Instantiating/Instantiated -> Instantiation Failed
Nov 9 11:43:48.267391 osafamfnd [26826:susm.cc:1701] TR SI-Assignment failed, Informing AVD

[Praveen]

https://sourceforge.net/p/opensaf/mailman/message/34749649/

changeset: 7265:259f9f90346a
branch: opensaf-4.6.x
parent: 7255:4d78415096a8
user: praveen.malviya@oracle.com
date: Thu Jan 28 17:09:10 2016 +0530
summary: amfnd: fix amfnd crash due to inst failure of a restarting npi comp [#1587]

changeset: 7266:ab7e1297ca5f
branch: opensaf-4.7.x
parent: 7259:88c4c9cb4167
user: praveen.malviya@oracle.com
date: Thu Jan 28 17:09:41 2016 +0530
summary: amfnd: fix amfnd crash due to inst failure of a restarting npi comp [#1587]

changeset: 7267:8af40542b113
tag: tip
parent: 7264:a30247a1affb
user: praveen.malviya@oracle.com
date: Thu Jan 28 17:10:01 2016 +0530
summary: amfnd: fix amfnd crash due to inst failure of a restarting npi comp [#1587]