Menu

#517 Amfnd: coredumps when calling immutil_saImmOmInitialize

5.1.FC
fixed
None
defect
amf
nd
4.3
major
2016-07-07
2013-07-24
No

avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
There are two problems here, immutil should not do abort and the other problem
is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

(gdb) bt
#0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
#1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
#2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
#3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)
at ../../../../../osaf/tools/safimm/src/immutil.c:1126
#4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
#5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=<optimized out="">) at avnd_susm.c:1236
#6 0x000000000042ffe0 in avnd_evt_process (evt=<optimized out="">) at avnd_proc.c:278
#7 avnd_main_process () at avnd_proc.c:219
#8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71

Related

Tickets: #1728
Tickets: #517
Wiki: ChangeLog-4.7.2
Wiki: ChangeLog-5.0.1

Discussion

1 2 3 > >> (Page 1 of 3)
  • Hans Nordebäck

    Hans Nordebäck - 2013-07-24
    • summary: Amfnd: --> Amfnd: coredumps when calling immutil_saImmOmInitialize
     
  • Nagendra Kumar

    Nagendra Kumar - 2013-08-13
    • status: unassigned --> assigned
    • assigned_to: Nagendra Kumar
     
  • Hans Feldt

    Hans Feldt - 2013-08-13

    This could happen if the local immnd is down, then there would be a deadlock like this. I think there are some old tickets on devel.opensaf.org that touches this

     
  • Anders Bjornerstedt

    I dont think this should be called a deadlock.
    The imm service would reply immediately with TRY_AGAIN on attempts to
    allocate an imm handle when the local
    immnd is down. Any TRY_AGAIN loop has to be limited in number of retries
    and sleep times, whether in
    immutils or implemented directly. So not really a deadlock.

    /AndersBj

    Hans Feldt wrote:

    This could happen if the local immnd is down, then there would be a deadlock like this. I think there are some old tickets on devel.opensaf.org that touches this


    [tickets:#517] Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Tue Aug 13, 2013 05:22 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt
    #0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    #1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    #2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    #3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)
    at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    #4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    #5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=<optimized out="">) at avnd_susm.c:1236
    #6 0x000000000042ffe0 in avnd_evt_process (evt=<optimized out="">) at avnd_proc.c:278
    #7 avnd_main_process () at avnd_proc.c:219
    #8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is subscribed to https://sourceforge.net/p/opensaf/tickets/

    To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.



    Get 100% visibility into Java/.NET code with AppDynamics Lite!
    It's a free troubleshooting tool designed for production.
    Get down to code-level detail for bottlenecks, with <2% overhead.
    Download for free and get started troubleshooting in minutes.
    http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk



    Opensaf-tickets mailing list
    Opensaf-tickets@lists.sourceforge.net
    https://lists.sourceforge.net/lists/listinfo/opensaf-tickets

     
  • Hans Feldt

    Hans Feldt - 2013-08-13

    Old ticket: https://sourceforge.net/p/opensaf/tickets/395/
    not sure if this one is needed. it contains no logs nothing...

     
  • Anders Bjornerstedt

     
  • Anders Bjornerstedt

    This kind of problem is also related to imm enhancement ticket [#27]

    https://sourceforge.net/p/opensaf/tickets/27/

    Not much help right now for solving this ticket.

    The idea in (#27) is that a select few operations, such as implementer-set, class-implementer-set, class-implementer-release
    could be possible to handle (i.e. forwarded) by the local immnd while it is still syncing, thus not exposing the handle.
    I am assuming in this case that the need for the oi-handle-initialize is due to a handle having been exposed.

     

    Related

    Tickets: #27

  • Nagendra Kumar

    Nagendra Kumar - 2013-08-14

    We need some more information on this issue. Is it an application SU or middleware SU instantiation ? I guess this is an application SU instantiation when opensafd is stoppoing and immnd is not available. Please confirm.

     

    Last edit: Nagendra Kumar 2013-08-14
  • Bertil Engelholm

    What triggered this problem was that they powered off SC-2 after a cluster reboot. This causes immnd on some of the PL's to restart (OUT OF ORDER messages). On one of the PL's (PL-5) the immnd failed to contact the immd after the restart causing the amfnd OmInitialize to timeout with TRY AGAIN.
    So the issues here are :
    Why did some of the immnd's restart when SC-2 was lost.
    Why did immnd on PL-5 fail to contact immd after it restarted.
    This indicates communication problems in MDS/TIPC.
    The question still remains what amfnd should do when IMM communication times out.

     
  • Anders Bjornerstedt

    Yes the answer to the two first Why issues is MDS problems (dropped messages without explicit link loss).
    A possible folowup question is Why there was MDS problems ?
    This error scenario is clearly unusual (or is it just that we dont test power off of av SC that often?).

    /AndersBj


    From: Bertil Engelholm [mailto:ebereng@users.sf.net]
    Sent: den 16 augusti 2013 11:05
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    What triggered this problem was that they powered off SC-2 after a cluster reboot. This causes immnd on some of the PL's to restart (OUT OF ORDER messages). On one of the PL's (PL-5) the immnd failed to contact the immd after the restart causing the amfnd OmInitialize to timeout with TRY AGAIN.
    So the issues here are :
    Why did some of the immnd's restart when SC-2 was lost.
    Why did immnd on PL-5 fail to contact immd after it restarted.
    This indicates communication problems in MDS/TIPC.
    The question still remains what amfnd should do when IMM communication times out.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Wed Aug 14, 2013 06:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     

    Related

    Tickets: #517

  • Anders Bjornerstedt

    The lost messages in this case are so called fevs (fake evs messaes).
    Each fevs message is individually checkpointed to standby SC and then "broadcast" to all IMMNDs using MDS brodcast
    to the service-id of IMMND. This broadcast I understand not to be a real broadcast but a multicast to the set of receivers.
    I can understand if the SC that is being shut down is the active IMMD tht it could succed partially in the multicast of
    one fevs message.

    The strange thing about this case is that the MDS loss of messages detected at one or more PLs was a gap of 30+ messages.
    This proves that it was not a case of the active IMMD being interrupted (in the MDS library) by the power off.
    THe issue seems more like a buffering issue. The send buffer at the active SC towards some PLs (but not all) apparently contained 30+ messages
    at time of power off.

    After failover the new active immnd will resend the two latest fevs messages it received over mbcp.
    Here would actually have needed to resend 30+ fevs messages.

    AndersBj


    From: Anders Bjornerstedt [mailto:andersbj@users.sf.net]
    Sent: den 16 augusti 2013 11:20
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    Yes the answer to the two first Why issues is MDS problems (dropped messages without explicit link loss).
    A possible folowup question is Why there was MDS problems ?
    This error scenario is clearly unusual (or is it just that we dont test power off of av SC that often?).

    /AndersBj


    From: Bertil Engelholm [mailto:ebereng@users.sf.net]
    Sent: den 16 augusti 2013 11:05
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    What triggered this problem was that they powered off SC-2 after a cluster reboot. This causes immnd on some of the PL's to restart (OUT OF ORDER messages). On one of the PL's (PL-5) the immnd failed to contact the immd after the restart causing the amfnd OmInitialize to timeout with TRY AGAIN.
    So the issues here are :
    Why did some of the immnd's restart when SC-2 was lost.
    Why did immnd on PL-5 fail to contact immd after it restarted.
    This indicates communication problems in MDS/TIPC.
    The question still remains what amfnd should do when IMM communication times out.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Wed Aug 14, 2013 06:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236
    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278
    7 avnd_main_process () at avnd_proc.c:219
    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Fri Aug 16, 2013 09:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     
  • Nagendra Kumar

    Nagendra Kumar - 2013-08-19

    The question still remains what amfnd should do when IMM communication times out.
    If Immnd is not responsive for configured timeout, then Amfnd aborts and it traslates into rebooting the node. It looks justifiable for Amfnd as untill it doesn't get configurable information, it can't proceeds and hence, finally it looks like local node issue.
    Immnd must respond in real time.

     

    Last edit: Nagendra Kumar 2013-08-19
  • Anders Bjornerstedt

    The reason that imm is not responsive appears to be that the local IMMND is down.
    So one question is why is IMMND down locally?
    Did it crash or exit ?

    Second, if IMMND goes down it will restart and try to get synced.
    Such a sync should take less than 60 seconds.
    It depends of course on the ammount of data and the hardware used.
    But a typical normal sync takes 10 - 20 seconds.

    If the sync can not get started (no IMMD service ?) then that would explain it.

    Since the problem was reproduced I assume there are syslogs.

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 06:37
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The question still remains what amfnd should do when IMM communication times out.
    If Immnd is not responsive for configured timeout, then Amfnd aborts and it traslates into rebooting the node. It looks justifiable for Amfnd as untill it doesn't get configurable information, it can't proceeds and hence, finally it looks like local node issue.
    Immnd must response in real time.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Fri Aug 16, 2013 09:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     
  • Anders Bjornerstedt

    In the code I can see that the unresponsive local IMMND can not be due
    to a long sync.
    The TRY_AGAIN is only given when the local IMMND is dow.
    If the local IMMND was syncing then it would be up.
    In that case it would discover the handle to be stale and it actually
    replies with SA_AIS_OK in
    saImmOiImplementerClear in this situation.

    So the reason that IMMND is not reponsing is that it has gone down and
    not been restarted.

    /AndersBj

    Anders Bjornerstedt wrote:

    The reason that imm is not responsive appears to be that the local IMMND is down.
    So one question is why is IMMND down locally?
    Did it crash or exit ?

    Second, if IMMND goes down it will restart and try to get synced.
    Such a sync should take less than 60 seconds.
    It depends of course on the ammount of data and the hardware used.
    But a typical normal sync takes 10 - 20 seconds.

    If the sync can not get started (no IMMD service ?) then that would explain it.

    Since the problem was reproduced I assume there are syslogs.

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 06:37
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The question still remains what amfnd should do when IMM communication times out.
    If Immnd is not responsive for configured timeout, then Amfnd aborts and it traslates into rebooting the node. It looks justifiable for Amfnd as untill it doesn't get configurable information, it can't proceeds and hence, finally it looks like local node issue.
    Immnd must response in real time.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Fri Aug 16, 2013 09:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517] Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt
    #0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    #1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    #2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    #3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)
    at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    #4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    #5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=<optimized out="">) at avnd_susm.c:1236
    #6 0x000000000042ffe0 in avnd_evt_process (evt=<optimized out="">) at avnd_proc.c:278
    #7 avnd_main_process () at avnd_proc.c:219
    #8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     
  • Nagendra Kumar

    Nagendra Kumar - 2013-08-19

    If we could improve immnd behaviour from 'restarting' to 'resync the data without restart' in case of counter mismatch(which is very frequent), these kind of problems may get resolved.

    -Nagu

    -----Original Message-----
    From: Anders Bjornerstedt [mailto:andersbj@users.sf.net]
    Sent: 19 August 2013 12:46
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The reason that imm is not responsive appears to be that the local IMMND is down.
    So one question is why is IMMND down locally?
    Did it crash or exit ?

    Second, if IMMND goes down it will restart and try to get synced.
    Such a sync should take less than 60 seconds.
    It depends of course on the ammount of data and the hardware used.
    But a typical normal sync takes 10 - 20 seconds.

    If the sync can not get started (no IMMD service ?) then that would explain it.

    Since the problem was reproduced I assume there are syslogs.

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 06:37
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The question still remains what amfnd should do when IMM communication times out.
    If Immnd is not responsive for configured timeout, then Amfnd aborts and it traslates into rebooting the node. It looks justifiable for Amfnd as untill it doesn't get configurable information, it can't proceeds and hence, finally it looks like local node issue.
    Immnd must response in real time.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Fri Aug 16, 2013 09:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517] Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt
    #0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    #1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    #2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    #3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)
    at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    #4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    #5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=<optimized out="">) at avnd_susm.c:1236
    #6 0x000000000042ffe0 in avnd_evt_process (evt=<optimized out="">) at avnd_proc.c:278
    #7 avnd_main_process () at avnd_proc.c:219
    #8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     
  • Anders Bjornerstedt

    What kind of counter missmatch are you talking about that is "frequent" ?

    And why is the IMMND not restarted ?

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 10:47
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    If we could improve immnd behaviour from 'restarting' to 'resync the data without restart' in case of counter mismatch(which is very frequent), these kind of problems may get resolved.

    -Nagu

    -----Original Message-----
    From: Anders Bjornerstedt [mailto:andersbj@users.sf.net]
    Sent: 19 August 2013 12:46
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The reason that imm is not responsive appears to be that the local IMMND is down.
    So one question is why is IMMND down locally?
    Did it crash or exit ?

    Second, if IMMND goes down it will restart and try to get synced.
    Such a sync should take less than 60 seconds.
    It depends of course on the ammount of data and the hardware used.
    But a typical normal sync takes 10 - 20 seconds.

    If the sync can not get started (no IMMD service ?) then that would explain it.

    Since the problem was reproduced I assume there are syslogs.

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 06:37
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The question still remains what amfnd should do when IMM communication times out.
    If Immnd is not responsive for configured timeout, then Amfnd aborts and it traslates into rebooting the node. It looks justifiable for Amfnd as untill it doesn't get configurable information, it can't proceeds and hence, finally it looks like local node issue.
    Immnd must response in real time.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Fri Aug 16, 2013 09:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236
    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278
    7 avnd_main_process () at avnd_proc.c:219
    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     
  • Anders Bjornerstedt

    Ok looked back at the ticket.
    Thiswas a fevs count missmatch and those I claim are rare.
    The can happen when MDS drops messages, which is rare.

    Or at least it used to be rare.
    Perhapse there is an introduced problem with MDS in some recent release.
    The ticklet is lacking both version and milestone so I cant tell.

    /AndersBj


    From: Anders Björnerstedt
    Sent: den 19 augusti 2013 10:52
    To: '[opensaf:tickets] '
    Subject: RE: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    What kind of counter missmatch are you talking about that is "frequent" ?

    And why is the IMMND not restarted ?

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 10:47
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    If we could improve immnd behaviour from 'restarting' to 'resync the data without restart' in case of counter mismatch(which is very frequent), these kind of problems may get resolved.

    -Nagu

    -----Original Message-----
    From: Anders Bjornerstedt [mailto:andersbj@users.sf.net]
    Sent: 19 August 2013 12:46
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The reason that imm is not responsive appears to be that the local IMMND is down.
    So one question is why is IMMND down locally?
    Did it crash or exit ?

    Second, if IMMND goes down it will restart and try to get synced.
    Such a sync should take less than 60 seconds.
    It depends of course on the ammount of data and the hardware used.
    But a typical normal sync takes 10 - 20 seconds.

    If the sync can not get started (no IMMD service ?) then that would explain it.

    Since the problem was reproduced I assume there are syslogs.

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 06:37
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The question still remains what amfnd should do when IMM communication times out.
    If Immnd is not responsive for configured timeout, then Amfnd aborts and it traslates into rebooting the node. It looks justifiable for Amfnd as untill it doesn't get configurable information, it can't proceeds and hence, finally it looks like local node issue.
    Immnd must response in real time.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Fri Aug 16, 2013 09:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236
    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278
    7 avnd_main_process () at avnd_proc.c:219
    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     

    Related

    Tickets: #517

  • Nagendra Kumar

    Nagendra Kumar - 2013-08-19

    The reason of immnd down is frequent asserts/aborts in the code all around.

    Logs like "OUT OF ORDER my highest processed" are frequent. immnd can flush its existing data and resync again.

    -Nagu
    -----Original Message-----
    From: Anders Bjornerstedt [mailto:andersbj@users.sf.net]
    Sent: 19 August 2013 14:23
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    What kind of counter missmatch are you talking about that is "frequent" ?

    And why is the IMMND not restarted ?

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 10:47
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    If we could improve immnd behaviour from 'restarting' to 'resync the data without restart' in case of counter mismatch(which is very frequent), these kind of problems may get resolved.

    -Nagu

    -----Original Message-----
    From: Anders Bjornerstedt [mailto:andersbj@users.sf.net]
    Sent: 19 August 2013 12:46
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The reason that imm is not responsive appears to be that the local IMMND is down.
    So one question is why is IMMND down locally?
    Did it crash or exit ?

    Second, if IMMND goes down it will restart and try to get synced.
    Such a sync should take less than 60 seconds.
    It depends of course on the ammount of data and the hardware used.
    But a typical normal sync takes 10 - 20 seconds.

    If the sync can not get started (no IMMD service ?) then that would explain it.

    Since the problem was reproduced I assume there are syslogs.

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 06:37
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The question still remains what amfnd should do when IMM communication times out.
    If Immnd is not responsive for configured timeout, then Amfnd aborts and it traslates into rebooting the node. It looks justifiable for Amfnd as untill it doesn't get configurable information, it can't proceeds and hence, finally it looks like local node issue.
    Immnd must response in real time.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Fri Aug 16, 2013 09:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236
    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278
    7 avnd_main_process () at avnd_proc.c:219
    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71 ________

    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517] Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Tue Aug 13, 2013 12:16 PM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt
    #0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    #1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    #2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    #3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)
    at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    #4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    #5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=<optimized out="">) at avnd_susm.c:1236
    #6 0x000000000042ffe0 in avnd_evt_process (evt=<optimized out="">) at avnd_proc.c:278
    #7 avnd_main_process () at avnd_proc.c:219
    #8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     

    Related

    Tickets: #517

  • Anders Bjornerstedt

    Can we please avoid non constructive comments like "Frequent asserts/aborts all arround".
    That statement just implies poor quality of the IMMND which I claim is not a justified claim.
    If thre are uncontrolled crashes of the immsv then they should be reported in a ticket.

    If the IMD really ios crashing so much then trying to "flush data" and resync without restarting would be insane.

    Lets try to focus again on this particular case, with the fevs count missmatch and try
    to avoid throwing arround ungrounded "blame".

    If fevs count problems are frequent then MDS is simply unreliable.
    This has NOT been a fequent problem before so iff it IS a problem now the OpenSAF has BIG
    problems with MDS. Trying to compensate newly discovered problems with MDS by having the
    immsv re-syncing is not the correct order od priority.

    The question then is why has the frequencey of MDS dropping messages increased so much.
    What has changed in MDS or the setup/configuration or test ??

    ISo if this is indeed a case (that we now see frequent problems with MDS dropping messages)
    then there needs to be a critical ticket looking in to thet.

    Finally, I woul still like to know why the IMMND is not restarted.
    Regardless of how frequent the FEVS count missmatch (i.e. MDS problems) are.

    /Anders


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 11:08
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The reason of immnd down is frequent asserts/aborts in the code all around.

    Logs like "OUT OF ORDER my highest processed" are frequent. immnd can flush its existing data and resync again.

    -Nagu
    -----Original Message-----
    From: Anders Bjornerstedt [mailto:andersbj@users.sf.net]
    Sent: 19 August 2013 14:23
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    What kind of counter missmatch are you talking about that is "frequent" ?

    And why is the IMMND not restarted ?

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 10:47
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    If we could improve immnd behaviour from 'restarting' to 'resync the data without restart' in case of counter mismatch(which is very frequent), these kind of problems may get resolved.

    -Nagu

    -----Original Message-----
    From: Anders Bjornerstedt [mailto:andersbj@users.sf.net]
    Sent: 19 August 2013 12:46
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] Re: #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The reason that imm is not responsive appears to be that the local IMMND is down.
    So one question is why is IMMND down locally?
    Did it crash or exit ?

    Second, if IMMND goes down it will restart and try to get synced.
    Such a sync should take less than 60 seconds.
    It depends of course on the ammount of data and the hardware used.
    But a typical normal sync takes 10 - 20 seconds.

    If the sync can not get started (no IMMD service ?) then that would explain it.

    Since the problem was reproduced I assume there are syslogs.

    /AndersBj


    From: Nagendra Kumar [mailto:nagendra-k@users.sf.net]
    Sent: den 19 augusti 2013 06:37
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    The question still remains what amfnd should do when IMM communication times out.
    If Immnd is not responsive for configured timeout, then Amfnd aborts and it traslates into rebooting the node. It looks justifiable for Amfnd as untill it doesn't get configurable information, it can't proceeds and hence, finally it looks like local node issue.
    Immnd must response in real time.


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/http://sourceforge.net/p/opensaf/tickets/517/http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Fri Aug 16, 2013 09:05 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236
    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278
    7 avnd_main_process () at avnd_proc.c:219
    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71 __

    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236
    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278
    7 avnd_main_process () at avnd_proc.c:219
    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6
    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6
    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70
    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0) at ../../../../../osaf/tools/safimm/src/immutil.c:1126
    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743
    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236
    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278
    7 avnd_main_process () at avnd_proc.c:219
    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano Last Updated: Tue Aug 13, 2013 12:16 PM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     

    Related

    Tickets: #517

  • Hans Feldt

    Hans Feldt - 2013-08-19

    First there are logs but they are still internal. And immnd is restarted by AMF

     
  • Anders Bjornerstedt

    Ok, my mistake, I was looking at #516 wihere saImmOiImplementerClear was the problem.
    In that ticket there is talk about "the imm becomming slow".
    But in actuality the IMMND is not at all "slow" there.
    It is simply not being restarted in case #516.

    In #517, the IMMND has apparently been restarted and is syncing, but there the problem seems to be trivial
    in that the default timeout is used in immutils. An imm sync can take more than 10 seconds.

    About the claim that the fevs-count missmatch problem happens "often".
    That claim nees to be evaluated, since if it is true then we dont have a reliable system.

    /AndersBj


    From: Hans Feldt [mailto:hansfeldt@users.sf.net]
    Sent: den 19 augusti 2013 11:41
    To: [opensaf:tickets]
    Subject: [opensaf:tickets] #517 Amfnd: coredumps when calling immutil_saImmOmInitialize

    First there are logs but they are still internal. And immnd is restarted by AMF


    [tickets:#517]http://sourceforge.net/p/opensaf/tickets/517/ Amfnd: coredumps when calling immutil_saImmOmInitialize

    Status: assigned
    Created: Wed Jul 24, 2013 08:29 AM UTC by hano
    Last Updated: Mon Aug 19, 2013 04:37 AM UTC
    Owner: Nagendra Kumar

    avnd_comp_config_get_su calls immutil_saImmOmInitialize and ImmutilWrapperProfile is not set by avnd so default values are used and immutil calls abort at errors.
    There are two problems here, immutil should not do abort and the other problem
    is that immutil do sleeps in try again loops. Avnd is event based and immutil is configurable but errors and try-again logic has to be managed by avnd.

    (gdb) bt

    0 0x00007ffac238cb35 in raise () from /lib64/libc.so.6

    1 0x00007ffac238e111 in abort () from /lib64/libc.so.6

    2 0x00000000004051e8 in defaultImmutilError (fmt=0x43fc70 "saImmOmInitialize FAILED, rc = %d") at ../../../../../osaf/tools/safimm/src/immutil.c:70

    3 0x00000000004065e4 in immutil_saImmOmInitialize (immHandle=0x7fff7098d380, immCallbacks=0x0, version=0x7fff7098d3a0)

    at ../../../../../osaf/tools/safimm/src/immutil.c:1126

    4 0x0000000000422551 in avnd_comp_config_get_su (su=0x66d8d0) at avnd_compdb.c:1743

    5 0x0000000000436560 in avnd_evt_avd_su_pres_evh (cb=0x6578c0, evt=) at avnd_susm.c:1236

    6 0x000000000042ffe0 in avnd_evt_process (evt=) at avnd_proc.c:278

    7 avnd_main_process () at avnd_proc.c:219

    8 0x0000000000408805 in main (argc=1, argv=0x7fff7098d578) at amfnd_main.c:71


    Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/517/

    To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

     
  • Nagendra Kumar

    Nagendra Kumar - 2013-08-22
    • Version: --> 4.3
     
  • Nagendra Kumar

    Nagendra Kumar - 2013-08-30

    In broad level, Amfnd reads configuration from Immnd in below scenario:
    1. During CSI Assignment.
    2. SU instantiation.
    3. Amfnd comming up.
    4. During clc script running.

    Mostly, we are getting stuck at #2.

    One Solution looks to me for #2 : In case Amfnd fails to get information from local Immnd(Timeout), it can get the information from Amfd by some means as Amfd has those information already.

    Any thoughts ?

     
    • Hans Feldt

      Hans Feldt - 2013-08-30

      For more info see: http://devel.opensaf.org/~hafe/AMF/amfnd-start.png

      Getting the config from amfd kind of means reverting back to the 3.0 state. Good or bad I don't know yet. But it seems unnecessary to distribute information that is already available locally on the node. At the same time there are some hard to solve problems.

      I envision a separate thread in amfnd that read from IMM and handles TRYAGAIN loops. Such thread could also be an applier for the case of detecting config changes

       
  • Nagendra Kumar

    Nagendra Kumar - 2013-08-30

    Or If Amfnd fails to get information from local Immnd(Timeout), it can get the information from any other non-local Immnd.

     
1 2 3 > >> (Page 1 of 3)

Log in to post a comment.