Menu

#942 AMF: SC failover during standby SC shutdown causes app problems

4.3.3
fixed
None
defect
amf
-
4.3
major
2014-06-26
2014-06-12
Hans Feldt
No

If system shutdown including opensaf on the standby controller is in progress when a SC failover happens, applications can have problems accessing opensaf services.

For example if some AMF modeled component tries to write and close a checkpoint as the last thing it does before exit, it gets stuck because there is no ckpt service available.

Proposed is to escalate the graceful shutdown to non-graceful, see attached patch.

1 Attachments

Related

Tickets: #942
Wiki: ChangeLog-4.3.3
Wiki: ChangeLog-4.4.1

Discussion

  • Praveen

    Praveen - 2014-06-13

    During opensaf shutdown on any node, AMNFD removes assignments from all
    the components.
    After successful removal of assignments, all the components are cleaned up.
    During removal of assignments all services will be available for
    applications.
    An application will keep only system resources for cleanup and not any
    opensaf related.
    So this problem is in removal or cleanup phase during shutdown?

    Thanks
    Praveen

    On 12-Jun-14 4:48 PM, Hans Feldt wrote:


    [tickets:#942] http://sourceforge.net/p/opensaf/tickets/942/ AMF:
    SC failover during standby SC shutdown causes app problems

    Status: unassigned
    Milestone: 4.3.3
    Created: Thu Jun 12, 2014 11:18 AM UTC by Hans Feldt
    Last Updated: Thu Jun 12, 2014 11:18 AM UTC
    Owner: nobody

    If system shutdown including opensaf on the standby controller is in
    progress when a SC failover happens, applications can have problems
    accessing opensaf services.

    For example if some AMF modeled component tries to write and close a
    checkpoint as the last thing it does before exit, it gets stuck
    because there is no ckpt service available.

    Proposed is to escalate the graceful shutdown to non-graceful, see
    attached patch.


    Sent from sourceforge.net because
    opensaf-tickets@lists.sourceforge.net is subscribed to
    https://sourceforge.net/p/opensaf/tickets/

    To unsubscribe from further messages, a project admin can change
    settings at https://sourceforge.net/p/opensaf/admin/tickets/options.
    Or, if this is a mailing list, you can unsubscribe from the mailing list.


    HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
    Find What Matters Most in Your Big Data with HPCC Systems
    Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
    Leverages Graph Analysis for Fast Processing & Easy Data Exploration
    http://p.sf.net/sfu/hpccsystems


    Opensaf-tickets mailing list
    Opensaf-tickets@lists.sourceforge.net
    https://lists.sourceforge.net/lists/listinfo/opensaf-tickets

     

    Related

    Tickets: #942

    • Hans Feldt

      Hans Feldt - 2014-06-13

      From: Praveen [mailto:praveenmalviya@users.sf.net]
      Sent: den 13 juni 2014 10:22
      To: [opensaf:tickets]
      Subject: [opensaf:tickets] Re: #942 AMF: SC failover during standby SC shutdown causes app problems

      During opensaf shutdown on any node, AMNFD removes assignments from all
      the components.
      After successful removal of assignments, all the components are cleaned up.
      During removal of assignments all services will be available for
      applications.
      An application will keep only system resources for cleanup and not any
      opensaf related.
      So this problem is in removal or cleanup phase during shutdown?

      [Hans] removal, SC standby being shutdown, SC failover happens. App stuck accessing opensaf services

      Thanks
      Praveen

      On 12-Jun-14 4:48 PM, Hans Feldt wrote:


      [tickets:#942]http://sourceforge.net/p/opensaf/tickets/942/ http://sourceforge.net/p/opensaf/tickets/942/ AMF:
      SC failover during standby SC shutdown causes app problems

      Status: unassigned
      Milestone: 4.3.3
      Created: Thu Jun 12, 2014 11:18 AM UTC by Hans Feldt
      Last Updated: Thu Jun 12, 2014 11:18 AM UTC
      Owner: nobody

      If system shutdown including opensaf on the standby controller is in
      progress when a SC failover happens, applications can have problems
      accessing opensaf services.

      For example if some AMF modeled component tries to write and close a
      checkpoint as the last thing it does before exit, it gets stuck
      because there is no ckpt service available.

      Proposed is to escalate the graceful shutdown to non-graceful, see
      attached patch.


      Sent from sourceforge.net because
      opensaf-tickets@lists.sourceforge.netopensaf-tickets@lists.sourceforge.net is subscribed to
      https://sourceforge.net/p/opensaf/tickets/

      To unsubscribe from further messages, a project admin can change
      settings at https://sourceforge.net/p/opensaf/admin/tickets/options.
      Or, if this is a mailing list, you can unsubscribe from the mailing list.


      HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
      Find What Matters Most in Your Big Data with HPCC Systems
      Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
      Leverages Graph Analysis for Fast Processing & Easy Data Exploration
      http://p.sf.net/sfu/hpccsystems


      Opensaf-tickets mailing list
      Opensaf-tickets@lists.sourceforge.netOpensaf-tickets@lists.sourceforge.net
      https://lists.sourceforge.net/lists/listinfo/opensaf-tickets


      [tickets:#942]http://sourceforge.net/p/opensaf/tickets/942/ AMF: SC failover during standby SC shutdown causes app problems

      Status: unassigned
      Milestone: 4.3.3
      Created: Thu Jun 12, 2014 11:18 AM UTC by Hans Feldt
      Last Updated: Thu Jun 12, 2014 11:18 AM UTC
      Owner: nobody

      If system shutdown including opensaf on the standby controller is in progress when a SC failover happens, applications can have problems accessing opensaf services.

      For example if some AMF modeled component tries to write and close a checkpoint as the last thing it does before exit, it gets stuck because there is no ckpt service available.

      Proposed is to escalate the graceful shutdown to non-graceful, see attached patch.


      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/942/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

       

      Related

      Tickets: #942

      • Praveen

        Praveen - 2014-06-17

        On 13-Jun-14 1:57 PM, Hans Feldt wrote:

        From: Praveen [mailto:praveenmalviya@users.sf.net]
        Sent: den 13 juni 2014 10:22
        To: [opensaf:tickets]
        Subject: [opensaf:tickets] Re: #942 AMF: SC failover during standby SC
        shutdown causes app problems

        During opensaf shutdown on any node, AMNFD removes assignments from all
        the components.
        After successful removal of assignments, all the components are
        cleaned up.
        During removal of assignments all services will be available for
        applications.
        An application will keep only system resources for cleanup and not any
        opensaf related.
        So this problem is in removal or cleanup phase during shutdown?

        [Hans] removal, SC standby being shutdown, SC failover happens. App
        stuck accessing opensaf services

        Please send the patch officially.Please add steps to reproduce and
        traces if possible in the ticket.

        Thanks,
        Praveen

        Thanks
        Praveen

        On 12-Jun-14 4:48 PM, Hans Feldt wrote:


        [tickets:#942]
        http://sourceforge.net/p/opensaf/tickets/942/http://sourceforge.net/p/opensaf/tickets/942/
        http://sourceforge.net/p/opensaf/tickets/942/ AMF:
        SC failover during standby SC shutdown causes app problems

        Status: unassigned
        Milestone: 4.3.3
        Created: Thu Jun 12, 2014 11:18 AM UTC by Hans Feldt
        Last Updated: Thu Jun 12, 2014 11:18 AM UTC
        Owner: nobody

        If system shutdown including opensaf on the standby controller is in
        progress when a SC failover happens, applications can have problems
        accessing opensaf services.

        For example if some AMF modeled component tries to write and close a
        checkpoint as the last thing it does before exit, it gets stuck
        because there is no ckpt service available.

        Proposed is to escalate the graceful shutdown to non-graceful, see
        attached patch.


        Sent from sourceforge.net because
        opensaf-tickets@lists.sourceforge.netopensaf-tickets@lists.sourceforge.net
        opensaf-tickets@lists.sourceforge.net is subscribed to
        https://sourceforge.net/p/opensaf/tickets/

        To unsubscribe from further messages, a project admin can change
        settings at https://sourceforge.net/p/opensaf/admin/tickets/options.
        Or, if this is a mailing list, you can unsubscribe from the mailing list.


        HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
        Find What Matters Most in Your Big Data with HPCC Systems
        Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
        Leverages Graph Analysis for Fast Processing & Easy Data Exploration
        http://p.sf.net/sfu/hpccsystems


        Opensaf-tickets mailing list
        Opensaf-tickets@lists.sourceforge.netOpensaf-tickets@lists.sourceforge.net
        Opensaf-tickets@lists.sourceforge.net
        https://lists.sourceforge.net/lists/listinfo/opensaf-tickets


        [tickets:#942]
        http://sourceforge.net/p/opensaf/tickets/942/http://sourceforge.net/p/opensaf/tickets/942/
        AMF: SC failover during standby SC shutdown causes app problems

        Status: unassigned
        Milestone: 4.3.3
        Created: Thu Jun 12, 2014 11:18 AM UTC by Hans Feldt
        Last Updated: Thu Jun 12, 2014 11:18 AM UTC
        Owner: nobody

        If system shutdown including opensaf on the standby controller is in
        progress when a SC failover happens, applications can have problems
        accessing opensaf services.

        For example if some AMF modeled component tries to write and close a
        checkpoint as the last thing it does before exit, it gets stuck
        because there is no ckpt service available.

        Proposed is to escalate the graceful shutdown to non-graceful, see
        attached patch.


        Sent from sourceforge.net because you indicated interest in
        https://sourceforge.net/p/opensaf/tickets/942/

        To unsubscribe from further messages, please visit
        https://sourceforge.net/auth/subscriptions/


        [tickets:#942] http://sourceforge.net/p/opensaf/tickets/942/ AMF:
        SC failover during standby SC shutdown causes app problems

        Status: unassigned
        Milestone: 4.3.3
        Created: Thu Jun 12, 2014 11:18 AM UTC by Hans Feldt
        Last Updated: Thu Jun 12, 2014 11:18 AM UTC
        Owner: nobody

        If system shutdown including opensaf on the standby controller is in
        progress when a SC failover happens, applications can have problems
        accessing opensaf services.

        For example if some AMF modeled component tries to write and close a
        checkpoint as the last thing it does before exit, it gets stuck
        because there is no ckpt service available.

        Proposed is to escalate the graceful shutdown to non-graceful, see
        attached patch.


        Sent from sourceforge.net because
        opensaf-tickets@lists.sourceforge.net is subscribed to
        http://sourceforge.net/p/opensaf/tickets/

        To unsubscribe from further messages, a project admin can change
        settings at http://sourceforge.net/p/opensaf/admin/tickets/options.
        Or, if this is a mailing list, you can unsubscribe from the mailing list.


        HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
        Find What Matters Most in Your Big Data with HPCC Systems
        Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
        Leverages Graph Analysis for Fast Processing & Easy Data Exploration
        http://p.sf.net/sfu/hpccsystems


        Opensaf-tickets mailing list
        Opensaf-tickets@lists.sourceforge.net
        https://lists.sourceforge.net/lists/listinfo/opensaf-tickets

         

        Related

        Tickets: #942

  • Nagendra Kumar

    Nagendra Kumar - 2014-06-17

    I think this situation can arise on Act controller as well during si swap.
    1. stop Act controller.
    2. Keep gdb in application remove callback.
    3. Perform si-swap.

    So, in this case, do we want to cleanup? If yes, then patch attached needs correction.

    Thanks
    -Nagu

     
  • Praveen

    Praveen - 2014-06-23
    • status: unassigned --> assigned
    • assigned_to: Hans Feldt
     
  • Praveen

    Praveen - 2014-06-25
    • status: assigned --> review
     
  • Praveen

    Praveen - 2014-06-26
    • status: review --> fixed
     
  • Praveen

    Praveen - 2014-06-26

    changeset: 5440:8a9922b1a259
    branch: opensaf-4.3.x
    parent: 5425:2708a5dae72f
    user: hans.feldt@ericsson.com
    date: Thu Jun 26 14:04:52 2014 +0530
    summary: amfnd : speed up shutdown of standby SC upon failover during its shutdown [#942]

    changeset: 5441:6c22d0adee66
    branch: opensaf-4.4.x
    parent: 5426:6c5ec93a5b68
    user: hans.feldt@ericsson.com
    date: Thu Jun 26 14:05:10 2014 +0530
    summary: amfnd : speed up shutdown of standby SC upon failover during its shutdown [#942]

    changeset: 5442:f76e4b952685
    tag: tip
    parent: 5439:ec5147693bd5
    user: hans.feldt@ericsson.com
    date: Thu Jun 26 14:05:21 2014 +0530
    summary: amfnd : speed up shutdown of standby SC upon failover during its shutdown [#942]

    [staging:8a9922]
    [staging:6c22d0]
    [staging:f76e4b]

     

    Related

    Tickets: #942
    Commit: [6c22d0]
    Commit: [8a9922]
    Commit: [f76e4b]


Log in to post a comment.