Menu

#1006 Unable to bring up opensaf on PPC machines.

4.3.3
fixed
None
defect
mds
-
4.5 FC
major
2014-09-03
2014-08-22
manu
No

Opensaf is not coming up on the peer nodes . Active controller has come up successfully but the other nodes are not joining the cluster.
trying to bring up payload along with the one controller.

Syslogs from payload

Aug 22 04:15:57 linux-cyt3 osafimmnd[24934]: WA Sync MESSAGE:1049 OUT OF ORDER my highest processed:1047
Aug 22 04:15:57 linux-cyt3 opensafd[24900]: ER Failed DESC:IMMND
Aug 22 04:15:57 linux-cyt3 opensafd[24900]: ER Going for recovery
Aug 22 04:15:57 linux-cyt3 opensafd[24900]: ER Trying To RESPAWN /usr/lib64/opensaf/clc-cli/osaf-immnd attempt #1
Aug 22 04:15:57 linux-cyt3 opensafd[24900]: ER Sending SIGKILL to IMMND, pid=24923
Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: Started
Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO SERVER STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING
Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO SERVER STATE: IMM_SERVER_CLUSTER_WAITING --> IMM_SERVER_LOADING_PENDING
Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING
Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO NODE STATE-> IMM_NODE_ISOLATED
Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO NODE STATE-> IMM_NODE_W_AVAILABLE
Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT
Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: WA Sync MESSAGE:1219 OUT OF ORDER my highest processed:1217
Aug 22 04:16:12 linux-cyt3 opensafd[24900]: ER Could Not RESPAWN IMMND
Aug 22 04:16:12 linux-cyt3 opensafd[24900]: ER Failed DESC:IMMND
Aug 22 04:16:12 linux-cyt3 opensafd[24900]: ER Trying To RESPAWN /usr/lib64/opensaf/clc-cli/osaf-immnd attempt #2
Aug 22 04:16:12 linux-cyt3 opensafd[24900]: ER Sending SIGKILL to IMMND, pid=24952
Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: Started
Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO SERVER STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING
Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO SERVER STATE: IMM_SERVER_CLUSTER_WAITING --> IMM_SERVER_LOADING_PENDING
Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING
Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO NODE STATE-> IMM_NODE_ISOLATED
Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO NODE STATE-> IMM_NODE_W_AVAILABLE
Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT
Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: WA Sync MESSAGE:1389 OUT OF ORDER my highest processed:1387
Aug 22 04:16:28 linux-cyt3 opensafd[24900]: ER Could Not RESPAWN IMMND
Aug 22 04:16:28 linux-cyt3 opensafd[24900]: ER Failed DESC:IMMND
Aug 22 04:16:28 linux-cyt3 opensafd[24900]: ER FAILED TO RESPAWN
Aug 22 04:16:28 linux-cyt3 kernel: TIPC: Disabling bearer <eth:eth1>
Aug 22 04:16:28 linux-cyt3 kernel: TIPC: Lost link <1.1.4:eth1-1.1.1:eth1> on network plane A
Aug 22 04:16:28 linux-cyt3 kernel: TIPC: Lost contact with <1.1.1>
Aug 22 04:16:28 linux-cyt3 kernel: TIPC: Left network mode</eth:eth1>

PPC MAchines are having GCC version 4.8.3


linux-pvra:/home # gcc --version
gcc (GCC) 4.8.3
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

1 Attachments

Related

Tickets: #1006

Discussion

  • Anders Bjornerstedt

    • Component: imm --> unknown
     
  • Anders Bjornerstedt

    The offending component is not imm here.

    Yes the symptom here manifests in imm but that symptom is due to
    problems with MDS connectivity or MDS dropping messages.
    Could be problems at the transport layer.
    Could be a configuration error.
    Definitely not an error at the imm level

     
  • manu

    manu - 2014-08-22

    Kernel version : Linux version 2.6.32.12-0.7-ppc64

    TIPC version : TIPC: Activated (version 1.6.4

     
  • Mathi Naickan

    Mathi Naickan - 2014-08-22

    I have doubts about the compatibility. Please provide the TIPC version and kernel version.

    Mathi.

    From: manu [mailto:mmalvi@users.sf.net]
    Sent: Friday, August 22, 2014 4:00 PM
    To: opensaf-tickets@lists.sourceforge.net
    Subject: [tickets] [opensaf:tickets] #1006 Unable to bring up opensaf on PPC machines.


    HYPERLINK "http://sourceforge.net/p/opensaf/tickets/1006"[tickets:#1006] Unable to bring up opensaf on PPC machines.

    Status: unassigned
    Milestone: 4.3.3
    Created: Fri Aug 22, 2014 10:29 AM UTC by manu
    Last Updated: Fri Aug 22, 2014 10:29 AM UTC
    Owner: nobody

    Opensaf is not coming up on the peer nodes . Active controller has come up successfully but the other nodes are not joining the cluster.
    trying to bring up payload along with the one controller.

    Syslogs from payload

    Aug 22 04:15:57 linux-cyt3 osafimmnd[24934]: WA Sync MESSAGE:1049 OUT OF ORDER my highest processed:1047
    Aug 22 04:15:57 linux-cyt3 opensafd[24900]: ER Failed DESC:IMMND
    Aug 22 04:15:57 linux-cyt3 opensafd[24900]: ER Going for recovery
    Aug 22 04:15:57 linux-cyt3 opensafd[24900]: ER Trying To RESPAWN /usr/lib64/opensaf/clc-cli/osaf-immnd attempt #1
    Aug 22 04:15:57 linux-cyt3 opensafd[24900]: ER Sending SIGKILL to IMMND, pid=24923
    Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: Started
    Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO SERVER STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING
    Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO SERVER STATE: IMM_SERVER_CLUSTER_WAITING --> IMM_SERVER_LOADING_PENDING
    Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING
    Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO NODE STATE-> IMM_NODE_ISOLATED
    Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO NODE STATE-> IMM_NODE_W_AVAILABLE
    Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT
    Aug 22 04:16:12 linux-cyt3 osafimmnd[24957]: WA Sync MESSAGE:1219 OUT OF ORDER my highest processed:1217
    Aug 22 04:16:12 linux-cyt3 opensafd[24900]: ER Could Not RESPAWN IMMND
    Aug 22 04:16:12 linux-cyt3 opensafd[24900]: ER Failed DESC:IMMND
    Aug 22 04:16:12 linux-cyt3 opensafd[24900]: ER Trying To RESPAWN /usr/lib64/opensaf/clc-cli/osaf-immnd attempt #2
    Aug 22 04:16:12 linux-cyt3 opensafd[24900]: ER Sending SIGKILL to IMMND, pid=24952
    Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: Started
    Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO SERVER STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING
    Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO SERVER STATE: IMM_SERVER_CLUSTER_WAITING --> IMM_SERVER_LOADING_PENDING
    Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING
    Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO NODE STATE-> IMM_NODE_ISOLATED
    Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO NODE STATE-> IMM_NODE_W_AVAILABLE
    Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT
    Aug 22 04:16:28 linux-cyt3 osafimmnd[24980]: WA Sync MESSAGE:1389 OUT OF ORDER my highest processed:1387
    Aug 22 04:16:28 linux-cyt3 opensafd[24900]: ER Could Not RESPAWN IMMND
    Aug 22 04:16:28 linux-cyt3 opensafd[24900]: ER Failed DESC:IMMND
    Aug 22 04:16:28 linux-cyt3 opensafd[24900]: ER FAILED TO RESPAWN
    Aug 22 04:16:28 linux-cyt3 kernel: TIPC: Disabling bearer <eth:eth1>
    Aug 22 04:16:28 linux-cyt3 kernel: TIPC: Lost link <1.1.4:eth1-1.1.1:eth1> on network plane A
    Aug 22 04:16:28 linux-cyt3 kernel: TIPC: Lost contact with <1.1.1>
    Aug 22 04:16:28 linux-cyt3 kernel: TIPC: Left network mode</eth:eth1>

    PPC MAchines are having GCC version 4.8.3


    linux-pvra:/home # gcc --version
    gcc (GCC) 4.8.3
    Copyright (C) 2013 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions. There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


    Sent from sourceforge.net because HYPERLINK "mailto:opensaf-tickets@lists.sourceforge.net"opensaf-tickets@lists.sourceforge.net is subscribed to HYPERLINK "https://sourceforge.net/p/opensaf/tickets"https://sourceforge.net/p/opensaf/tickets/

    To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

     

    Related

    Tickets: #1006
    Tickets: tickets

  • A V Mahesh (AVM)

    • status: unassigned --> fixed
    • assigned_to: A V Mahesh (AVM)
    • Component: unknown --> mds
     
  • A V Mahesh (AVM)

    Till TIPC 1.7.7 release following multicast fixes got fixed that support Linux kernels 2.6.32.12-0.7-ppc64 and above
    so please use TIPC 1.7.7 for multicast to work properly or optionally apply below TIPC fixes to your existing TIPC version rebuild tipc.ko
    and use.

    =====================================================================================
    1) prevent delivery/non-delivery of multicast messages to out-of-scope/in-scope destinations [http://sourceforge.net/p/tipc/bugs/75/]
    fixed in TIPC 1.7.6

    A long-standing bug in the delivery of multicast messages, affecting TIPC 1.5.3 through 1.7.5 (and possibly affecting earlier versions), has been discovered. This bug may result in an incoming multicast message being incorrectly delivered to ports that published corresponding names with "node" scope, or in a multicast message not being correctly delivered to         ports that published corresponding names with "cluster" or "zone" scope.
    
    Note: This bug will only cause problems if a TIPC node's name table contains publications for the same name (or name sequence) that use both "node" and "cluster"/"zone" scope.
    
    example: An application creates a socket that binds the name {X,Y} with "node" scope; a second application (possibly on a different node) creates another socket that binds the  name     {X,Y} with "cluster" scope.
    

    2) optimization to multicast name lookup algorithm [http://sourceforge.net/p/tipc/bugs/76/] fixed in TIPC 1.7.6

    TIPC's algorithm for identifying on-node and off-node destinations that overlap a multicast name sequence range is not as efficient as it could be. Rather than traversing the list of all known name publications within the cluster, it should just traverse the (potentially much shorter) list of name publications made by the node itself, and determine if any off-node         destinations exist by comparing the sizes of the two lists. (Since the node list must be a subset of the cluster list, a difference in sizes means that at least one off-node destination
     must exist.)
    

    3) renamed "multicast-link" to "broadcast-link" fixed in TIPC 1.7.4

    4) eliminated slight risk of incorrect routing of an incoming bundled multicast message fixed in TIPC 1.7.3

    Just update the readme with preferred TIPC to 1.7.7.

    changeset: 5707:a219a3c45510
    branch: opensaf-4.5.x
    parent: 5705:ed502f4ceb97
    user: A V Mahesh mahesh.valla@oracle.com
    date: Wed Sep 03 11:24:46 2014 +0530
    summary: README: Adjust the TIPC prefered version to 1.7.7 [#1006]

    changeset: 5708:31971a926e22
    tag: tip
    parent: 5706:12251687a7e6
    user: A V Mahesh mahesh.valla@oracle.com
    date: Wed Sep 03 11:26:50 2014 +0530
    summary: README: Adjust the TIPC prefered version to 1.7.7 [#1006]

     

    Related

    Tickets: #1006


Log in to post a comment.