#2211 SMF: Unnecessary sleep during callAdminOperation causes too much traffic outage during upgrade

[Tai Dinh]

SmfImmUtils::callAdminOperation currently has a 2 seconds sleep at the end of each admin operation.
The initial purpose of this is to provdie a short breath for the system before retrying again on the failure case.

But unconditional sleep also slowdown the function call lead to longer time for the service to be up.

In case of single step campaign, where the service is only activated at procWrapup action, if we have about 300 SUs then we'll have more 20 minutes servcie outage which is not acceptable.

Sleep should only be done at retry.

See attached patch for more information.

[Tai Dinh]

Note: there are similar issue on other places too. They should also be considered to be fixed.
e.g:

int retry = 100;
do {
    TRACE("immutil_saImmOmAdminOperationInvoke_2 %s", __FUNCTION__);
    m_errno = immutil_saImmOmAdminOperationInvoke_2(
        m_ownerHandle,
        &nodeGroupName, 0, adminOp, params,
        &oi_rc, smfd_cb->adminOpTimeout);

    if (retry <= 0) {
        LOG_NO("Fail to invoke admin operation, too many OI "
            "TRY_AGAIN, giving up. %s", __FUNCTION__);
        break;
    }
    sleep(2);
    retry--;
} while (m_errno == SA_AIS_OK && oi_rc == SA_AIS_ERR_TRY_AGAIN);

Since we normally do not have too much NGs, this is lower priority.

/Tai

[Neelakanta Reddy]

changeset: 8398:bc37759532ea
branch: opensaf-5.0.x
parent: 8384:441994664aec
user: Neelakanta Reddy reddy.neelakanta@oracle.com
date: Thu Dec 01 16:36:33 2016 +0530
summary: smf: Avoid unconditional sleep when calling adminoperation[#2211]

changeset: 8399:c493dfa77eab
branch: opensaf-5.1.x
parent: 8396:7c92427bfd93
user: Neelakanta Reddy reddy.neelakanta@oracle.com
date: Thu Dec 01 16:36:33 2016 +0530
summary: smf: Avoid unconditional sleep when calling adminoperation[#2211]

changeset: 8400:8fa2efee5365
tag: tip
parent: 8397:21094b948d29
user: Neelakanta Reddy reddy.neelakanta@oracle.com
date: Thu Dec 01 16:36:33 2016 +0530
summary: smf: Avoid unconditional sleep when calling adminoperation[#2211]