#2463 ntf: ntftest should not run suite 35 if longDnsAllowed is not enabled

[Nguyen TK Luu]

Suite 35 of the NTF API test requires that long-dns support is enabled in the system.

Currently, when the user just runs ntftest without specifying any test suite, and without enabling long-dns support, suite 35 is also run and fails as a result (see below log). This is not expected.

The test should instead check for long-dns support to whether run or not run suite 35.

root@SC-1:~# immcfg -f /tmp/ntfsv_test_classes.xml
root@SC-1:~# ntftest
/usr/local/bin/rdegetrole
This is a controller node

Suite 1: Life cycle, initialize, API 1
1 PASSED saNtfInitialize SA_AIS_OK;
2 PASSED saNtfInitialize with NULL pointer to handle;
3 PASSED saNtfInitialize with NULL pointer to callbacks;
4 PASSED saNtfInitialize with NULL callbacks AND version;
5 PASSED saNtfInitialize with uninitialized handle;
6 PASSED saNtfInitialize with uninitialized version;
7 PASSED saNtfInitialize with too high release level;
8 PASSED saNtfInitialize with minor version set too high;
9 PASSED saNtfInitialize with major version set too high;
10 PASSED saNtfInitialize with minor version set to lower;
11 PASSED saNtfInitialize with major version set to lower;
12 PASSED saNtfInitialize with version A.0.0;

Suite 2: Life cycle, finalize, API 2
1 PASSED saNtfFinalize SA_AIS_OK;
2 PASSED saNtfFinalize SA_AIS_ERR_BAD_HANDLE - invalid handle;
3 PASSED saNtfFinalize SA_AIS_ERR_BAD_HANDLE - handle already returned;
Return value from thread:1
4 PASSED Finalize and Unsubscribe in parallel SA_AIS_OK;
Return value from thread:1
5 PASSED Finalize and saNtfStateChangeNotificationAllocate in parallel SA_AIS_OK;

Suite 3: Life cycle, selectObject, API 3
1 PASSED saNtfSelectionObjectGet SA_AIS_OK;
2 PASSED saNtfSelectionObjectGet NULL handle SA_AIS_ERR_BAD_HANDLE;
3 PASSED saNtfSelectionObjectGet invalid handle SA_AIS_ERR_BAD_HANDLE;
4 PASSED saNtfSelectionObjectGet NULL selectionObject SA_AIS_ERR_INVALID_PARAM;

Suite 4: Life cycle, dispatch, API 4
1 PASSED saNtfDispatch - SA_AIS_OK SA_DISPATCH_ALL;
2 PASSED saNtfDispatch - invalid handle SA_AIS_ERR_BAD_HANDLE;
3 PASSED saNtfDispatch - zero flag SA_AIS_ERR_INVALID_PARAM;

Suite 6: Producer API 2 allocate
1 PASSED saNtfMiscellaneousNotificationAllocate, Not supported;
2 PASSED saNtfMiscellaneousNotificationAllocate SA_AIS_ERR_BAD_HANDLE;
3 PASSED saNtfMiscellaneousNotificationAllocate SA_AIS_ERR_INVALID_PARAM;
4 PASSED saNtfAlarmNotificationAllocate SA_AIS_OK;
5 PASSED saNtfAlarmNotificationAllocate SA_AIS_ERR_BAD_HANDLE;
6 PASSED saNtfAlarmNotificationAllocate SA_AIS_ERR_INVALID_PARAM;
7 PASSED saNtfAttributeChangeNotificationAllocate SA_AIS_OK;
8 PASSED saNtfAttributeChangeNotificationAllocate SA_AIS_ERR_BAD_HANDLE;
9 PASSED saNtfAttributeChangeNotificationAllocate SA_AIS_ERR_INVALID_PARAM;
10 PASSED saNtfStateChangeNotificationAllocate SA_AIS_OK;
11 PASSED saNtfStateChangeNotificationAllocate SA_AIS_ERR_BAD_HANDLE;
12 PASSED saNtfStateChangeNotificationAllocate NULL notification SA_AIS_ERR_INVALID_PARAM;
13 PASSED saNtfSecurityAlarmNotificationAllocate SA_AIS_OK;
14 PASSED saNtfSecurityAlarmNotificationAllocate SA_AIS_ERR_BAD_HANDLE;
15 PASSED saNtfSecurityAlarmNotificationAllocate SA_AIS_ERR_INVALID_PARAM;
16 PASSED saNtfObjectCreateDeleteNotificationAllocate SA_AIS_OK;
17 PASSED saNtfObjectCreateDeleteNotificationAllocate SA_AIS_ERR_BAD_HANDLE;
18 PASSED saNtfObjectCreateDeleteNotificationAllocate SA_AIS_ERR_INVALID_PARAM;

Suite 7: Consumer operations - filter allocate
1 PASSED saNtfStateChangeNotificationFilterAllocate - SA_AIS_OK;
2 PASSED saNtfStateChangeNotificationFilterAllocate - handle null SA_AIS_ERR_BAD_HANDLE;
3 PASSED saNtfStateChangeNotificationFilterAllocate - handle returned SA_AIS_ERR_BAD_HANDLE;
4 PASSED saNtfStateChangeNotificationFilterAllocate - SA_AIS_ERR_INVALID_PARAM;
5 PASSED saNtfSecurityAlarmNotificationFilterAllocate - SA_AIS_OK;
6 PASSED saNtfSecurityAlarmNotificationFilterAllocate - handle null SA_AIS_ERR_BAD_HANDLE;
7 PASSED saNtfSecurityAlarmNotificationFilterAllocate - handle returned SA_AIS_ERR_BAD_HANDLE;
8 PASSED saNtfSecurityAlarmNotificationFilterAllocate - SA_AIS_ERR_INVALID_PARAM;
9 PASSED saNtfObjectCreateDeleteNotificationFilterAllocate - SA_AIS_OK;
10 PASSED saNtfObjectCreateDeleteNotificationFilterAllocate - handle null SA_AIS_ERR_BAD_HANDLE;
11 PASSED saNtfObjectCreateDeleteNotificationFilterAllocate - handle returned SA_AIS_ERR_BAD_HANDLE;
12 PASSED saNtfObjectCreateDeleteNotificationFilterAllocate - SA_AIS_ERR_INVALID_PARAM;
13 PASSED saNtfAttributeChangeNotificationFilterAllocate - SA_AIS_OK;
14 PASSED saNtfAttributeChangeNotificationFilterAllocate - handle NULL SA_AIS_ERR_BAD_HANDLE;
15 PASSED saNtfAttributeChangeNotificationFilterAllocate - handle returned SA_AIS_ERR_BAD_HANDLE;
16 PASSED saNtfAttributeChangeNotificationFilterAllocate - SA_AIS_ERR_INVALID_PARAM;
17 PASSED saNtfAlarmNotificationFilterAllocate - SA_AIS_OK;
18 PASSED saNtfAlarmNotificationFilterAllocate - handle null SA_AIS_ERR_BAD_HANDLE;
19 PASSED saNtfAlarmNotificationFilterAllocate - handle returned SA_AIS_ERR_BAD_HANDLE;
20 PASSED saNtfAlarmNotificationFilterAllocate - SA_AIS_ERR_INVALID_PARAM;

Suite 8: Producer API 3 send
1 PASSED saNtfNotificationSend Alarm;
2 PASSED saNtfNotificationSend ObjectCreateDeleteNotification;
3 PASSED saNtfNotificationSend AttributeChangeNotification;
4 PASSED saNtfNotificationSend StateChangeNotification;
5 PASSED saNtfNotificationSend StateChangeNotification invalid oldStatePresent;
6 PASSED saNtfNotificationSend SecurityAlarm;
7 PASSED saNtfNotificationSend Miscellaneous, Not supported;
8 PASSED saNtfNotificationSend ObjectCreateDeleteNotification SaNameT length=0;
9 PASSED saNtfNotificationSend ObjectCreateDeleteNotification SaNameT length too large;
10 PASSED changedAttributes.attributeType failed SA_AIS_ERR_INVALID_PARAM;
11 PASSED securityAlarmDetector.valueType failed SA_AIS_ERR_INVALID_PARAM;
12 PASSED saNtfNotificationSend ObjectCreateDeleteNotification SaNameT length == 256;
13 PASSED saNtfNotificationSend with mismatched in additionalText and lengthAdditionalText;

Suite 10: Consumer operations - subscribe
1 PASSED saNtfNotificationSubscribe first simple SA_AIS_OK;
2 PASSED saNtfNotificationSubscribe null handle SA_AIS_ERR_INVALID_PARAM;
3 PASSED saNtfNotificationSubscribe All filter handles null SA_AIS_ERR_INVALID_PARAM;
4 PASSED saNtfNotificationSubscribe subscriptionId exist SA_AIS_ERR_EXIST;
5 PASSED saNtfNotificationSubscribe subscriptionId if old handle has subscriptionId had been finalized SA_AIS_OK;

Suite 11: Consumer operations - unsubscribe
1 PASSED saNtfNotificationUnsubscribe first simple SA_AIS_OK;
2 PASSED saNtfNotificationUnsubscribe wrong id SA_AIS_ERR_NOT_EXIST;
3 PASSED saNtfNotificationUnsubscribe no subscription exist SA_AIS_ERR_NOT_EXIST;

Suite 14: Consumer operations - callback
1 PASSED saNtfNotificationCallbackT SA_DISPATCH_ONE;
2 PASSED saNtfNotificationCallbackT SA_DISPATCH_ALL;

Suite 20: Consumer operations - Reader API 1
1 PASSED saNtfNotificationReadInitialize searchCriteria SA_NTF_SEARCH_BEFORE_OR_AT_TIME;
2 PASSED saNtfNotificationReadInitialize searchCriteria SA_NTF_SEARCH_AT_TIME;
3 PASSED saNtfNotificationReadInitialize searchCriteria SA_NTF_SEARCH_AT_OR_AFTER_TIME;
4 PASSED saNtfNotificationReadInitialize searchCriteria SA_NTF_SEARCH_BEFORE_TIME;
5 PASSED saNtfNotificationReadInitialize searchCriteria SA_NTF_SEARCH_AFTER_TIME;
6 PASSED saNtfNotificationReadInitialize searchCriteria SA_NTF_SEARCH_NOTIFICATION_ID;
7 PASSED saNtfNotificationReadInitialize searchCriteria SA_NTF_SEARCH_ONLY_FILTER;
8 PASSED saNtfNotificationReadInitialize searchCriteria invalid SaNtfSearchModeT too big;
9 PASSED saNtfNotificationReadInitialize searchCriteria invalid SaNtfSearchModeT -1;
10 PASSED saNtfNotificationReadInitialize filter NULL pointer SA_AIS_ERR_INVALID_PARAM;
11 PASSED saNtfNotificationReadInitialize filterHandle freed SA_AIS_ERR_BAD_HANDLE;
12 PASSED saNtfNotificationReadInitialize multiple filters with objCreateDelete SA_AIS_ERR_NOT_SUPPORTED;
13 PASSED saNtfNotificationReadInitialize multiple filters with stateChangeFilter SA_AIS_ERR_NOT_SUPPORTED;
14 PASSED saNtfNotificationReadInitialize multiple filters with attributeChange SA_AIS_ERR_NOT_SUPPORTED;

Suite 21: Consumer operations - Reader API 2
1 PASSED saNtfNotificationReadFinalize SA_AIS_OK;

Suite 22: Consumer operations - Reader API 3
1 PASSED saNtfNotificationReadNext SA_AIS_OK;
2 PASSED saNtfNotificationReadNext 3 search younger 3 search older;
3 PASSED saNtfSearchCriteriaT test all;

Suite 23: Consumer operations - discarded notification
1 PASSED saNtfNotificationDiscardedCallbackT Ok;

Suite 28: Producer API
1 PASSED saNtfArrayValAllocate SA_AIS_OK;
2 PASSED saNtfArrayValAllocate bad handle SA_AIS_ERR_BAD_HANDLE;
3 PASSED saNtfArrayValAllocate handle freed SA_AIS_ERR_BAD_HANDLE;
4 PASSED saNtfArrayValAllocate bad dataPtr SA_AIS_ERR_INVLID_PARAM;
5 PASSED saNtfArrayValAllocate bad value pointer SA_AIS_ERR_INVLID_PARAM;
6 PASSED datasize bigger than implementation specific system SA_AIS_ERR_TOO_BIG;
7 PASSED saNtfPtrValAllocate SA_AIS_OK;
8 PASSED saNtfPtrValAllocate bad handle SA_AIS_ERR_BAD_HANDLE;
9 PASSED saNtfPtrValAllocate handle freed SA_AIS_ERR_BAD_HANDLE;
10 PASSED saNtfPtrValAllocate bad dataPtr SA_AIS_ERR_INVLID_PARAM;
11 PASSED saNtfPtrValAllocate bad value pointer SA_AIS_ERR_INVLID_PARAM;
12 PASSED datasize bigger than implementation specific system limit SA_AIS_ERR_TOO_BIG;

Suite 29: Consumer API
1 PASSED saNtfArrayValGet SA_AIS_OK;
2 PASSED saNtfArrayValGet provoke SA_AIS_ERR_BAD_HANDLE/SA_AIS_ERR_INVLID_PARAM;
3 PASSED saNtfPtrValGet SA_AIS_OK;
4 PASSED saNtfPtrValGet provoke SA_AIS_ERR_BAD_HANDLE/SA_AIS_ERR_INVLID_PARAM;

Suite 30: Notification Content Verification
1 PASSED AlarmNotification;
2 PASSED AlarmNotification values;
3 PASSED ObjectCreateDeleteNotification;
4 PASSED AttributeChangeNotification;
5 PASSED StateChangeNotification;
6 PASSED SecurityAlarmNotification;
7 PASSED MiscellaneousNotification, Not supported;
8 PASSED All Notifications at once;

Suite 31: Filter Verification
1 PASSED HeaderFilter all set;
2 PASSED HeaderFilter eventType;
3 PASSED HeaderFilter notificationObjects;
4 PASSED HeaderFilter notifyingObject;
5 PASSED HeaderFilter notificationClassIds;
6 PASSED AlarmNotificationFilter all set;
7 PASSED AlarmNotification probableCause;
8 PASSED AlarmNotification perceivedSeverities;
9 PASSED AlarmNotification trends;
10 PASSED ObjectCreateDeleteNotificationFilter sourceIndicators;
11 PASSED AttributeChangeNotificationFilter sourceIndicators;
12 PASSED StateChangeNotificationFilter sourceIndicators;
13 PASSED StateChangeNotificationFilter stateIds;
14 PASSED SecurityAlarmNotificationFilter probableCauses;
15 PASSED SecurityAlarmNotificationFilter severities;
16 PASSED SecurityAlarmNotificationFilter securityAlarmDetectors;
17 PASSED SecurityAlarmNotificationFilter serviceUsers;
18 PASSED SecurityAlarmNotificationFilter serviceProviders;
19 PASSED Combined filter test;

Suite 32: CM notifications test
1 PASSED CREATE, runtime (OsafNtfCmTestRT) object;
2 PASSED runtime, attr ch, REPLACE (UINT32, FLOAT);
3 PASSED runtime, attr ch, REPLACE (NAME, ANY);
4 PASSED runtime, attr ch, ADD (INT32);
5 PASSED runtime, attr ch, ADD (UINT32);
6 PASSED runtime, attr ch, ADD (INT64);
7 PASSED runtime, attr ch, ADD (UINT64);
8 PASSED runtime, attr ch, ADD (FLOAT);
9 PASSED runtime, attr ch, ADD (DOUBLE);
10 PASSED runtime, attr ch, ADD (TIME);
11 PASSED runtime, attr ch, ADD (STRING);
12 PASSED runtime, attr ch, ADD (NAME);
13 PASSED runtime, attr ch, ADD (ANY);
14 PASSED runtime, attr ch, REPLACE (STRING, DOUBLE);
15 PASSED runtime, attr ch, REPLACE (TIME);
16 PASSED runtime, attr ch, ADD (STRING, INT32);
17 PASSED runtime, attr ch, DELETE (STRING, INT32);
18 PASSED runtime, attr ch, ADD (ALL TYPES);
19 PASSED DELETE, runtime (OsafNtfCmTestRT) object;
20 PASSED CREATE, config (OsafNtfCmTestCFG) object;
21 PASSED config, attr ch, REPLACE (UINT32, FLOAT);
22 PASSED config, attr ch, REPLACE (NAME, ANY);
23 PASSED config, attr ch, ADD (INT32);
24 PASSED config, attr ch, ADD (UINT32);
25 PASSED config, attr ch, ADD (INT64);
26 PASSED config, attr ch, ADD (UINT64);
27 PASSED config, attr ch, ADD (FLOAT);
28 PASSED config, attr ch, ADD (DOUBLE);
29 PASSED config, attr ch, ADD (TIME);
30 PASSED config, attr ch, ADD (STRING);
31 PASSED config, attr ch, ADD (NAME);
32 PASSED config, attr ch, ADD (ANY);
33 PASSED config, attr ch, REPLACE (STRING, DOUBLE);
34 PASSED config, attr ch, REPLACE (TIME);
35 PASSED config, attr ch, ADD (STRING, INT32);
36 PASSED config, attr ch, DELETE (STRING, INT32);
37 PASSED config, attr ch, ADD (ALL TYPES);
38 PASSED config, multiple op in ccb, 2 REPLACE;
39 PASSED config, multiple op in ccb, ADD, REPLACE, DELETE;
40 PASSED DELETE, config (OsafNtfCmTestCFG) object;

Suite 34: CM notifications test, persistent runtime
1 PASSED CREATE, runtime (OsafNtfCmTestRT1) object;
2 PASSED runtime, attr ch, REPLACE (UINT32, FLOAT);
3 PASSED runtime, attr ch, ADD (INT32);
4 PASSED DELETE, runtime (OsafNtfCmTestRT1) object;

Suite 35: CM notification test for extended name attribute
1 FAILED CREATE, runtime (OsafNtfCmTestRT) object, extended name attribute (expected SA_AIS_OK (1), got SA_AIS_ERR_FAILED_OPERATION (21));
2 FAILED runtime, attr ch, REPLACE (EXTENDED NAME, ANY) (expected SA_AIS_OK (1), got SA_AIS_ERR_FAILED_OPERATION (21));
3 FAILED runtime, attr ch, ADD (EXTENDED NAME) (expected SA_AIS_OK (1), got SA_AIS_ERR_FAILED_OPERATION (21));
4 FAILED DELETE, runtime (OsafNtfCmTestRT) object (expected SA_AIS_OK (1), got SA_AIS_ERR_FAILED_OPERATION (21));
error - saImmOmCcbObjectCreate_2 FAILED with SA_AIS_ERR_NAME_TOO_LONG (13)
5 FAILED CREATE, config (OsafNtfCmTestCFG) object, extended name attribute (expected SA_AIS_OK (1), got SA_AIS_ERR_FAILED_OPERATION (21));
Object with DN 'stringRdnCfg=TestObject' does not exist
error - creating attribute from 'testNameCfg=DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD'
6 FAILED config, attr ch, REPLACE (EXTENDED NAME, ANY) (expected SA_AIS_OK (1), got SA_AIS_ERR_FAILED_OPERATION (21));
Object with DN 'stringRdnCfg=TestObject' does not exist
error - creating attribute from 'testNameCfg+=CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC'
7 FAILED config, attr ch, ADD (EXTENDED NAME) (expected SA_AIS_OK (1), got SA_AIS_ERR_FAILED_OPERATION (21));
error - object does not exist
8 FAILED DELETE, config (OsafNtfCmTestCFG) object (expected SA_AIS_OK (1), got SA_AIS_ERR_FAILED_OPERATION (21));

Suite 36: Long DNs Test
1 PASSED Test additional info with extended SaNameT type;
2 PASSED Test filter with longDn objects;
3 PASSED Test Alarm notification with longDn objects;
4 PASSED Test Object Create/Delete notification with longDn objects;
5 PASSED Test Attribute Change notification with longDn objects;
6 PASSED Test State Change notification with longDn objects;
7 PASSED Test Security Alarm notification with longDn objects;

=====================================================================================

Test Result:
Total: 206
Passed: 198
Failed: 8
root@SC-1:~#

[Minh Hon Chau]

why is #2463 a defect?

The src/ntf/apitest/README has documented that suite 35 needs to have IMM LongDN enabled.
When you run ntftest (no arg), or ntftest 35, you see it failed. What's the next step? You would look into to suite 35 to see why it fails, or you just want ntftest passes unconditionally?
Both running ntftest (or ntftest 35) fail as it should, because user has not enabled IMM LongDN support.
It is a defect if ntftest is doing something wrong as it should not.
If you don't want to see any suite failed because of user's misconfiguration, you can skip it, then it is an enhancement. And when you run a test with misconfiguration, then the test should fail.

[Nguyen TK Luu]

Hi Minh,

Speaking of ntf/apitest/README, it also mentions that suite 32 and 34 require some test classes be loaded in advance. And currently, ntftest does check for the existence of the file 'ntfsv_test_classes.xml' before running those suites; otherwise the following message is printed out and those suites will not be run.

if (rc != 0) {
    printf(
        "ntfsv_test_classes.xml file not installed (see README)");
    return;
}

That means ntftest already had some pre-conditions checks before running test. That's why I think we can have similar check for long-dn support for suite 35. And with the class file check was already there, the missing of long-dn check shall be considered be a defect, in my opinion.

Besides, other services' apitests also check (CLM) or make sure (LOG) long-dn support is enabled before running or skipping specific test cases.

It's my point of view on this issue. As the NTF maintainer, you can have final decision whether this is a defect or enhancement.

Thanks,
/Nguyen

[Minh Hon Chau]

commit 169e313feafe10d0a866a8abe231edf5d503760a
Author: Nguyen Luu nguyen.tk.luu@dektech.com.au
Date: Wed May 24 10:05:01 2017 +1000

commit ec979abf57f5d229ab5576b72e89d6e8d006d224
Author: Nguyen Luu nguyen.tk.luu@dektech.com.au
Date: Wed May 24 10:05:01 2017 +1000