NmtNodeCommandConfErr in demo_mn_console

[Axel Fischer]

Hi everybody,

I'm currently porting openPowerlink on Fiasco/OC (a member of the L4 Microkernel family) and Genode (operating system framework). For testing, I run the system on an emulated PBX-A9 board using QEMU with VDE.

When I start the ported openPowerlink stack with the included demo_cn_console application (running on Fiasco/OC and Genode) and trying to connect to a MN using the demo_mn_console application (running on Ubuntu 16.04 LTS), I receive the following output/error at the MN:

[...]
2018/04/04-20:15:21 EVENT STATE_CHANGE NmtGsResetCommunication->NmtGsResetConfiguration Originating event:NmtEventEnterResetConfig
2018/04/04-20:15:21 EVENT STATE_CHANGE NmtGsResetConfiguration->NmtMsNotActive Originating event:NmtEventEnterMsNotActive
Stack entered state: NmtMsNotActive
2018/04/04-20:15:22 EVENT STATE_CHANGE NmtMsNotActive->NmtMsPreOperational1 Originating event:NmtEventTimerMsPreOp1
Stack entered state: NmtMsPreOperational1
2018/04/04-20:15:23 EVENT STATE_CHANGE NmtMsPreOperational1->NmtMsPreOperational2 Originating event:NmtEventTimerMsPreOp2
Stack entered state: NmtMsPreOperational2
2018/04/04-20:15:23 EVENT STATE_CHANGE NmtMsPreOperational2->NmtMsReadyToOperate Originating event:NmtEventEnterReadyToOperate
Stack entered state: NmtMsReadyToOperate
2018/04/04-20:15:23 EVENT STATE_CHANGE NmtMsReadyToOperate->NmtMsOperational Originating event:NmtEventEnterMsOperational
Stack entered state: NmtMsOperational
2018/04/04-20:15:39 EVENT NODE Node= 1, NmtNodeEventNmtState State:NmtCsPreOperational2
Node 1 entered state NmtCsPreOperational2
2018/04/04-20:15:39 EVENT NODE Node= 1, NmtNodeEventFound State:NmtCsPreOperational2
Stack found node 1
2018/04/04-20:15:39 EVENT NODE Node= 1, NmtNodeEventCheckConf State:NmtCsPreOperational2
2018/04/04-20:15:39 EVENT NODE Node= 1, NmtNodeEventError State:NmtCsNotActive
2018/04/04-20:15:39 EVENT NODE Node= 1, NmtNodeEventNmtState State:NmtCsNotActive
Node 1 entered state NmtCsNotActive
2018/04/04-20:15:39 EVENT CFM_PROGRESS Node= 1, Object 0x1011/001, 4/ 162 Bytes -> SDO Abort=SDO_AC_DATA_NOT_TRANSF_DUE_LOCAL_CONTROL(0x08000021), Error=0x0000
2018/04/04-20:15:42 EVENT NODE Node= 1, NmtNodeEventNmtState State:NmtCsPreOperational2
Node 1 entered state NmtCsPreOperational2
2018/04/04-20:15:42 EVENT NODE Node= 1, NmtNodeEventFound State:NmtCsPreOperational2
Stack found node 1
2018/04/04-20:15:42 EVENT NODE Node= 1, NmtNodeEventCheckConf State:NmtCsPreOperational2
2018/04/04-20:15:42 EVENT NODE Node= 1, NmtNodeEventError State:NmtCsNotActive
2018/04/04-20:15:42 EVENT NODE Node= 1, NmtNodeEventNmtState State:NmtCsNotActive
Node 1 entered state NmtCsNotActive
2018/04/04-20:15:42 EVENT CFM_PROGRESS Node= 1, Object 0x1011/001, 4/ 162 Bytes -> SDO Abort=SDO_AC_DATA_NOT_TRANSF_DUE_LOCAL_CONTROL(0x08000021), Error=0x0000

The CN continues to reset and retry.
What could be potential causes of this error?

Thank you very much for your help!

Best regards
Axel

[Powerlink-team Kalycito]

Hi Axel Fischer,

The error you have highlighted is not the reason for the issue you are facing. This error usually occurs when the configuration gets downloaded to CN.

The reset and retry behaviour that you see in CN might be because of the timeout or cycle time values. Kindly let us know if you have used the default CDC generated or provide following details if you have customized the CDC to assist you better
1. Cycle time
2. CN PRestimeout

Thanks,
Powerlink-Team-Kalycito

[Johannes Fischer]

Hi Powerlink Team,

I continue the work of Axel Fischer and got stuck at the same error after a full
porting to v2.7.1. The source code will be published soon, so let me know if you
want to take a deeper look at the porting.

The setup did not change a lot since Axel's attempt.

• Master Node (MN) runs on Ubuntu 18.04.4 LTS
• Client Node (CN) runs with demo_cn_console application and openPOWERLINK is linked as complete library.
• The CN runs within the OS Genode on the microkernel Fiasco.OC, which is emulated on a PBX-A9 board using QEMU
• CN and MN are connected through a network bridge
• asynchronous phase works

I should mention, that the network traffic is really slow, which is due to the
ARM emulation. We work on bringing the CN on a real Zynq-7000 hardware.

Kindly let us know if you have used the default CDC generated or provide
following details if you have customized the CDC to assist you better.

I used the default CDC for the following logs.

The log of the MN (full log is in attachment mn.txt):

2020/10/03-14:53:25 EVENT    NODE           Node=  1, NmtNodeEventCheckConf State:NmtCsPreOperational2
2020/10/03-14:53:30 EVENT    CFM_PROGRESS   Node=  1, Object 0x1011/001,    4/ 162 Bytes
2020/10/03-14:53:35 EVENT    CFM_PROGRESS   Node=  1, Object 0x1600/000,   11/ 162 Bytes
2020/10/03-14:53:35 EVENT    CFM_RESULT     Node=  1, NmtNodeCommandConfErr
2020/10/03-14:53:35 EVENT    NODE           Node=  1, NmtNodeEventError State:NmtCsNotActive
2020/10/03-14:53:35 EVENT    NODE           Node=  1, NmtNodeEventNmtState State:NmtCsNotActive
Node 1 entered state NmtCsNotActive
2020/10/03-14:53:38 EVENT    NODE           Node=  1, NmtNodeEventNmtState State:NmtCsPreOperational2
Node 1 entered state NmtCsPreOperational2
2020/10/03-14:53:38 EVENT    NODE           Node=  1, NmtNodeEventFound State:NmtCsPreOperational2
Stack found node 1
2020/10/03-14:53:38 EVENT    NODE           Node=  1, NmtNodeEventCheckConf State:NmtCsPreOperational2
2020/10/03-14:53:38 EVENT    NODE           Node=  1, NmtNodeEventError State:NmtCsNotActive
2020/10/03-14:53:38 EVENT    NODE           Node=  1, NmtNodeEventNmtState State:NmtCsNotActive
Node 1 entered state NmtCsNotActive
2020/10/03-14:53:38 EVENT    CFM_PROGRESS   Node=  1, Object 0x1011/001,    4/
162 Bytes -> SDO Abort=SDO_AC_DATA_NOT_TRANSF_DUE_LOCAL_CONTROL(0x08000021),
Error=0x0000

The log of the CN (full log is in attachment cn.txt):

[init -> demo_cn_console] 0/0/0-0:0:0 EVENT    STATE_CHANGE   NmtCsPreOperational1->NmtCsPreOperational2 Originating event:NmtEventDllCeSoc
[init -> demo_cn_console] Stack entered state:NmtCsPreOperational2
[init -> demo_cn_console] U2K type:EventTypeNmtEvent(1) sink:EventSinkNmtk(1) size:1!
[init -> demo_cn_console]
[init -> demo_cn_console] nmtu_processEvent(): NMT state machine announce change of NMT state
[init -> demo_cn_console]
[init -> demo_cn_console] UINT  type:EventTypeAsndRx(15) sink:EventSinkDlluCal(3) size:60!
[init -> demo_cn_console]
[init -> demo_cn_console] Handle: 0x4000 , First data Byte 0x0
[init -> demo_cn_console]
[init -> demo_cn_console] sdoseq: processStateInit2()
[init -> demo_cn_console]
[init -> demo_cn_console] Error during initialization
[init -> demo_cn_console]
[init -> demo_cn_console] pdoklut_getChannel() channel:255 index:0 nodeId:0
[init -> demo_cn_console]
[init -> demo_cn_console] pdoklut_getChannel() channel:255 index:0 nodeId:0

The "Error during initialization" is posted from src/user/sdo/sdoseq.c:993 within the processStateInit2(...) function.

I thought that due to the slow emulation, the timing is a problem. Therefore I also used a custom CDC with Cycle time = 5 000 000 and CN PRestimeout = 1 000 000. But it results in a similar error as with the default CDC.

Any glue or ideas? Let' me know, if you need further information.

Best regards,
Johannes

[AEICOR IIoT Team]

Are you still having this error, it is typically related to the cycle time and response timeouts; but also note that there are also max values of the timeouts in the OD, which affect how large a value you are allowed to set in the CDC (i.e. you also have to edit the max value indices).

[Johannes Fischer]

Hi,
we've been able to fix the error. One one hand, not all network packages were forwarded to the openPOWERLINK stack by our self-written EDRV driver. One the other hand, we switched from CIRCBUF_QUEUE to DIRECT_QUEUE. The CN successfully runs under Genode OS with the Fiasco.OC microkernel now. We are still working on the MN.

Thank you for your reply and input.

[AEICOR IIoT Team]

Hi Johannes, thanks for the update. Good to know. Let me know if you need any support.