Using QUTEST to test on real hardware waiting for an external event

[mricciutelli]

Dear board,

I have read that Qutest is meant to be used as a unit test framework, but since it sports a very broad capability to inject commands and test responses I was wondering if there was a way to employ qutest scripts to launch simple tests on the real hardware of a device.

As an example my goal could be to trigger the mcu to query an external device via script command, and then wait for the response to be then received and processed checking that the appropriate signals transitions are performed.

So to accomplish it, since external hw has certain latency, I was looking for some sort of adjustable timeout feature on the "expect" function to wait for the external device to produce the events that would lead to the intended ao behavior.
I was thinking some combination of expect_pause-continue_test but I am not sure if it is appropriate for this task.
Is there a clean approach to this or am i going outside qutest dsl scope?

Kind regards,
Matteo

[Quantum Leaps]

Hi Matteo,
You are right that QUTest is capable to do much more than just unit tests. In particular, QUTest can be used for integration tests on target hardware. In fact, this capability is already demonstrated by the test in the <qpc|qpcpp>/test/ subdirectory. Please see the README file in that directory. Note the different configuration Q_UTEST=0.

By the nature of stimulus-response QUTest testing strategy, the CUT running on the target must not do anything "by itself". Instead, everything that the target does must be somehow stimulated from the test script. For, example the tests included in qpc/qpcpp are the full QP applications, but with suppressed clock-tick (so that they don't do anything by themselves.) Clock ticks can be then initiated from the scripts. I hope you see the point.

Your requirement for extra long timeouts is somehow unusual and currently there are no provisions for changing the timeout for expectations programmatically from the test script. Such feature could be added (perhaps with a special command expect_within(exp, tout)). So perhaps you could file a feature request for this.

--MMS

[mricciutelli]

Thank you for pointing me into that direction, I understand those tests do not use the qutest port but the actual chosen kernel.
Just a clarification regarding the scenario I mentioned earliar, are you saying that in the case I initiate an action via test script using a signal, the consequences of that signal should not produce the emission of another signal? (ie : a routine to interrogate an external device is triggered by the script signal->the queried peripheral responds->causes an interrupt and that interrupt would trigger a signal to be emitted to an ao)

Matteo

[Quantum Leaps]

Hi Matteo,
The beauty of the QUTest approach is that it doesn't really care what mechanisms are unleashed inside the target after the stimulus provided by the test script. The target can respond with the whole avalanche of events and triggered interrupts. As long as the response is repeatable, the test script can verify the expectations. Just try it!
--MMS

[Giulio Dalla Vecchia]

Hi Miro,

Is simply putting the Python script to sleep for a certain period before checking the expectations from the target enough to achieve what Matteo wants?

[Quantum Leaps]

Hi Giulio,

Is simply putting the Python script to sleep for a certain period before checking the expectations from the target enough to achieve what Matteo wants?

The problem is with choosing this "certain period", so that the QSpy output generated by the test fixture is repeatable. If the delay is one-sided (e.g., the test script must wait for XX milliseconds or longer) than it might work. But if this is a two-sided delay (e.g., more than XX but less than YY milliseconds), then it would be problematic.

Alternatively, the delay might be executed by the test fixgure itself. For example, a precise delay required might be coded inside the QS_onCommand() callback.