#2 Create an automatic TestBench

[Carle Cote]

Implementing a standardized testbench within
flowDesigner would help developpers creating unit tests
for blocks they've created. This can greatly enhance
block stability, block reusability and software evolution
(by being able to automatically check if a core
modification brake blocks functionnalities for example)

It can be implemented using many tools :

• Specialized blocks and network for testbench activities
  (load inputs data sets to test, load corresponding
  expected data to be read on outputs, apply data on
  inputs and compares outputs, asserts, log message,
  etc...)
• Scripts that execute unit tests and give tests results
• etc...

[Jean-Marc Valin]

Logged In: YES
user_id=1494

Does the content of data-flow/testcase do the job?
Otherwise, what's missing (outside of "more testcases")?

[Carle Cote]

Logged In: YES
user_id=699938

From what I understand of what's found in data-
flow/testcase, I would say that almost all files are testcases
and that there's a script to launch all tests. Right? Many of
the testcases validation are made by checking that the
application have terminated correctly. This is a first level of
validation but it does'nt verify internal functionnalities
automatically (ex.: the factorial.n test does'nt check if the
output is valid). I would need to open every output files and
validates manually if testcases are giving the expected
behaviors.

What would be interesting, is to creates blocks that would
help building testcases and automating testcase execution.

Ex.:

• A block that would fetch testdata set to inject in a block
  (from preformated files for example)
• A block that would fetch expected output to be read for
  each injected testdata set
• A mechanism that would apply testdata set and compare
  with expected output automatically, and asserts on failure
• etc.

Hope that it helps understand the request!

[Nobody/Anonymous]

Logged In: NO

Actually, there are two types of tests. When a ".out" file
is present, then the script checks that the output is infact
valid. The tests that do not have a .out file are cases
where there is an error in the .n, so what the script does
is check that the error does not cause a segfault. Knowing
that, does it include all the features you want?