Re: [myhdl-list] Pythonic Test frameworks

[Bob C. <Fl...@gm...>]

On 11/07/2011 08:07 PM, Christopher Felton wrote:
> On 11/7/11 7:21 PM, Bob Cunningham wrote:
>> FWIW, py.test seems to be the way MyHDL itself is going...
>>
>> -BobC
> ...
>
> The following, http://bit.ly/sc3vrI, is a brief mention of py.test used
> instead of unittest (from a couple years ago) from the myhdl-list.
>
> But I do believe py.test is limited.  If you are ok writing with only
> simple asserts, I believe py.test is ok.  But unittest supports more
> advanced features, automatically verify raised exceptions (continue if
> raised fail if not).
>
> I guess an argument against py.test, is that it is another package that
> needs to be installed.  Where as, unittest is part of the standard
> Python library, hmmm.
>
> Regards,
> Chris

Well, MyHDL is, itself, "another package that needs to be installed"!

See the end of this page: http://www.myhdl.org/doku.php/cookbook:stopwatch

And py.test does handle *expected* exceptions: Unexpected exceptions will crash the test with a suitable dump.  See http://pytest.org/latest/getting-started.html#asserting-that-a-certain-exception-is-raised

FWIW, py.test is faster and easier, and doesn't much care about the structure of your MyHDL code, but it has limitations.  Unittest is more powerful, but is harder to use.

As a true newbie to both digital design and MyHDL, I've been coding and testing in the smallest increments possible, with tiny blocks glued together.  For this specific development path, py.test seems ideal.  At this point, I have no idea if it will work as well with more complex designs.

However, I am quite biased: In my embedded/real-time software systems I often used "programming by contract" (PBC), which is extensively assert-based (to validate the contracts), so py.test seems quite natural to me.  It is also a form of 'white-box' testing.

The complementary approach, test-driven design (TDD), puts more burden on the test harness, which means the test tools must be more powerful and sophisticated.  This is generally a form of 'black-box' testing.

In the software world, the two test methods work well together in practice:  If you use PBC at the low level, the higher-level tests can be simpler.

Has anyone tried to see if py.test and unittest can play nice together (at different levels of abstraction) within a single MyHDL project?

-BobC