myhdl-list

[myhdl-list] Pythonic Test frameworks

[Christopher F. <chr...@gm...>]

Looking for some opinions on methods to make my testbenches more 
Pythonic.  I have been humbled more often than not with my Python 
programming skills, or lack there of (http://bit.ly/uQeRGm).  And my 
testbenches seem to be old-school version of my Verilog/VHDL equivalent 
testbences.  Looking to move from a testbench to a test framework.

Frequently I will create a single testbench and have a bunch of 
testcases in the testbench.  In some cases I will create multiple 
(different) forms in test_ functions to use with py.test.

But too often, majority of my test cases are in this single testbench. 
What I would like to do is leverage the py.test framework but only 
define connections, models, etc *once*.  Using the USB interface project 
(USBP, http://bit.ly/sPEMC1) as an example I used something like the 
following.

~~~ [Example Code] ~~~
class BaseSim()

    def __init__(self, ...):
       # all top-level signal definitions, dut instance and interface
       # model instances
       ...

   def GetGens():
       # return all the generators for

def test_case1():
    bs = BaseSim()

    @instance
    def tb_stimulus():
       yield bs.WriteRegister(0x00, 42)
       rval = [0]
       yield bs.ReadRegister(0x00, val)
       assert rval[0] == 42
       raise StopSimulation

    Simulation((tb_stimulus, bs.GetGens())).run()

def test_case2():
    bs = BaseSim()

    @instance
    def tb_stimulus
       bs.someSignal.next = True
       yield delay(3)
       bs.someSignal.next = False
       # check some condition
       raise StopSimulation

   Simulation((tb_stimulus, bs.GetGens())).run()

~~~ [End Example Code]~~~

Other methods that are more flexible?  Is it worth it to add the 
instantiations and connection signals to a class or just in a module?

Thanks in advance,
Chris

Re: [myhdl-list] Pythonic Test frameworks

[Uri N. <ur...@gm...>]

On 7 November 2011 08:30, Christopher Felton <chr...@gm...> wrote:

> Looking for some opinions on methods to make my testbenches more
> Pythonic.  I have been humbled more often than not with my Python
> programming skills, or lack there of (http://bit.ly/uQeRGm).  And my
> testbenches seem to be old-school version of my Verilog/VHDL equivalent
> testbences.  Looking to move from a testbench to a test framework.
>
> Frequently I will create a single testbench and have a bunch of
> testcases in the testbench.  In some cases I will create multiple
> (different) forms in test_ functions to use with py.test.
>
> But too often, majority of my test cases are in this single testbench.
> What I would like to do is leverage the py.test framework but only
> define connections, models, etc *once*.  Using the USB interface project
> (USBP, http://bit.ly/sPEMC1) as an example I used something like the
> following.
>
> ~~~ [Example Code] ~~~
> class BaseSim()
>
>    def __init__(self, ...):
>       # all top-level signal definitions, dut instance and interface
>       # model instances
>       ...
>
>   def GetGens():
>       # return all the generators for
>
> def test_case1():
>    bs = BaseSim()
>
>    @instance
>    def tb_stimulus():
>       yield bs.WriteRegister(0x00, 42)
>       rval = [0]
>       yield bs.ReadRegister(0x00, val)
>       assert rval[0] == 42
>       raise StopSimulation
>
>    Simulation((tb_stimulus, bs.GetGens())).run()
>
> def test_case2():
>    bs = BaseSim()
>
>    @instance
>    def tb_stimulus
>       bs.someSignal.next = True
>       yield delay(3)
>       bs.someSignal.next = False
>       # check some condition
>       raise StopSimulation
>
>   Simulation((tb_stimulus, bs.GetGens())).run()
>
> ~~~ [End Example Code]~~~
>
> Other methods that are more flexible?  Is it worth it to add the
> instantiations and connection signals to a class or just in a module?
>
> Thanks in advance,
> Chris
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> myhdl-list mailing list
> myh...@li...
> https://lists.sourceforge.net/lists/listinfo/myhdl-list
>

Hi Chris,

  I find the standard lib unittest flexible enough, allowing something like
the example below.
  Any special reason for using py.test?

Cheers,
  Uri

~~~ [Example Code] ~~~
import itertools
import unittest
from myhdl import Signal, always, delay, Simulation

def Clock(clk):
    @always(delay(1))
    def logic():
        clk.next ^= 1
    return logic

def DUT(clk, din, dout):
    @always(clk.posedge)
    def logic():
        dout.next = din.val + 1
    return logic


class TestDUT(unittest.TestCase):
    # unittest methods
    def setUp(self):
        self.clk = Signal(0)
        self.din = Signal(0)
        self.dout = Signal(0)
        self.clk_inst = Clock(self.clk)
        self.dut_inst = DUT(self.clk, self.din, self.dout)

    def tearDown(self):
        pass

    # core test routines
    def exec_test(self, stimulus, ticks):
        sim = Simulation(self.dut_inst, self.clk_inst, stimulus)
        sim.run(ticks)

    def stim_gen(self, clk, gen_data, init_value):
        sig_gen = itertools.count(init_value)
        @always(clk.posedge)
        def logic():
            gen_data.next = sig_gen.next()
        return logic

    # specific test methods (scenarios)
    def test_1(self):
        stim_inst = self.stim_gen(self.clk, self.din, 3)
        self.exec_test(stim_inst, 7)
        self.assertEqual(self.dout.val, 6)

    def test_2(self):
        stim_inst = self.stim_gen(self.clk, self.din, 3)
        self.exec_test(stim_inst, 8)
        self.assertEqual(self.dout.val, 6)


if __name__ == '__main__':
    mytests = unittest.TestLoader().loadTestsFromTestCase(TestDUT)
    unittest.TextTestRunner(verbosity=2).run(mytests)

~~~ [End Example Code]~~~

Re: [myhdl-list] Pythonic Test frameworks

[Christopher F. <chr...@gm...>]

On 11/7/2011 3:50 PM, Uri Nix wrote:
> On 7 November 2011 08:30, Christopher Felton<chr...@gm...>  wrote:
>
>> Looking for some opinions on methods to make my testbenches more
>> Pythonic.  I have been humbled more often than not with my Python
>> programming skills, or lack there of (http://bit.ly/uQeRGm).  And my
>> testbenches seem to be old-school version of my Verilog/VHDL equivalent
>> testbences.  Looking to move from a testbench to a test framework.
>>
>> Frequently I will create a single testbench and have a bunch of
>> testcases in the testbench.  In some cases I will create multiple
>> (different) forms in test_ functions to use with py.test.
>>
>> But too often, majority of my test cases are in this single testbench.
>> What I would like to do is leverage the py.test framework but only
>> define connections, models, etc *once*.  Using the USB interface project
>> (USBP, http://bit.ly/sPEMC1) as an example I used something like the
>> following.
>>
>> ~~~ [Example Code] ~~~
>> class BaseSim()
>>
>>     def __init__(self, ...):
>>        # all top-level signal definitions, dut instance and interface
>>        # model instances
>>        ...
>>
>>    def GetGens():
>>        # return all the generators for
>>
>> def test_case1():
>>     bs = BaseSim()
>>
>>     @instance
>>     def tb_stimulus():
>>        yield bs.WriteRegister(0x00, 42)
>>        rval = [0]
>>        yield bs.ReadRegister(0x00, val)
>>        assert rval[0] == 42
>>        raise StopSimulation
>>
>>     Simulation((tb_stimulus, bs.GetGens())).run()
>>
>> def test_case2():
>>     bs = BaseSim()
>>
>>     @instance
>>     def tb_stimulus
>>        bs.someSignal.next = True
>>        yield delay(3)
>>        bs.someSignal.next = False
>>        # check some condition
>>        raise StopSimulation
>>
>>    Simulation((tb_stimulus, bs.GetGens())).run()
>>
>> ~~~ [End Example Code]~~~
>>
>> Other methods that are more flexible?  Is it worth it to add the
>> instantiations and connection signals to a class or just in a module?
>>
>> Thanks in advance,
>> Chris
>>
>>
>>
>> ------------------------------------------------------------------------------
>> RSA(R) Conference 2012
>> Save $700 by Nov 18
>> Register now
>> http://p.sf.net/sfu/rsa-sfdev2dev1
>> _______________________________________________
>> myhdl-list mailing list
>> myh...@li...
>> https://lists.sourceforge.net/lists/listinfo/myhdl-list
>>
>
> Hi Chris,
>
>    I find the standard lib unittest flexible enough, allowing something like
> the example below.
>    Any special reason for using py.test?

No special reason for the py.test.  I guess I have used py.test because 
you simply create test_ functions and use asserts.  And I have not use 
the unittest as much.  You pointed out the piece I think I was looking 
for.  The idea of a setup!  This is where all the connections etc can be 
defined and defined only once (thanks for the code example).

Searching, py.test has a similar mechanism.  You need to define all the 
test_ functions/methods in a class and use the "setup_method" which is 
equivalent to the setUp in the unittest.

Thanks!
Chris

Re: [myhdl-list] Pythonic Test frameworks

[Bob C. <Fl...@gm...>]

FWIW, py.test seems to be the way MyHDL itself is going...

-BobC


On 11/07/2011 02:15 PM, Christopher Felton wrote:
> On 11/7/2011 3:50 PM, Uri Nix wrote:
>> On 7 November 2011 08:30, Christopher Felton<chr...@gm...>   wrote:
>>
>>> Looking for some opinions on methods to make my testbenches more
>>> Pythonic.  I have been humbled more often than not with my Python
>>> programming skills, or lack there of (http://bit.ly/uQeRGm).  And my
>>> testbenches seem to be old-school version of my Verilog/VHDL equivalent
>>> testbences.  Looking to move from a testbench to a test framework.
>>>
>>> Frequently I will create a single testbench and have a bunch of
>>> testcases in the testbench.  In some cases I will create multiple
>>> (different) forms in test_ functions to use with py.test.
>>>
>>> But too often, majority of my test cases are in this single testbench.
>>> What I would like to do is leverage the py.test framework but only
>>> define connections, models, etc *once*.  Using the USB interface project
>>> (USBP, http://bit.ly/sPEMC1) as an example I used something like the
>>> following.
>>>
>>> ~~~ [Example Code] ~~~
>>> class BaseSim()
>>>
>>>      def __init__(self, ...):
>>>         # all top-level signal definitions, dut instance and interface
>>>         # model instances
>>>         ...
>>>
>>>     def GetGens():
>>>         # return all the generators for
>>>
>>> def test_case1():
>>>      bs = BaseSim()
>>>
>>>      @instance
>>>      def tb_stimulus():
>>>         yield bs.WriteRegister(0x00, 42)
>>>         rval = [0]
>>>         yield bs.ReadRegister(0x00, val)
>>>         assert rval[0] == 42
>>>         raise StopSimulation
>>>
>>>      Simulation((tb_stimulus, bs.GetGens())).run()
>>>
>>> def test_case2():
>>>      bs = BaseSim()
>>>
>>>      @instance
>>>      def tb_stimulus
>>>         bs.someSignal.next = True
>>>         yield delay(3)
>>>         bs.someSignal.next = False
>>>         # check some condition
>>>         raise StopSimulation
>>>
>>>     Simulation((tb_stimulus, bs.GetGens())).run()
>>>
>>> ~~~ [End Example Code]~~~
>>>
>>> Other methods that are more flexible?  Is it worth it to add the
>>> instantiations and connection signals to a class or just in a module?
>>>
>>> Thanks in advance,
>>> Chris
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> RSA(R) Conference 2012
>>> Save $700 by Nov 18
>>> Register now
>>> http://p.sf.net/sfu/rsa-sfdev2dev1
>>> _______________________________________________
>>> myhdl-list mailing list
>>> myh...@li...
>>> https://lists.sourceforge.net/lists/listinfo/myhdl-list
>>>
>> Hi Chris,
>>
>>     I find the standard lib unittest flexible enough, allowing something like
>> the example below.
>>     Any special reason for using py.test?
> No special reason for the py.test.  I guess I have used py.test because
> you simply create test_ functions and use asserts.  And I have not use
> the unittest as much.  You pointed out the piece I think I was looking
> for.  The idea of a setup!  This is where all the connections etc can be
> defined and defined only once (thanks for the code example).
>
> Searching, py.test has a similar mechanism.  You need to define all the
> test_ functions/methods in a class and use the "setup_method" which is
> equivalent to the setUp in the unittest.
>
> Thanks!
> Chris
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> myhdl-list mailing list
> myh...@li...
> https://lists.sourceforge.net/lists/listinfo/myhdl-list
>

Re: [myhdl-list] Pythonic Test frameworks

[Christopher F. <chr...@gm...>]

On 11/7/11 7:21 PM, Bob Cunningham wrote:
> FWIW, py.test seems to be the way MyHDL itself is going...
>
> -BobC

Yeah we have had some brief conversations in the past, that py.test was 
a convenient test framework.  Some of the new tests are written for 
py.test.  But tests are still added under the unittest framework as 
well.  I don't know if it is accurate that MyHDL is moving towards 
py.test and away from unittest.  I think it depends what is being 
tested.  Example, the tests for the modbv (recent addition) were added 
in the unittest area.

I like the py.test but I have little experience with unittest and 
limited with py.test, case in point I was not aware (or forgot) of the 
setup/tear down capabilities.  I don't have enough information at this 
point to suggest one over the other.  It looks like, if you require 
teardown along with setup you will need unittest framework and not py.test.

The following, http://bit.ly/sc3vrI, is a brief mention of py.test used 
instead of unittest (from a couple years ago) from the myhdl-list.

But I do believe py.test is limited.  If you are ok writing with only 
simple asserts, I believe py.test is ok.  But unittest supports more 
advanced features, automatically verify raised exceptions (continue if 
raised fail if not).

I guess an argument against py.test, is that it is another package that 
needs to be installed.  Where as, unittest is part of the standard 
Python library, hmmm.

Regards,
Chris

>
>
> On 11/07/2011 02:15 PM, Christopher Felton wrote:
>> On 11/7/2011 3:50 PM, Uri Nix wrote:
>>> On 7 November 2011 08:30, Christopher Felton<chr...@gm...>    wrote:
>>>
>>>> Looking for some opinions on methods to make my testbenches more
>>>> Pythonic.  I have been humbled more often than not with my Python
>>>> programming skills, or lack there of (http://bit.ly/uQeRGm).  And my
>>>> testbenches seem to be old-school version of my Verilog/VHDL equivalent
>>>> testbences.  Looking to move from a testbench to a test framework.
>>>>
>>>> Frequently I will create a single testbench and have a bunch of
>>>> testcases in the testbench.  In some cases I will create multiple
>>>> (different) forms in test_ functions to use with py.test.
>>>>
>>>> But too often, majority of my test cases are in this single testbench.
>>>> What I would like to do is leverage the py.test framework but only
>>>> define connections, models, etc *once*.  Using the USB interface project
>>>> (USBP, http://bit.ly/sPEMC1) as an example I used something like the
>>>> following.
>>>>
>>>> ~~~ [Example Code] ~~~
>>>> class BaseSim()
>>>>
>>>>       def __init__(self, ...):
>>>>          # all top-level signal definitions, dut instance and interface
>>>>          # model instances
>>>>          ...
>>>>
>>>>      def GetGens():
>>>>          # return all the generators for
>>>>
>>>> def test_case1():
>>>>       bs = BaseSim()
>>>>
>>>>       @instance
>>>>       def tb_stimulus():
>>>>          yield bs.WriteRegister(0x00, 42)
>>>>          rval = [0]
>>>>          yield bs.ReadRegister(0x00, val)
>>>>          assert rval[0] == 42
>>>>          raise StopSimulation
>>>>
>>>>       Simulation((tb_stimulus, bs.GetGens())).run()
>>>>
>>>> def test_case2():
>>>>       bs = BaseSim()
>>>>
>>>>       @instance
>>>>       def tb_stimulus
>>>>          bs.someSignal.next = True
>>>>          yield delay(3)
>>>>          bs.someSignal.next = False
>>>>          # check some condition
>>>>          raise StopSimulation
>>>>
>>>>      Simulation((tb_stimulus, bs.GetGens())).run()
>>>>
>>>> ~~~ [End Example Code]~~~
>>>>
>>>> Other methods that are more flexible?  Is it worth it to add the
>>>> instantiations and connection signals to a class or just in a module?
>>>>
>>>> Thanks in advance,
>>>> Chris
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> RSA(R) Conference 2012
>>>> Save $700 by Nov 18
>>>> Register now
>>>> http://p.sf.net/sfu/rsa-sfdev2dev1
>>>> _______________________________________________
>>>> myhdl-list mailing list
>>>> myh...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/myhdl-list
>>>>
>>> Hi Chris,
>>>
>>>      I find the standard lib unittest flexible enough, allowing something like
>>> the example below.
>>>      Any special reason for using py.test?
>> No special reason for the py.test.  I guess I have used py.test because
>> you simply create test_ functions and use asserts.  And I have not use
>> the unittest as much.  You pointed out the piece I think I was looking
>> for.  The idea of a setup!  This is where all the connections etc can be
>> defined and defined only once (thanks for the code example).
>>
>> Searching, py.test has a similar mechanism.  You need to define all the
>> test_ functions/methods in a class and use the "setup_method" which is
>> equivalent to the setUp in the unittest.
>>
>> Thanks!
>> Chris
>>
>>
>> ------------------------------------------------------------------------------
>> RSA(R) Conference 2012
>> Save $700 by Nov 18
>> Register now
>> http://p.sf.net/sfu/rsa-sfdev2dev1
>> _______________________________________________
>> myhdl-list mailing list
>> myh...@li...
>> https://lists.sourceforge.net/lists/listinfo/myhdl-list
>>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1

Re: [myhdl-list] Pythonic Test frameworks

[Christopher F. <chr...@gm...>]

On 11/8/11 7:04 PM, Bob Cunningham wrote:
> On 11/07/2011 08:07 PM, Christopher Felton wrote:
>> On 11/7/11 7:21 PM, Bob Cunningham wrote:
>>> FWIW, py.test seems to be the way MyHDL itself is going...
>>>
>>> -BobC
>> ...
>>
>> The following, http://bit.ly/sc3vrI, is a brief mention of py.test used
>> instead of unittest (from a couple years ago) from the myhdl-list.
>>
>> But I do believe py.test is limited.  If you are ok writing with only
>> simple asserts, I believe py.test is ok.  But unittest supports more
>> advanced features, automatically verify raised exceptions (continue if
>> raised fail if not).
>>
>> I guess an argument against py.test, is that it is another package that
>> needs to be installed.  Where as, unittest is part of the standard
>> Python library, hmmm.
>>
>> Regards,
>> Chris
>
> Well, MyHDL is, itself, "another package that needs to be installed"!
>
> See the end of this page: http://www.myhdl.org/doku.php/cookbook:stopwatch

Thanks for the reminder, this is a good commentary on py.test versus 
unittest.  This is a good read for anyone trying to determine which test 
framework to use.

>
> And py.test does handle *expected* exceptions: Unexpected exceptions will crash the test with a suitable dump.  See http://pytest.org/latest/getting-started.html#asserting-that-a-certain-exception-is-raised
>

I came across similar information while refreshing on py.test.  It does 
not seem as limited as I first thought.  Also, the setup (and teardown) 
have been enhance.  The "setup_method" is not used anymore but rather 
the funcarg.

> FWIW, py.test is faster and easier, and doesn't much care about the structure of your MyHDL code, but it has limitations.  Unittest is more powerful, but is harder to use.
>
I believe you mean faster to develop, faster to come up to speed.  Not 
necessarily faster execution.

> As a true newbie to both digital design and MyHDL, I've been coding and testing in the smallest increments possible, with tiny blocks glued together.  For this specific development path, py.test seems ideal.  At this point, I have no idea if it will work as well with more complex designs.
>
I think it will scale nicely with complex designs.  But only time and 
experience will prove if it does.

> However, I am quite biased: In my embedded/real-time software systems I often used "programming by contract" (PBC), which is extensively assert-based (to validate the contracts), so py.test seems quite natural to me.  It is also a form of 'white-box' testing.
>
> The complementary approach, test-driven design (TDD), puts more burden on the test harness, which means the test tools must be more powerful and sophisticated.  This is generally a form of 'black-box' testing.
>
> In the software world, the two test methods work well together in practice:  If you use PBC at the low level, the higher-level tests can be simpler.
>
> Has anyone tried to see if py.test and unittest can play nice together (at different levels of abstraction) within a single MyHDL project?
No I have not, I guess in theory you could have both.  I am not sure why 
you would.  I think once you go through the trouble of using unittest 
might as well stay in that framework.  But I guess, if you started with 
py.test, defined a bunch of tests.  Then found something that was not 
supported nicely in py.test, unittest could be added separately.  I 
think they would, essentially, be separate but testing the same entity.

Regards,
Chris

Re: [myhdl-list] Pythonic Test frameworks

[Bob C. <Fl...@gm...>]

On 11/07/2011 08:07 PM, Christopher Felton wrote:
> On 11/7/11 7:21 PM, Bob Cunningham wrote:
>> FWIW, py.test seems to be the way MyHDL itself is going...
>>
>> -BobC
> ...
>
> The following, http://bit.ly/sc3vrI, is a brief mention of py.test used
> instead of unittest (from a couple years ago) from the myhdl-list.
>
> But I do believe py.test is limited.  If you are ok writing with only
> simple asserts, I believe py.test is ok.  But unittest supports more
> advanced features, automatically verify raised exceptions (continue if
> raised fail if not).
>
> I guess an argument against py.test, is that it is another package that
> needs to be installed.  Where as, unittest is part of the standard
> Python library, hmmm.
>
> Regards,
> Chris

Well, MyHDL is, itself, "another package that needs to be installed"!

See the end of this page: http://www.myhdl.org/doku.php/cookbook:stopwatch

And py.test does handle *expected* exceptions: Unexpected exceptions will crash the test with a suitable dump.  See http://pytest.org/latest/getting-started.html#asserting-that-a-certain-exception-is-raised

FWIW, py.test is faster and easier, and doesn't much care about the structure of your MyHDL code, but it has limitations.  Unittest is more powerful, but is harder to use.

As a true newbie to both digital design and MyHDL, I've been coding and testing in the smallest increments possible, with tiny blocks glued together.  For this specific development path, py.test seems ideal.  At this point, I have no idea if it will work as well with more complex designs.

However, I am quite biased: In my embedded/real-time software systems I often used "programming by contract" (PBC), which is extensively assert-based (to validate the contracts), so py.test seems quite natural to me.  It is also a form of 'white-box' testing.

The complementary approach, test-driven design (TDD), puts more burden on the test harness, which means the test tools must be more powerful and sophisticated.  This is generally a form of 'black-box' testing.

In the software world, the two test methods work well together in practice:  If you use PBC at the low level, the higher-level tests can be simpler.

Has anyone tried to see if py.test and unittest can play nice together (at different levels of abstraction) within a single MyHDL project?

-BobC

Re: [myhdl-list] Pythonic Test frameworks

[Christopher F. <chr...@gm...>]

On 11/8/11 9:01 PM, Christopher Felton wrote:
> On 11/8/11 7:04 PM, Bob Cunningham wrote:
>> On 11/07/2011 08:07 PM, Christopher Felton wrote:
>>> On 11/7/11 7:21 PM, Bob Cunningham wrote:
>>>> FWIW, py.test seems to be the way MyHDL itself is going...
>>>>
>>>> -BobC
>>> ...
>>>
>>> The following, http://bit.ly/sc3vrI, is a brief mention of py.test used
>>> instead of unittest (from a couple years ago) from the myhdl-list.
>>>
>>> But I do believe py.test is limited.  If you are ok writing with only
>>> simple asserts, I believe py.test is ok.  But unittest supports more
>>> advanced features, automatically verify raised exceptions (continue if
>>> raised fail if not).
>>>
>>> I guess an argument against py.test, is that it is another package that
>>> needs to be installed.  Where as, unittest is part of the standard
>>> Python library, hmmm.
>>>
>>> Regards,
>>> Chris
>>
>> Well, MyHDL is, itself, "another package that needs to be installed"!
>>
>> See the end of this page: http://www.myhdl.org/doku.php/cookbook:stopwatch
>
> Thanks for the reminder, this is a good commentary on py.test versus
> unittest.  This is a good read for anyone trying to determine which test
> framework to use.

I think this is the important quote from the py.test commentary at the 
end of the stopwatch example.

"""
However, I believe that the benefits are far more important than the 
disadvantages. Moreover, some disadvantages may disappear over time. 
Consequently, I plan to promote py.test as the unit testing framework of 
choice for MyHDL in the future.
"""

I don't think anything has changed since this was posted.

Regards,
Chris

Re: [myhdl-list] Pythonic Test frameworks

[Uri N. <ur...@gm...>]

On 9 November 2011 05:14, Christopher Felton <chr...@gm...> wrote:

> On 11/8/11 9:01 PM, Christopher Felton wrote:
> > On 11/8/11 7:04 PM, Bob Cunningham wrote:
> >> On 11/07/2011 08:07 PM, Christopher Felton wrote:
> >>> On 11/7/11 7:21 PM, Bob Cunningham wrote:
> >>>> FWIW, py.test seems to be the way MyHDL itself is going...
> >>>>
> >>>> -BobC
> >>> ...
> >>
> >>> The following, http://bit.ly/sc3vrI, is a brief mention of py.test
> used
> >>> instead of unittest (from a couple years ago) from the myhdl-list.
> >>>
> >>> But I do believe py.test is limited.  If you are ok writing with only
> >>> simple asserts, I believe py.test is ok.  But unittest supports more
> >>> advanced features, automatically verify raised exceptions (continue if
> >>> raised fail if not).
> >>>
> >>> I guess an argument against py.test, is that it is another package that
> >>> needs to be installed.  Where as, unittest is part of the standard
> >>> Python library, hmmm.
> >>>
> >>> Regards,
> >>> Chris
> >>
> >> Well, MyHDL is, itself, "another package that needs to be installed"!
> >>
> >> See the end of this page:
> http://www.myhdl.org/doku.php/cookbook:stopwatch
> >
> > Thanks for the reminder, this is a good commentary on py.test versus
> > unittest.  This is a good read for anyone trying to determine which test
> > framework to use.
>
> I think this is the important quote from the py.test commentary at the
> end of the stopwatch example.
>
> """
> However, I believe that the benefits are far more important than the
> disadvantages. Moreover, some disadvantages may disappear over time.
> Consequently, I plan to promote py.test as the unit testing framework of
> choice for MyHDL in the future.
> """
>
> I don't think anything has changed since this was posted.
>
> Regards,
> Chris
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> myhdl-list mailing list
> myh...@li...
> https://lists.sourceforge.net/lists/listinfo/myhdl-list
>

Hi,

Since I kind of started this py.test vs unittest discussion, I might as
well add my own take :)
IMO a non-standard library should have some really compelling feature to be
required as a project's dependency, and unittest had fulfills all my needs:
from regression framework for external (black box) testing, to pure Python
and of course myHDL projects.
A beneficial feature of py.test would appear to be its scalability across
multiple processors. A remarkable feature of Nose (another popular
framework, used by numpy) is the ability to generate tests using
generators. Both however are a bit overkill for my purposes.
Regarding the requirement of testing for exceptions - I would recommend
checking the latest enhancements in 2.7, which made unittest even more
comprehensive.

Cheers,
Uri