myhdl-list — Mailing list for discussions on MyHDL

Re: [myhdl-list] cpython, pypy, and py.test

[Dan W. <eti...@gm...>]

On Sun, Jun 26, 2011 at 1:52 PM, Christopher Felton
<chr...@gm...> wrote:
> Does anyone have a good method to use either, cpython or pypy, with
> py.test?


Jan's speed post came about just as I was testing some HDL for an
upcoming mixed-signal ASIC, here's the hack-way I've done it:
(requires the extra pytest-xdist plugin)

py.test --dist=load --tx 4*popen//python=pypy
and half the processes when using cosimulation

It's been a killer combination for long-running tests, especially when
using Rackspace instances to keep my main machine free while they run.
 For some reason, running as a module under pypy fails if there are
extra cmd line arguments related to xdist, hence the above
work-around:

"python -m pytest foo.py" and
"python -m pytest -n4 foo.py" and
"pypy -m pytest -v foo.py" work properly, but any options fail with e.g.:
"pypy -m pytest -n4 foo.py"

tox would be better for more formal multi-platform release stuff, though.

HTH,

Dan


-- 
SDG
www.whiteaudio.com

Re: [myhdl-list] cpython, pypy, and py.test

[Christopher F. <chr...@gm...>]

On 6/26/11 4:26 PM, Martín Gaitán wrote:
> On Sun, Jun 26, 2011 at 3:52 PM, Christopher Felton
> <chr...@gm...>  wrote:
>> Does anyone have a good method to use either, cpython or pypy, with
>> py.test?
>>
>> When running test suites built around py.test it would be nice to
>> indicate which interpreter to use.
>>
>> Couple issues I ran into:
>>
>>   1.  Installing py.test requires easy_install (or pip,
>>       no source install available?) and I did not see how
>>       to indicate which python installation to use with easy_install?
>>       (So that the py.test framework is installed for both versions)
>>
>>   2.  Would you simply modify the py.test top script to indicate which
>>       python interpreter to use?
>>
>> Regards,
>> Chris
>
> I told few days ago [0], this is what Tox [1] is for
>
> [1] http://tox.readthedocs.org
>
> [0] http://article.gmane.org/gmane.comp.python.myhdl/1997/match=tox
>

Thanks, I guess I missed tox usage or forget.  D'oh, thanks for the 
reminder.

Chris

Re: [myhdl-list] cpython, pypy, and py.test

[Martín G. <ga...@gm...>]

On Sun, Jun 26, 2011 at 3:52 PM, Christopher Felton
<chr...@gm...> wrote:
> Does anyone have a good method to use either, cpython or pypy, with
> py.test?
>
> When running test suites built around py.test it would be nice to
> indicate which interpreter to use.
>
> Couple issues I ran into:
>
>  1.  Installing py.test requires easy_install (or pip,
>      no source install available?) and I did not see how
>      to indicate which python installation to use with easy_install?
>      (So that the py.test framework is installed for both versions)
>
>  2.  Would you simply modify the py.test top script to indicate which
>      python interpreter to use?
>
> Regards,
> Chris

I told few days ago [0], this is what Tox [1] is for

[1] http://tox.readthedocs.org

[0] http://article.gmane.org/gmane.comp.python.myhdl/1997/match=tox

[myhdl-list] cpython, pypy, and py.test

[Christopher F. <chr...@gm...>]

Does anyone have a good method to use either, cpython or pypy, with 
py.test?

When running test suites built around py.test it would be nice to 
indicate which interpreter to use.

Couple issues I ran into:

  1.  Installing py.test requires easy_install (or pip,
      no source install available?) and I did not see how
      to indicate which python installation to use with easy_install?
      (So that the py.test framework is installed for both versions)

  2.  Would you simply modify the py.test top script to indicate which
      python interpreter to use?

Regards,
Chris

Re: [myhdl-list] Signal has multiple drivers - Solution ?

[Christopher F. <chr...@gm...>]

On 6/16/2011 7:46 AM, Sadique Sheik wrote:
> Hi Benoit,
>
> Thanks for the quick reply. It was very informative.
>
> I still have a few quick queries though.
>>
>> The easiest sample of such an arbiter is a multiplexer:
>>
>> def mux2(addr1, addr2, sel, addr):
>>    @always_comb:
>>    def muxLogic():
>>      addr.next = addr1
>>      if sel == 1:
>>        addr.next = addr2
>>    return muxLogic
>
> The multiplexer doesn't seem to solve the problem, atleast the way i see
> it. It seems that now instead of using the 'addr' signal i will endup
> driving the 'sel' signal by multiple components/locations.
>

The multiplexer solves the specific problem of multiple drivers.  It 
might not solve the *design* issue.  Per the description thus far, you 
only have one communication channel to the RAM.  You need to decide how 
this resource (RAM addr and data) are shared.  What are the "rules" for 
sharing.

As Benoit suggested, you might need some sort of arbitration (you can 
look at the wishbone documentations, or other on-chip buses for some 
examples).  A Dual-Port RAM might be appropriate in your case, if you 
have a separate writer and reader.

In general, once you determine the rules for sharing, the implementation 
can be a mux, a bunch of ORs (or ANDs).  As you read, tri-states are 
usually avoided for on-chip bus sharing.

Chris

Re: [myhdl-list] Signal has multiple drivers - Solution ?

[Sadique S. <sa...@in...>]

Hi Benoit,

Thanks for the quick reply. It was very informative.

I still have a few quick queries though.
> 
> The easiest sample of such an arbiter is a multiplexer:
> 
> def mux2(addr1, addr2, sel, addr):
>   @always_comb:
>   def muxLogic():
>     addr.next = addr1
>     if sel == 1:
>       addr.next = addr2
>   return muxLogic

The multiplexer doesn't seem to solve the problem, atleast the way i see
it. It seems that now instead of using the 'addr' signal i will endup
driving the 'sel' signal by multiple components/locations.

In the case of only two components, its alright because you can use one
of the components as a master and the other as a slave that relies on
the other component to set the correct select signal.

Did i get that right ? Or did i miss something very crucial .. 


> In the case of your RAM, a common seen practice is to have two `raddr`
> (read address) input in you module and two associated `dout` (data
> output) allowing two simultaneous read to happen from different
> modules.
This sounds good to me. I am going to go ahead and use it for my
simulations.

Although, I wonder how things are gona work if there are multiple (more
than 2) components, wont that just increase the number of ports ?

May be i should do some proper literature reading and get my basics
right.. 

Re: [myhdl-list] Signal has multiple drivers - Solution ?

[Ben <ben...@gm...>]

On Thu, Jun 16, 2011 at 11:57, Sadique Sheik <sa...@in...> wrote:
> Hi,
>
> I am absolutely new to HDL (just started 3 weeks ago!).
>
> I started with some reading on VHDL but fell in love with MyHDL.
> I have been trying to do some basic simulations where i use a simple
> RAM. I basically copied it from the documentation.
> http://www.myhdl.org/doc/0.6/manual/conversion_examples.html#ram-inference
>
> Now in my simulation i use the addr signal in more than one location to
> read from the ram. While simulating, everything works as expected. But
> when i try conversion to VHDL it complains :
>
> Signal has multiple drivers: addr
>

What he means there, is that in your current design, multiple source
can set the value of the `addr` signal. While this may go well in your
simulation process (for instance, because you care that only one
source at a time tries to set it), the convertor cannot make sure that
it will always work and for this reason ask you to reduce the number
of soures to 1.

> A bit of googling on this issue tells me that I should either set the
> signal to "tristate" while i am not using it or use a "resolution
> function."
>
> A quick search in MyHDL documentation tells me that tristate thing is
> not a very nice thing to do. Can some one tell me how this is usually
> handled ?
>
> Can I get an example on how to define and use a "resolution function".

A resolution function, sometimes also called `arbiter`, will care
about setting the value of the `addr` signal. Thus he will be the only
one writing to this signal. The input of this module will consists of
all the whished `addr` values from all the modules trying to set it,
*and* a synchronisation mechanism that will instruct the arbiter who
tries to set the `addr` value. Those might go from very simple to
quite elaborated, depending on your design.

The easiest sample of such an arbiter is a multiplexer:

def mux2(addr1, addr2, sel, addr):
  @always_comb:
  def muxLogic():
    addr.next = addr1
    if sel == 1:
      addr.next = addr2
  return muxLogic

But as said, such module can become really complicated.

In the case of your RAM, a common seen practice is to have two `raddr`
(read address) input in you module and two associated `dout` (data
output) allowing two simultaneous read to happen from different
modules.

Hope this helps.

Regards
Benoît.

[myhdl-list] Signal has multiple drivers - Solution ?

[Sadique S. <sa...@in...>]

Hi, 

I am absolutely new to HDL (just started 3 weeks ago!). 

I started with some reading on VHDL but fell in love with MyHDL.
I have been trying to do some basic simulations where i use a simple
RAM. I basically copied it from the documentation. 
http://www.myhdl.org/doc/0.6/manual/conversion_examples.html#ram-inference

Now in my simulation i use the addr signal in more than one location to
read from the ram. While simulating, everything works as expected. But
when i try conversion to VHDL it complains :

Signal has multiple drivers: addr

A bit of googling on this issue tells me that I should either set the
signal to "tristate" while i am not using it or use a "resolution
function."

A quick search in MyHDL documentation tells me that tristate thing is
not a very nice thing to do. Can some one tell me how this is usually
handled ? 

Can I get an example on how to define and use a "resolution function".

Thanks a lot
Sadique

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Jan D. <ja...@ja...>]

Sounds like we are changing too many things at the same time  :-)

1. I wouldn't worry about non up-to-date nightly builds.
PyPy is in heavy test-driven development.

2. The bug I found could easily be described in words, which
I did on the irc channel. Armin Rigo fixed it before I could
enter it into the bug tracker :-)

3. there is no way I can explain that things would work
with traceSignals but not without. However, in your stack
trace I notice a reference to the old, deprecated compiler
package. Seems a really old version of MyHDL. Since 0.7,
we use the supported ast package. Perhaps the deprecated
compiler package was changed in 2.6 - but if so we will
not fix related problems in MyHDL.

Jan


On 06/10/2011 12:46 AM, Ben wrote:
>
> On Jun 9, 2011, at 11:32 PM, Jan Decaluwe wrote:
>
>> On 06/09/2011 10:56 PM, Ben wrote:
>>>
>>> On Jun 9, 2011, at 1:11 PM, Christopher Felton wrote:
>>>>
>>>> The relevant information here is the ~2x increase and ~8x increase
>>>> with pypy.
>>>
>>> Just to mention my testsuite went from 83.213s to 22.339s. So that's almost a 4x gain.
>>
>> Is this a suite that consists of a number of tests? Do you see
>> the largest gains with those tests that run for the longest time,
>> as can be expected from the JIT compiler?
>
> Yes, those are 23 tests. I just don't really know which one could be big enough to be interesting ... It's been more than a year I looked at that project ...
>
> So I thought I could try to run a big Simulation on the same project ... Knowing that it previously failed with the 1.5 release of pypy (traceback including traceSignal), I downloaded the latest nightly build I could find for my platform (http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-44082-2650f0a670c0-osx64.tar.bz2) unfortunately, it looks like it is something like 800 revisions behind the tip ...
>
> I did not got the traceback with traceSignals though. So I don't really know If I found a new bug, If I get a temporary bug that was present at this moment in time or something else.
>
> Here is my stack trace (it doesn't looks good though):
>
> [bogus _immutable_field_ declaration:<GcPtrFieldDescr pypy.interpreter.pyopcode.FrameBlock.inst_previous 16>]
> RPython traceback:
>    File "implement.c", line 1656, in entry_point
>    File "implement_7.c", line 31942, in PyFrame_execute_frame
>    File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
>    File "implement_22.c", line 4090, in portal_1
>    File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
>    File "implement_30.c", line 37827, in dispatch_bytecode__AccessDirect_None
>    File "implement_33.c", line 60311, in call_function__AccessDirect_None
>    File "implement_7.c", line 31942, in PyFrame_execute_frame
>    File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
>    File "implement_22.c", line 4090, in portal_1
>    File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
>    File "implement_30.c", line 37440, in dispatch_bytecode__AccessDirect_None
>    File "implement_33.c", line 60311, in call_function__AccessDirect_None
>    File "implement_7.c", line 31942, in PyFrame_execute_frame
>    File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
>    File "implement_22.c", line 4090, in portal_1
>    File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
>    File "implement_30.c", line 38583, in dispatch_bytecode__AccessDirect_None
>    File "implement_33.c", line 60311, in call_function__AccessDirect_None
>    File "implement_7.c", line 31942, in PyFrame_execute_frame
>    File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
>    File "implement_22.c", line 4090, in portal_1
>    File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
>    File "implement_30.c", line 38476, in dispatch_bytecode__AccessDirect_None
>    File "implement_34.c", line 785, in EXEC_STMT__AccessDirect_None
>    File "implement_7.c", line 31942, in PyFrame_execute_frame
>    File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
>    File "implement_22.c", line 4090, in portal_1
>    File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
>    File "implement_30.c", line 40399, in dispatch_bytecode__AccessDirect_None
>    File "implement_34.c", line 8061, in CALL_METHOD__AccessDirect_star_1
>    File "implement_7.c", line 31942, in PyFrame_execute_frame
>    File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
>    File "implement_22.c", line 4090, in portal_1
>    File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
>    File "implement_33.c", line 53338, in jump_absolute__AccessDirect_None
>    File "implement_28.c", line 4715, in maybe_compile_and_run__star_5_1
>    File "implement_30.c", line 33036, in compile_and_run_once___pypy_jit_metainterp_jitdr_1
>    File "implement_30.c", line 46244, in MetaInterp__compile_and_run_once
>    File "implement_34.c", line 26936, in MetaInterp_interpret
>    File "implement_37.c", line 6223, in MetaInterp__interpret
>    File "implement_39.c", line 40711, in MIFrame_run_one_step
>    File "implement_52.c", line 63116, in MIFrame_opimpl_jit_merge_point
>    File "implement_57.c", line 46696, in MetaInterp_reached_loop_header
>    File "implement_61.c", line 5901, in MetaInterp_compile_bridge
>    File "implement_61.c", line 16186, in compile_new_bridge
>    File "implement_64.c", line 64874, in optimize_bridge
>    File "implement_69.c", line 61909, in _optimize_bridge
>    File "implement_76.c", line 10608, in Optimizer_propagate_all_forward
>    File "implement_79.c", line 55511, in OptHeap_optimize_SETFIELD_GC
> Fatal RPython error: BogusPureField
> Abort trap
>
> Jan, where did you reported your bug ?
>
> Pushing it a bit further, i removed the traceSignals ...
>
> I know have the following trouble: cPython complains with the following traceback: (I had no trouble with the traceSignals though ...)
>
> Traceback (most recent call last):
>    File "traceProc.py", line 187, in<module>
>      tb = traceBench()#traceSignals(traceBench)
>    File "traceProc.py", line 67, in traceBench
>      @instance
>    File "/Library/Python/2.6/site-packages/myhdl/_instance.py", line 42, in instance
>      return _Instantiator(genFunc)
>    File "/Library/Python/2.6/site-packages/myhdl/_instance.py", line 49, in __init__
>      self.waiter = _inferWaiter(self.gen)
>    File "/Library/Python/2.6/site-packages/myhdl/_Waiter.py", line 201, in _inferWaiter
>      ast = compiler.parse(s)
>    File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/compiler/transformer.py", line 51, in parse
>      return Transformer().parsesuite(buf)
>    File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/compiler/transformer.py", line 128, in parsesuite
>      return self.transform(parser.suite(text))
>    File "<string>", line 25
>      print "*-->\t%s\t
>
> Did I found a bug in python2.6 ?
>
> Anyway, further removing those annoying print statements, pypy1.5 get two times **slower** than cPython ...
>
> Not a really conclusive test though ...
>
> Regards
> Benoît
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev


-- 
Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
     Python as a HDL: http://www.myhdl.org
     VHDL development, the modern way: http://www.sigasi.com
     Analog design automation: http://www.mephisto-da.com
     World-class digital design: http://www.easics.com

Re: [myhdl-list] Conversion Example Error in 0.7 Manual

[Martín G. <ga...@gm...>]

2011/6/9 Christopher Felton <chr...@gm...>

> On 6/9/2011 4:29 PM, Jan Decaluwe wrote:
> > On 06/09/2011 12:59 PM, Christopher Felton wrote:
> >> A new user to MyHDL was trying to convert one of the examples in the
> >> manual on pages 67-68 in the pdf version for release 0.7
> >> (conversion_examples.rst)
> >> http://www.myhdl.org/doc/current/manual/conversion_examples.html.
> >>
> >> The example failed to convert for a "output port read internally" error.
> >>     This error only appears to occur with the 0.7 release.  This might
> be
> >> a known issue.
> >>
> >> With the latest hg repo, 0.8dev, the error does not occur but rather a
> >> warning.
> >
> > Are you sure? I quickly investigated the repo, and it seems that
> > this was always a warning. At some point however, there was no
> > warning and the VHDL would be incorrect because it generated
> > an 'out' instead of an 'inout'. But that's quite some time ago.
> >
> > Jan
>

Hi Jan,

btw,
Lastly I've been using tox [1] which is a great tool to automatize the test
running in different scenarios (py26, py27, pypy, etc) , and it's easily
integrable with sphinx doc testing [2] . I think it would be useful for
myhdl's dev process.

[1] http://tox.testrun.org/en/latest/
[2]
http://tox.testrun.org/en/latest/example/general.html#integrating-sphinx-documentation-checks

Re: [myhdl-list] Conversion Example Error in 0.7 Manual

[Christopher F. <chr...@gm...>]

On 6/9/2011 4:29 PM, Jan Decaluwe wrote:
> On 06/09/2011 12:59 PM, Christopher Felton wrote:
>> A new user to MyHDL was trying to convert one of the examples in the
>> manual on pages 67-68 in the pdf version for release 0.7
>> (conversion_examples.rst)
>> http://www.myhdl.org/doc/current/manual/conversion_examples.html.
>>
>> The example failed to convert for a "output port read internally" error.
>>     This error only appears to occur with the 0.7 release.  This might be
>> a known issue.
>>
>> With the latest hg repo, 0.8dev, the error does not occur but rather a
>> warning.
>
> Are you sure? I quickly investigated the repo, and it seems that
> this was always a warning. At some point however, there was no
> warning and the VHDL would be incorrect because it generated
> an 'out' instead of an 'inout'. But that's quite some time ago.
>
> Jan
>

Nope, not sure.  Looking at it again I am in error.  It was something 
much more basic.  When the acquaintance copied the example from the pdf 
version of the document,

(from page 67-68 of:
    http://www.myhdl.org/lib/exe/fetch.php/doc:myhdl.pdf),

his copy started at the def and did not include the ACTIVE_LOW ... line. 
  When I quickly (ahh, there's the issue, quickly) read his stack trace 
I combined the warning with all the other stuff and jumped to the wrong 
conclusion.  All this was in an IM (chat) so when I sent a modified 
version back to test I didn't even notice I copied it correctly where as 
he copied it incorrectly.

In summary, there is no error in the example, just the possibility to 
cut-n-paste incorrectly :)  Sorry for the distraction, thanks for checking.

Chris

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Ben <ben...@gm...>]

On Jun 9, 2011, at 11:32 PM, Jan Decaluwe wrote:

> On 06/09/2011 10:56 PM, Ben wrote:
>> 
>> On Jun 9, 2011, at 1:11 PM, Christopher Felton wrote:
>>> 
>>> The relevant information here is the ~2x increase and ~8x increase
>>> with pypy.
>> 
>> Just to mention my testsuite went from 83.213s to 22.339s. So that's almost a 4x gain.
> 
> Is this a suite that consists of a number of tests? Do you see
> the largest gains with those tests that run for the longest time,
> as can be expected from the JIT compiler?

Yes, those are 23 tests. I just don't really know which one could be big enough to be interesting ... It's been more than a year I looked at that project ...

So I thought I could try to run a big Simulation on the same project ... Knowing that it previously failed with the 1.5 release of pypy (traceback including traceSignal), I downloaded the latest nightly build I could find for my platform (http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-44082-2650f0a670c0-osx64.tar.bz2) unfortunately, it looks like it is something like 800 revisions behind the tip ...

I did not got the traceback with traceSignals though. So I don't really know If I found a new bug, If I get a temporary bug that was present at this moment in time or something else.

Here is my stack trace (it doesn't looks good though):

[bogus _immutable_field_ declaration: <GcPtrFieldDescr pypy.interpreter.pyopcode.FrameBlock.inst_previous 16>]
RPython traceback:
  File "implement.c", line 1656, in entry_point
  File "implement_7.c", line 31942, in PyFrame_execute_frame
  File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "implement_22.c", line 4090, in portal_1
  File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
  File "implement_30.c", line 37827, in dispatch_bytecode__AccessDirect_None
  File "implement_33.c", line 60311, in call_function__AccessDirect_None
  File "implement_7.c", line 31942, in PyFrame_execute_frame
  File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "implement_22.c", line 4090, in portal_1
  File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
  File "implement_30.c", line 37440, in dispatch_bytecode__AccessDirect_None
  File "implement_33.c", line 60311, in call_function__AccessDirect_None
  File "implement_7.c", line 31942, in PyFrame_execute_frame
  File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "implement_22.c", line 4090, in portal_1
  File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
  File "implement_30.c", line 38583, in dispatch_bytecode__AccessDirect_None
  File "implement_33.c", line 60311, in call_function__AccessDirect_None
  File "implement_7.c", line 31942, in PyFrame_execute_frame
  File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "implement_22.c", line 4090, in portal_1
  File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
  File "implement_30.c", line 38476, in dispatch_bytecode__AccessDirect_None
  File "implement_34.c", line 785, in EXEC_STMT__AccessDirect_None
  File "implement_7.c", line 31942, in PyFrame_execute_frame
  File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "implement_22.c", line 4090, in portal_1
  File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
  File "implement_30.c", line 40399, in dispatch_bytecode__AccessDirect_None
  File "implement_34.c", line 8061, in CALL_METHOD__AccessDirect_star_1
  File "implement_7.c", line 31942, in PyFrame_execute_frame
  File "implement_18.c", line 51028, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "implement_22.c", line 4090, in portal_1
  File "implement_28.c", line 5233, in handle_bytecode__AccessDirect_None
  File "implement_33.c", line 53338, in jump_absolute__AccessDirect_None
  File "implement_28.c", line 4715, in maybe_compile_and_run__star_5_1
  File "implement_30.c", line 33036, in compile_and_run_once___pypy_jit_metainterp_jitdr_1
  File "implement_30.c", line 46244, in MetaInterp__compile_and_run_once
  File "implement_34.c", line 26936, in MetaInterp_interpret
  File "implement_37.c", line 6223, in MetaInterp__interpret
  File "implement_39.c", line 40711, in MIFrame_run_one_step
  File "implement_52.c", line 63116, in MIFrame_opimpl_jit_merge_point
  File "implement_57.c", line 46696, in MetaInterp_reached_loop_header
  File "implement_61.c", line 5901, in MetaInterp_compile_bridge
  File "implement_61.c", line 16186, in compile_new_bridge
  File "implement_64.c", line 64874, in optimize_bridge
  File "implement_69.c", line 61909, in _optimize_bridge
  File "implement_76.c", line 10608, in Optimizer_propagate_all_forward
  File "implement_79.c", line 55511, in OptHeap_optimize_SETFIELD_GC
Fatal RPython error: BogusPureField
Abort trap

Jan, where did you reported your bug ?

Pushing it a bit further, i removed the traceSignals ... 

I know have the following trouble: cPython complains with the following traceback: (I had no trouble with the traceSignals though ...)

Traceback (most recent call last):
  File "traceProc.py", line 187, in <module>
    tb = traceBench()#traceSignals(traceBench)
  File "traceProc.py", line 67, in traceBench
    @instance
  File "/Library/Python/2.6/site-packages/myhdl/_instance.py", line 42, in instance
    return _Instantiator(genFunc)
  File "/Library/Python/2.6/site-packages/myhdl/_instance.py", line 49, in __init__
    self.waiter = _inferWaiter(self.gen)
  File "/Library/Python/2.6/site-packages/myhdl/_Waiter.py", line 201, in _inferWaiter
    ast = compiler.parse(s)
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/compiler/transformer.py", line 51, in parse
    return Transformer().parsesuite(buf)
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/compiler/transformer.py", line 128, in parsesuite
    return self.transform(parser.suite(text))
  File "<string>", line 25
    print "*-->\t%s\t

Did I found a bug in python2.6 ?

Anyway, further removing those annoying print statements, pypy1.5 get two times **slower** than cPython ...

Not a really conclusive test though ...

Regards
Benoît

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Jan D. <ja...@ja...>]

On 06/09/2011 10:56 PM, Ben wrote:
>
> On Jun 9, 2011, at 1:11 PM, Christopher Felton wrote:
>>
>> The relevant information here is the ~2x increase and ~8x increase
>> with pypy.
>
> Just to mention my testsuite went from 83.213s to 22.339s. So that's almost a 4x gain.

Is this a suite that consists of a number of tests? Do you see
the largest gains with those tests that run for the longest time,
as can be expected from the JIT compiler?

Jan

-- 
Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
     Python as a HDL: http://www.myhdl.org
     VHDL development, the modern way: http://www.sigasi.com
     Analog design automation: http://www.mephisto-da.com
     World-class digital design: http://www.easics.com

Re: [myhdl-list] Conversion Example Error in 0.7 Manual

[Jan D. <ja...@ja...>]

On 06/09/2011 12:59 PM, Christopher Felton wrote:
> A new user to MyHDL was trying to convert one of the examples in the
> manual on pages 67-68 in the pdf version for release 0.7
> (conversion_examples.rst)
> http://www.myhdl.org/doc/current/manual/conversion_examples.html.
>
> The example failed to convert for a "output port read internally" error.
>    This error only appears to occur with the 0.7 release.  This might be
> a known issue.
>
> With the latest hg repo, 0.8dev, the error does not occur but rather a
> warning.

Are you sure? I quickly investigated the repo, and it seems that
this was always a warning. At some point however, there was no
warning and the VHDL would be incorrect because it generated
an 'out' instead of an 'inout'. But that's quite some time ago.

Jan

-- 
Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
     Python as a HDL: http://www.myhdl.org
     VHDL development, the modern way: http://www.sigasi.com
     Analog design automation: http://www.mephisto-da.com
     World-class digital design: http://www.easics.com

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Ben <ben...@gm...>]

On Jun 9, 2011, at 1:11 PM, Christopher Felton wrote:
> 
> The relevant information here is the ~2x increase and ~8x increase
> with pypy.

Just to mention my testsuite went from 83.213s to 22.339s. So that's almost a 4x gain.

Great work to all who allowed that to happen !

Benoît

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Jan D. <ja...@ja...>]

On 06/09/2011 06:38 PM, Angel Ezquerra wrote:
> On Thursday, June 9, 2011, Jan Decaluwe<ja...@ja...>  wrote:
>> On 06/09/2011 05:30 PM, Angel Ezquerra wrote:
>>
>>> Jan, any idea why calling traceSignals would fail?
>>
>> Actually yes, for the same reason why conversion doesn't
>> work with PyPy 1.5. The hierarchy extractor reuses the
>> Python profiler, and there is a bug in PyPy with it. I had found
>> that while trying conversion, and reported it. It's
>> fixed in PyPy development. I will add this info to the
>> page on myhdl.org.
>>
>> I see our experiments as a preparation for PyPy 1.5.1
>> when hopefully everything will work (don't know about numpy)
>> with perhaps some additional speed boost.
>>
>> Also, now that it was worth the trouble, I have rewritten
>> some parts of the Signal handling considerably in 0.8-dev,
>> with a significant speed boost (independent of PyPy) for some
>> simulation. Also worth trying out.
>>
>> Jan
>
> Cool!
>
> Does that mean that with ploy 1.5.1 both tracesignals and conversion will work?

My working hypothesis is that with PyPy 1.5.1, MyHDL will work exactly
as with cPython.

Jan

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Angel E. <ezq...@gm...>]

On Thursday, June 9, 2011, Jan Decaluwe <ja...@ja...> wrote:
> On 06/09/2011 05:30 PM, Angel Ezquerra wrote:
>
>> Jan, any idea why calling traceSignals would fail?
>
> Actually yes, for the same reason why conversion doesn't
> work with PyPy 1.5. The hierarchy extractor reuses the
> Python profiler, and there is a bug in PyPy with it. I had found
> that while trying conversion, and reported it. It's
> fixed in PyPy development. I will add this info to the
> page on myhdl.org.
>
> I see our experiments as a preparation for PyPy 1.5.1
> when hopefully everything will work (don't know about numpy)
> with perhaps some additional speed boost.
>
> Also, now that it was worth the trouble, I have rewritten
> some parts of the Signal handling considerably in 0.8-dev,
> with a significant speed boost (independent of PyPy) for some
> simulation. Also worth trying out.
>
> Jan

Cool!

Does that mean that with ploy 1.5.1 both tracesignals and conversion will work?

Angel

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Jan D. <ja...@ja...>]

On 06/09/2011 05:30 PM, Angel Ezquerra wrote:

> Jan, any idea why calling traceSignals would fail?

Actually yes, for the same reason why conversion doesn't
work with PyPy 1.5. The hierarchy extractor reuses the
Python profiler, and there is a bug in PyPy with it. I had found
that while trying conversion, and reported it. It's
fixed in PyPy development. I will add this info to the
page on myhdl.org.

I see our experiments as a preparation for PyPy 1.5.1
when hopefully everything will work (don't know about numpy)
with perhaps some additional speed boost.

Also, now that it was worth the trouble, I have rewritten
some parts of the Signal handling considerably in 0.8-dev,
with a significant speed boost (independent of PyPy) for some
simulation. Also worth trying out.

Jan




-- 
Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
     Python as a HDL: http://www.myhdl.org
     VHDL development, the modern way: http://www.sigasi.com
     Analog design automation: http://www.mephisto-da.com
     World-class digital design: http://www.easics.com

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Angel E. <ezq...@gm...>]

On Thursday, June 9, 2011, Christopher Felton <chr...@gm...> wrote:
> <snip>
>>
>>
>> Simulations
>> ===========
>> As discussed, the first couple simulations I tried failed because of the
>> unsupported dependencies.  Which required some investigation on the
>> failures and possible solutions.  At this time there is no solution for
>> the numpy and company other than rewriting the simulations, not to use
>> numpy.  The following is the list of my projects I attempted to run with
>> pypy.  Most of these are described on the myhdl wiki.
>>
>>     1. Recursive FFT (failed, numpy)
>>     2. GnuRadio Compatible USRP CIC Filter (failed, numpy)
>>     3. Fixed-Point SOS (failed, numpy)
>>     4. USBP (FX2 USB Interface) (failed in myhdl??)
>>
>> All in all, I could not get any of them to run with pypy?  Some were
>> expected but some were not.
>>
>>
>
> Ok, I *learnt* something since yesterday.  First, pypy will not work
> on my PPC Mac.  I should have realized this right away, the JIT is
> for x86 only (correct?).  But the surprising thing is that python
> is crazy slow on my PPC-Mac (circa 2003).  I am running py27 with
> the latest trunk myhdl.  py27 installed with macports (port install)
> To get another system in the mix I stole my wifes laptop which can
> run pypy (side note, how quickly things can be setup with MyHDL/Python).
>
> Also, my non-numpy simulations, yesterday, were failing.
> They failed, because I was using the traceSignals (?).  When I
> removed the traceSignals function I was able to run two of the
> simulations with pypy :)
>
> Below is a table of execution times for the different simulations.
>
> +---------+---------+----------+-----------+----------+----------+
> |         | OSX-CPy | OSXi-CPy | OSXi-pypy | Ubu-CPy  | Ubu-pypy |
> +---------+=========+==========+===========+==========+==========+
> | rFFT    | 10m.52s | 4m.6s    | failed*   | 4m.36s   | failed*  |
> +---------+---------+----------+-----------+----------+----------+
> | CIC     | tbr     | tbr      | failed*   | tbr      | failed*  |
> +---------+---------+----------+-----------+----------+----------+
> | USBP    | tbr     | 4m.30s   | 2m.20s    | 4m.50s   | 2m.28s   |
> +---------+---------+----------+-----------+----------+----------+
> | LED     | 30m.4s  | 15m.15s  | 2m.10s    | 16m.13s  | 1m.59s   |
> +---------+---------+----------+-----------+----------+----------+
> * failed due to the numpy dependency
>
> System configurations
> OSX  - Dual 2GHz PPC G5, 4.5GiB RAM, OS X 10.5.8
> OSXi - 2.26GHz Intel Core 2 Duo, 2GiB RAM, OS X 10.6.7
> Ubu  - AMD Athlon LE1620 1.7GiB RAM, Ubuntu (natty)
>
> The relevant information here is the ~2x increase and ~8x increase
> with pypy.
>
> Hope this is useful.
>
> Chris Felton

Awesome, thanks for the info. Sonit seems that when it works the
speedup is significant!

Jan, any idea why calling traceSignals would fail?

Angel

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Christopher F. <chr...@gm...>]

<snip>
>
>
> Simulations
> ===========
> As discussed, the first couple simulations I tried failed because of the
> unsupported dependencies.  Which required some investigation on the
> failures and possible solutions.  At this time there is no solution for
> the numpy and company other than rewriting the simulations, not to use
> numpy.  The following is the list of my projects I attempted to run with
> pypy.  Most of these are described on the myhdl wiki.
>
>     1. Recursive FFT (failed, numpy)
>     2. GnuRadio Compatible USRP CIC Filter (failed, numpy)
>     3. Fixed-Point SOS (failed, numpy)
>     4. USBP (FX2 USB Interface) (failed in myhdl??)
>
> All in all, I could not get any of them to run with pypy?  Some were
> expected but some were not.
>
>

Ok, I *learnt* something since yesterday.  First, pypy will not work
on my PPC Mac.  I should have realized this right away, the JIT is
for x86 only (correct?).  But the surprising thing is that python
is crazy slow on my PPC-Mac (circa 2003).  I am running py27 with
the latest trunk myhdl.  py27 installed with macports (port install)
To get another system in the mix I stole my wifes laptop which can
run pypy (side note, how quickly things can be setup with MyHDL/Python).

Also, my non-numpy simulations, yesterday, were failing.
They failed, because I was using the traceSignals (?).  When I
removed the traceSignals function I was able to run two of the
simulations with pypy :)

Below is a table of execution times for the different simulations.

+---------+---------+----------+-----------+----------+----------+
|         | OSX-CPy | OSXi-CPy | OSXi-pypy | Ubu-CPy  | Ubu-pypy |
+---------+=========+==========+===========+==========+==========+
| rFFT    | 10m.52s | 4m.6s    | failed*   | 4m.36s   | failed*  |
+---------+---------+----------+-----------+----------+----------+
| CIC     | tbr     | tbr      | failed*   | tbr      | failed*  |
+---------+---------+----------+-----------+----------+----------+
| USBP    | tbr     | 4m.30s   | 2m.20s    | 4m.50s   | 2m.28s   |
+---------+---------+----------+-----------+----------+----------+
| LED     | 30m.4s  | 15m.15s  | 2m.10s    | 16m.13s  | 1m.59s   |
+---------+---------+----------+-----------+----------+----------+
* failed due to the numpy dependency

System configurations
OSX  - Dual 2GHz PPC G5, 4.5GiB RAM, OS X 10.5.8
OSXi - 2.26GHz Intel Core 2 Duo, 2GiB RAM, OS X 10.6.7
Ubu  - AMD Athlon LE1620 1.7GiB RAM, Ubuntu (natty)

The relevant information here is the ~2x increase and ~8x increase
with pypy.

Hope this is useful.

Chris Felton

[myhdl-list] Conversion Example Error in 0.7 Manual

[Christopher F. <chr...@gm...>]

A new user to MyHDL was trying to convert one of the examples in the 
manual on pages 67-68 in the pdf version for release 0.7 
(conversion_examples.rst) 
http://www.myhdl.org/doc/current/manual/conversion_examples.html.

The example failed to convert for a "output port read internally" error. 
  This error only appears to occur with the 0.7 release.  This might be 
a known issue.

With the latest hg repo, 0.8dev, the error does not occur but rather a 
warning.

Regards,
Chris

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Angel E. <ang...@gm...>]

On Wed, Jun 8, 2011 at 11:13 AM, Christopher Felton
<chr...@gm...> wrote:
> On 6/8/11 4:07 AM, Angel Ezquerra wrote:
>> On Wed, Jun 8, 2011 at 10:45 AM, Christopher Felton
>> <chr...@gm...>  wrote:
>>> On 6/7/11 7:15 AM, Jan Decaluwe wrote:
>>>> A few weeks ago, I posted the following:
>>>>
>>>>> From: Jan Decaluwe<ja...@ja...>
>>>>> Newsgroups: gmane.comp.python.myhdl
>>>>> Subject: My take on MyHDL weaknesses - and solutions
>>>>> Date: Mon, 04 Apr 2011 11:00:23 +0200
>>>>> ...
>>>>> Solution 2: The PyPy project (future). Python doesn't have
>>>>> to be slow: by employing clever JIT techniques, performance
>>>>> can be improved drastically e.g. 5-10x. I believe that MyHDL
>>>>> is a good candidate for this type of optimization. The day
>>>>> this happens would be one of the most important ones in
>>>>> MyHDL's history: the performance limitations would for a
>>>>> large part go away, and it would be the ultimate validation
>>>>> of the concept: benefitting from advances without having to
>>>>> do anything yourself :-)
>>>>
>>>> I'm very happy to report that it looks like this day has
>>>> arrived now. PyPy 1.5 is compatible with Python 2.7, and
>>>> I'm seeing really spectacular speedup factors.
>>>>
>>>> I have documented my findings extensively here:
>>>>
>>>>        http://www.myhdl.org/doku.php/performance
>>>>
>>>> Obviously, I would like to create some buzz about this, but
>>>> I want to move carefully. First I would like to know if
>>>> people with relevant simulations get similar results.
>>>>
>>>> Feedback is therefore very welcome; I think everything needed
>>>> to get started is documented on the page mentioned above.
>>>>
>>>> Jan
>>>>
>>>
>>> The following are some notes at my attempt to use pypy on some of my
>>> projects today.
>>>
>>> Jump to the Chase
>>> =================
>>> My experience was mixed.  Majority of my projects, simulations, utilize
>>> numpy and other packages (matplotlib, etc).  Most of these do not work
>>> with pypy.  Majority of my project simulations failed with pypy.  I read
>>> there are efforts to get numpy working with pypy.  But it is unknown
>>> when these would be available.  Most of my active projects have some
>>> dependency that failed and prevented the simulations from running with
>>> pypy.
>>>
>>> Note: My testbenches utilize numpy and other common Python libraries.
>>> The hardware descriptions do *not* use the libraries.
>>>
>>>
>>> Disclaimer
>>> ==========
>>> I did not set out to run a set of simulations to preform similar task
>>> that Jan Decaluwe did in his excellent write-up.  But rather test pypy
>>> against existing projects (in existing states) and record the results
>>> with minimal modifications and effort.  Other than getting the
>>> *existing* simulations to run no additional work was performed to create
>>> a good test suite.
>>>
>>>
>>> Notes on Installing pypy
>>> ========================
>>> I usually work on an ancient mac and pypy1.5 is available through
>>> macports.  Should be easy to install ...
>>>
>>>    ``>>  port install pypy``
>>>
>>> But, it failed to build and it wasn't clear why, next step try building
>>> from the source, binaries not available for my machine.  This was
>>> confusing.  The pypy installing PYPY page_ has instructions where to get
>>> the source (tarball or hg).  But then jumps to testing the install and
>>> not the actual act of installing?
>>>
>>> .. _page: pypy.org/download.html#install
>>>
>>> But installing on the Ubuntu box was pretty straight forward.  Download
>>> the binaries and off you go.  The simulations were attempted on an
>>> Ubuntu system.  Verified pypy worked.
>>>
>>>
>>> Simulations
>>> ===========
>>> As discussed, the first couple simulations I tried failed because of the
>>> unsupported dependencies.  Which required some investigation on the
>>> failures and possible solutions.  At this time there is no solution for
>>> the numpy and company other than rewriting the simulations, not to use
>>> numpy.  The following is the list of my projects I attempted to run with
>>> pypy.  Most of these are described on the myhdl wiki.
>>>
>>>    1. Recursive FFT (failed, numpy)
>>>    2. GnuRadio Compatible USRP CIC Filter (failed, numpy)
>>>    3. Fixed-Point SOS (failed, numpy)
>>>    4. USBP (FX2 USB Interface) (failed in myhdl??)
>>>
>>> All in all, I could not get any of them to run with pypy?  Some were
>>> expected but some were not.
>>>
>>>
>>> Summary
>>> =======
>>> I think pypy is very promising but there are some limitations.  I don't
>>> know how many others would be affected by the limitations but for me it
>>> was fairly severe.  But I also need to devote some more time resolving
>>> the non-dependency issues and working through those.  I look forward to
>>> the day more packages are supported by pypy!
>>>
>>> Chris Felton
>>
>> Chris,
>>
>> it seems that you are not the only one who'd like to get numpy to work
>> with pypy. Check these two posts on the pypy blog:
>>
>> http://morepypy.blogspot.com/2011/05/numpy-in-pypy-status-and-roadmap.html
>> http://morepypy.blogspot.com/2011/06/report-back-from-our-survey.html
>>
>> In short, they know this is important, they have a plan on how to
>> proceed, they believe they can make numpy even faster than it is, but
>> they don't have many resources allocated to the task (yet?).
>>
>> So I think we can be cautiously optimist that this problem will be
>> solved eventually.
>>
>> In the meantime I'd love to know if your projects work when you remove
>> the numpy stuff.
>>
>> Angel
>
> I imagine I could get them to work without the numpy.  I really only use
> the numerical packages to create the input signals and analyze the
> output signals.  A bunch of it is for analyzing the response, creating
> psd plots, etc.  If I remove a bunch of the post-simulation analysis I
> should be able to reproduce the signal generation with lists and the
> basic Python math libs.  At that point I could measure the speed
> improvements with pypy.
>
> Thanks for the links!
>
> Chris
>
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> myhdl-list mailing list
> myh...@li...
> https://lists.sourceforge.net/lists/listinfo/myhdl-list
>

Someone posted the following story to the python subreddit:

http://www.reddit.com/r/Python/comments/httng/pypy_gives_massive_speed_boost_to_myhdl/

There is already a question asking whether someone uses MyHDL for
serious work. I think it is an excellent opportunity to let people now
that MyHDL can be used instead of the regular VHDL or Verilog workflow
:-)

Angel

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Christopher F. <chr...@gm...>]

On 6/8/11 4:07 AM, Angel Ezquerra wrote:
> On Wed, Jun 8, 2011 at 10:45 AM, Christopher Felton
> <chr...@gm...>  wrote:
>> On 6/7/11 7:15 AM, Jan Decaluwe wrote:
>>> A few weeks ago, I posted the following:
>>>
>>>> From: Jan Decaluwe<ja...@ja...>
>>>> Newsgroups: gmane.comp.python.myhdl
>>>> Subject: My take on MyHDL weaknesses - and solutions
>>>> Date: Mon, 04 Apr 2011 11:00:23 +0200
>>>> ...
>>>> Solution 2: The PyPy project (future). Python doesn't have
>>>> to be slow: by employing clever JIT techniques, performance
>>>> can be improved drastically e.g. 5-10x. I believe that MyHDL
>>>> is a good candidate for this type of optimization. The day
>>>> this happens would be one of the most important ones in
>>>> MyHDL's history: the performance limitations would for a
>>>> large part go away, and it would be the ultimate validation
>>>> of the concept: benefitting from advances without having to
>>>> do anything yourself :-)
>>>
>>> I'm very happy to report that it looks like this day has
>>> arrived now. PyPy 1.5 is compatible with Python 2.7, and
>>> I'm seeing really spectacular speedup factors.
>>>
>>> I have documented my findings extensively here:
>>>
>>>        http://www.myhdl.org/doku.php/performance
>>>
>>> Obviously, I would like to create some buzz about this, but
>>> I want to move carefully. First I would like to know if
>>> people with relevant simulations get similar results.
>>>
>>> Feedback is therefore very welcome; I think everything needed
>>> to get started is documented on the page mentioned above.
>>>
>>> Jan
>>>
>>
>> The following are some notes at my attempt to use pypy on some of my
>> projects today.
>>
>> Jump to the Chase
>> =================
>> My experience was mixed.  Majority of my projects, simulations, utilize
>> numpy and other packages (matplotlib, etc).  Most of these do not work
>> with pypy.  Majority of my project simulations failed with pypy.  I read
>> there are efforts to get numpy working with pypy.  But it is unknown
>> when these would be available.  Most of my active projects have some
>> dependency that failed and prevented the simulations from running with
>> pypy.
>>
>> Note: My testbenches utilize numpy and other common Python libraries.
>> The hardware descriptions do *not* use the libraries.
>>
>>
>> Disclaimer
>> ==========
>> I did not set out to run a set of simulations to preform similar task
>> that Jan Decaluwe did in his excellent write-up.  But rather test pypy
>> against existing projects (in existing states) and record the results
>> with minimal modifications and effort.  Other than getting the
>> *existing* simulations to run no additional work was performed to create
>> a good test suite.
>>
>>
>> Notes on Installing pypy
>> ========================
>> I usually work on an ancient mac and pypy1.5 is available through
>> macports.  Should be easy to install ...
>>
>>    ``>>  port install pypy``
>>
>> But, it failed to build and it wasn't clear why, next step try building
>> from the source, binaries not available for my machine.  This was
>> confusing.  The pypy installing PYPY page_ has instructions where to get
>> the source (tarball or hg).  But then jumps to testing the install and
>> not the actual act of installing?
>>
>> .. _page: pypy.org/download.html#install
>>
>> But installing on the Ubuntu box was pretty straight forward.  Download
>> the binaries and off you go.  The simulations were attempted on an
>> Ubuntu system.  Verified pypy worked.
>>
>>
>> Simulations
>> ===========
>> As discussed, the first couple simulations I tried failed because of the
>> unsupported dependencies.  Which required some investigation on the
>> failures and possible solutions.  At this time there is no solution for
>> the numpy and company other than rewriting the simulations, not to use
>> numpy.  The following is the list of my projects I attempted to run with
>> pypy.  Most of these are described on the myhdl wiki.
>>
>>    1. Recursive FFT (failed, numpy)
>>    2. GnuRadio Compatible USRP CIC Filter (failed, numpy)
>>    3. Fixed-Point SOS (failed, numpy)
>>    4. USBP (FX2 USB Interface) (failed in myhdl??)
>>
>> All in all, I could not get any of them to run with pypy?  Some were
>> expected but some were not.
>>
>>
>> Summary
>> =======
>> I think pypy is very promising but there are some limitations.  I don't
>> know how many others would be affected by the limitations but for me it
>> was fairly severe.  But I also need to devote some more time resolving
>> the non-dependency issues and working through those.  I look forward to
>> the day more packages are supported by pypy!
>>
>> Chris Felton
>
> Chris,
>
> it seems that you are not the only one who'd like to get numpy to work
> with pypy. Check these two posts on the pypy blog:
>
> http://morepypy.blogspot.com/2011/05/numpy-in-pypy-status-and-roadmap.html
> http://morepypy.blogspot.com/2011/06/report-back-from-our-survey.html
>
> In short, they know this is important, they have a plan on how to
> proceed, they believe they can make numpy even faster than it is, but
> they don't have many resources allocated to the task (yet?).
>
> So I think we can be cautiously optimist that this problem will be
> solved eventually.
>
> In the meantime I'd love to know if your projects work when you remove
> the numpy stuff.
>
> Angel

I imagine I could get them to work without the numpy.  I really only use 
the numerical packages to create the input signals and analyze the 
output signals.  A bunch of it is for analyzing the response, creating 
psd plots, etc.  If I remove a bunch of the post-simulation analysis I 
should be able to reproduce the signal generation with lists and the 
basic Python math libs.  At that point I could measure the speed 
improvements with pypy.

Thanks for the links!

Chris

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Angel E. <ang...@gm...>]

On Wed, Jun 8, 2011 at 10:45 AM, Christopher Felton
<chr...@gm...> wrote:
> On 6/7/11 7:15 AM, Jan Decaluwe wrote:
>> A few weeks ago, I posted the following:
>>
>>> From: Jan Decaluwe<ja...@ja...>
>>> Newsgroups: gmane.comp.python.myhdl
>>> Subject: My take on MyHDL weaknesses - and solutions
>>> Date: Mon, 04 Apr 2011 11:00:23 +0200
>>> ...
>>> Solution 2: The PyPy project (future). Python doesn't have
>>> to be slow: by employing clever JIT techniques, performance
>>> can be improved drastically e.g. 5-10x. I believe that MyHDL
>>> is a good candidate for this type of optimization. The day
>>> this happens would be one of the most important ones in
>>> MyHDL's history: the performance limitations would for a
>>> large part go away, and it would be the ultimate validation
>>> of the concept: benefitting from advances without having to
>>> do anything yourself :-)
>>
>> I'm very happy to report that it looks like this day has
>> arrived now. PyPy 1.5 is compatible with Python 2.7, and
>> I'm seeing really spectacular speedup factors.
>>
>> I have documented my findings extensively here:
>>
>>       http://www.myhdl.org/doku.php/performance
>>
>> Obviously, I would like to create some buzz about this, but
>> I want to move carefully. First I would like to know if
>> people with relevant simulations get similar results.
>>
>> Feedback is therefore very welcome; I think everything needed
>> to get started is documented on the page mentioned above.
>>
>> Jan
>>
>
> The following are some notes at my attempt to use pypy on some of my
> projects today.
>
> Jump to the Chase
> =================
> My experience was mixed.  Majority of my projects, simulations, utilize
> numpy and other packages (matplotlib, etc).  Most of these do not work
> with pypy.  Majority of my project simulations failed with pypy.  I read
> there are efforts to get numpy working with pypy.  But it is unknown
> when these would be available.  Most of my active projects have some
> dependency that failed and prevented the simulations from running with
> pypy.
>
> Note: My testbenches utilize numpy and other common Python libraries.
> The hardware descriptions do *not* use the libraries.
>
>
> Disclaimer
> ==========
> I did not set out to run a set of simulations to preform similar task
> that Jan Decaluwe did in his excellent write-up.  But rather test pypy
> against existing projects (in existing states) and record the results
> with minimal modifications and effort.  Other than getting the
> *existing* simulations to run no additional work was performed to create
> a good test suite.
>
>
> Notes on Installing pypy
> ========================
> I usually work on an ancient mac and pypy1.5 is available through
> macports.  Should be easy to install ...
>
>   ``>> port install pypy``
>
> But, it failed to build and it wasn't clear why, next step try building
> from the source, binaries not available for my machine.  This was
> confusing.  The pypy installing PYPY page_ has instructions where to get
> the source (tarball or hg).  But then jumps to testing the install and
> not the actual act of installing?
>
> .. _page: pypy.org/download.html#install
>
> But installing on the Ubuntu box was pretty straight forward.  Download
> the binaries and off you go.  The simulations were attempted on an
> Ubuntu system.  Verified pypy worked.
>
>
> Simulations
> ===========
> As discussed, the first couple simulations I tried failed because of the
> unsupported dependencies.  Which required some investigation on the
> failures and possible solutions.  At this time there is no solution for
> the numpy and company other than rewriting the simulations, not to use
> numpy.  The following is the list of my projects I attempted to run with
> pypy.  Most of these are described on the myhdl wiki.
>
>   1. Recursive FFT (failed, numpy)
>   2. GnuRadio Compatible USRP CIC Filter (failed, numpy)
>   3. Fixed-Point SOS (failed, numpy)
>   4. USBP (FX2 USB Interface) (failed in myhdl??)
>
> All in all, I could not get any of them to run with pypy?  Some were
> expected but some were not.
>
>
> Summary
> =======
> I think pypy is very promising but there are some limitations.  I don't
> know how many others would be affected by the limitations but for me it
> was fairly severe.  But I also need to devote some more time resolving
> the non-dependency issues and working through those.  I look forward to
> the day more packages are supported by pypy!
>
> Chris Felton

Chris,

it seems that you are not the only one who'd like to get numpy to work
with pypy. Check these two posts on the pypy blog:

http://morepypy.blogspot.com/2011/05/numpy-in-pypy-status-and-roadmap.html
http://morepypy.blogspot.com/2011/06/report-back-from-our-survey.html

In short, they know this is important, they have a plan on how to
proceed, they believe they can make numpy even faster than it is, but
they don't have many resources allocated to the task (yet?).

So I think we can be cautiously optimist that this problem will be
solved eventually.

In the meantime I'd love to know if your projects work when you remove
the numpy stuff.

Angel

Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Christopher F. <chr...@gm...>]

On 6/7/11 7:15 AM, Jan Decaluwe wrote:
> A few weeks ago, I posted the following:
>
>> From: Jan Decaluwe<ja...@ja...>
>> Newsgroups: gmane.comp.python.myhdl
>> Subject: My take on MyHDL weaknesses - and solutions
>> Date: Mon, 04 Apr 2011 11:00:23 +0200
>> ...
>> Solution 2: The PyPy project (future). Python doesn't have
>> to be slow: by employing clever JIT techniques, performance
>> can be improved drastically e.g. 5-10x. I believe that MyHDL
>> is a good candidate for this type of optimization. The day
>> this happens would be one of the most important ones in
>> MyHDL's history: the performance limitations would for a
>> large part go away, and it would be the ultimate validation
>> of the concept: benefitting from advances without having to
>> do anything yourself :-)
>
> I'm very happy to report that it looks like this day has
> arrived now. PyPy 1.5 is compatible with Python 2.7, and
> I'm seeing really spectacular speedup factors.
>
> I have documented my findings extensively here:
>
>       http://www.myhdl.org/doku.php/performance
>
> Obviously, I would like to create some buzz about this, but
> I want to move carefully. First I would like to know if
> people with relevant simulations get similar results.
>
> Feedback is therefore very welcome; I think everything needed
> to get started is documented on the page mentioned above.
>
> Jan
>

The following are some notes at my attempt to use pypy on some of my 
projects today.

Jump to the Chase
=================
My experience was mixed.  Majority of my projects, simulations, utilize 
numpy and other packages (matplotlib, etc).  Most of these do not work 
with pypy.  Majority of my project simulations failed with pypy.  I read 
there are efforts to get numpy working with pypy.  But it is unknown 
when these would be available.  Most of my active projects have some 
dependency that failed and prevented the simulations from running with 
pypy.

Note: My testbenches utilize numpy and other common Python libraries. 
The hardware descriptions do *not* use the libraries.


Disclaimer
==========
I did not set out to run a set of simulations to preform similar task 
that Jan Decaluwe did in his excellent write-up.  But rather test pypy 
against existing projects (in existing states) and record the results 
with minimal modifications and effort.  Other than getting the 
*existing* simulations to run no additional work was performed to create 
a good test suite.


Notes on Installing pypy
========================
I usually work on an ancient mac and pypy1.5 is available through 
macports.  Should be easy to install ...

   ``>> port install pypy``

But, it failed to build and it wasn't clear why, next step try building 
from the source, binaries not available for my machine.  This was 
confusing.  The pypy installing PYPY page_ has instructions where to get 
the source (tarball or hg).  But then jumps to testing the install and 
not the actual act of installing?

.. _page: pypy.org/download.html#install

But installing on the Ubuntu box was pretty straight forward.  Download 
the binaries and off you go.  The simulations were attempted on an 
Ubuntu system.  Verified pypy worked.


Simulations
===========
As discussed, the first couple simulations I tried failed because of the 
unsupported dependencies.  Which required some investigation on the 
failures and possible solutions.  At this time there is no solution for 
the numpy and company other than rewriting the simulations, not to use 
numpy.  The following is the list of my projects I attempted to run with 
pypy.  Most of these are described on the myhdl wiki.

   1. Recursive FFT (failed, numpy)
   2. GnuRadio Compatible USRP CIC Filter (failed, numpy)
   3. Fixed-Point SOS (failed, numpy)
   4. USBP (FX2 USB Interface) (failed in myhdl??)

All in all, I could not get any of them to run with pypy?  Some were 
expected but some were not.


Summary
=======
I think pypy is very promising but there are some limitations.  I don't 
know how many others would be affected by the limitations but for me it 
was fairly severe.  But I also need to devote some more time resolving 
the non-dependency issues and working through those.  I look forward to 
the day more packages are supported by pypy!

Chris Felton