Re: [myhdl-list] It's a beautiful day (somewhat long response)

[Christopher F. <chr...@gm...>]

On 6/8/11 4:07 AM, Angel Ezquerra wrote:
> On Wed, Jun 8, 2011 at 10:45 AM, Christopher Felton
> <chr...@gm...>  wrote:
>> On 6/7/11 7:15 AM, Jan Decaluwe wrote:
>>> A few weeks ago, I posted the following:
>>>
>>>> From: Jan Decaluwe<ja...@ja...>
>>>> Newsgroups: gmane.comp.python.myhdl
>>>> Subject: My take on MyHDL weaknesses - and solutions
>>>> Date: Mon, 04 Apr 2011 11:00:23 +0200
>>>> ...
>>>> Solution 2: The PyPy project (future). Python doesn't have
>>>> to be slow: by employing clever JIT techniques, performance
>>>> can be improved drastically e.g. 5-10x. I believe that MyHDL
>>>> is a good candidate for this type of optimization. The day
>>>> this happens would be one of the most important ones in
>>>> MyHDL's history: the performance limitations would for a
>>>> large part go away, and it would be the ultimate validation
>>>> of the concept: benefitting from advances without having to
>>>> do anything yourself :-)
>>>
>>> I'm very happy to report that it looks like this day has
>>> arrived now. PyPy 1.5 is compatible with Python 2.7, and
>>> I'm seeing really spectacular speedup factors.
>>>
>>> I have documented my findings extensively here:
>>>
>>>        http://www.myhdl.org/doku.php/performance
>>>
>>> Obviously, I would like to create some buzz about this, but
>>> I want to move carefully. First I would like to know if
>>> people with relevant simulations get similar results.
>>>
>>> Feedback is therefore very welcome; I think everything needed
>>> to get started is documented on the page mentioned above.
>>>
>>> Jan
>>>
>>
>> The following are some notes at my attempt to use pypy on some of my
>> projects today.
>>
>> Jump to the Chase
>> =================
>> My experience was mixed.  Majority of my projects, simulations, utilize
>> numpy and other packages (matplotlib, etc).  Most of these do not work
>> with pypy.  Majority of my project simulations failed with pypy.  I read
>> there are efforts to get numpy working with pypy.  But it is unknown
>> when these would be available.  Most of my active projects have some
>> dependency that failed and prevented the simulations from running with
>> pypy.
>>
>> Note: My testbenches utilize numpy and other common Python libraries.
>> The hardware descriptions do *not* use the libraries.
>>
>>
>> Disclaimer
>> ==========
>> I did not set out to run a set of simulations to preform similar task
>> that Jan Decaluwe did in his excellent write-up.  But rather test pypy
>> against existing projects (in existing states) and record the results
>> with minimal modifications and effort.  Other than getting the
>> *existing* simulations to run no additional work was performed to create
>> a good test suite.
>>
>>
>> Notes on Installing pypy
>> ========================
>> I usually work on an ancient mac and pypy1.5 is available through
>> macports.  Should be easy to install ...
>>
>>    ``>>  port install pypy``
>>
>> But, it failed to build and it wasn't clear why, next step try building
>> from the source, binaries not available for my machine.  This was
>> confusing.  The pypy installing PYPY page_ has instructions where to get
>> the source (tarball or hg).  But then jumps to testing the install and
>> not the actual act of installing?
>>
>> .. _page: pypy.org/download.html#install
>>
>> But installing on the Ubuntu box was pretty straight forward.  Download
>> the binaries and off you go.  The simulations were attempted on an
>> Ubuntu system.  Verified pypy worked.
>>
>>
>> Simulations
>> ===========
>> As discussed, the first couple simulations I tried failed because of the
>> unsupported dependencies.  Which required some investigation on the
>> failures and possible solutions.  At this time there is no solution for
>> the numpy and company other than rewriting the simulations, not to use
>> numpy.  The following is the list of my projects I attempted to run with
>> pypy.  Most of these are described on the myhdl wiki.
>>
>>    1. Recursive FFT (failed, numpy)
>>    2. GnuRadio Compatible USRP CIC Filter (failed, numpy)
>>    3. Fixed-Point SOS (failed, numpy)
>>    4. USBP (FX2 USB Interface) (failed in myhdl??)
>>
>> All in all, I could not get any of them to run with pypy?  Some were
>> expected but some were not.
>>
>>
>> Summary
>> =======
>> I think pypy is very promising but there are some limitations.  I don't
>> know how many others would be affected by the limitations but for me it
>> was fairly severe.  But I also need to devote some more time resolving
>> the non-dependency issues and working through those.  I look forward to
>> the day more packages are supported by pypy!
>>
>> Chris Felton
>
> Chris,
>
> it seems that you are not the only one who'd like to get numpy to work
> with pypy. Check these two posts on the pypy blog:
>
> http://morepypy.blogspot.com/2011/05/numpy-in-pypy-status-and-roadmap.html
> http://morepypy.blogspot.com/2011/06/report-back-from-our-survey.html
>
> In short, they know this is important, they have a plan on how to
> proceed, they believe they can make numpy even faster than it is, but
> they don't have many resources allocated to the task (yet?).
>
> So I think we can be cautiously optimist that this problem will be
> solved eventually.
>
> In the meantime I'd love to know if your projects work when you remove
> the numpy stuff.
>
> Angel

I imagine I could get them to work without the numpy.  I really only use 
the numerical packages to create the input signals and analyze the 
output signals.  A bunch of it is for analyzing the response, creating 
psd plots, etc.  If I remove a bunch of the post-simulation analysis I 
should be able to reproduce the signal generation with lists and the 
basic Python math libs.  At that point I could measure the speed 
improvements with pypy.

Thanks for the links!

Chris