Re: [myhdl-list] Starting over (was: Migen logic design toolbox - now with simulator)
Brought to you by:
jandecaluwe
From: David G. <dsg...@gm...> - 2012-04-22 20:02:09
|
Hi Bob, I wanted to let you know about my own solution to this problem, a language embedded in Clojure called Piplin. I initially sought to use MyHDL for some of the reasons you did (integration ease) and some other reasons (leveraging type systems and new syntax to improve expressiveness). Yesterday, I successfully synthesized the code to verilog. I've implemented the type system and simulator too, but there's a lot of work to be done. Since it's embedded in Java, it's easy to use all kinds of numerical libraries and integrate with many systems. Here's the code: github.com/dgrnbrg/piplin The best way to see what it looks like in context is here: https://github.com/dgrnbrg/piplin/blob/master/src/piplin/mips.clj I started this project in February from a blank slate, so although it's not ready yet, the ideas are there, and I'd love your feedback on syntax, semantics, and features that are important. Thanks, David On Sun, Apr 22, 2012 at 2:53 PM, Bob Cunningham <fl...@gm...> wrote: > Thank-you Jan for your careful and caring response. It is clear I haven't gone far enough down my own learning path to make well-founded criticisms of HDL tools or approaches. While I continue to closely monitor this list and developments in the general world of "alternative" HDLs, a recent job change has temporarily eliminated my free time to pursue learning HDLs: My FPGA development board is gathering dust on my workbench. Still, when I finally do resume my efforts, I hope to have a better path to pursue. > > Rather than attempt to respond to your points individually, please permit me instead to step back and restate my goals and describe how they have evolved since encountering MyHDL and Migen. > > As an embedded real-time engineer specializing primarily in software, but with proven skills in system design and high-level circuit design (mainly processor and peripheral chip selection and integration), my greatest goal is to minimize the amount of new software in the products I create. Statistically, nothing adds bugs to a system faster, and delays product shipping longer, than adding a single new line of code. The buggiest code tends to occur at the lowest levels: At the functional level, the error rate for assembler is a multiple of that for C which in turn is a multiple of that for Python. > > It's not just about levels of abstraction, though that is certainly a vital factor. It also concerns the availability of proven code, and how easily that code can be integrated onto the current project. Part of the issue is how easy or difficult it is to create portable and reusable code on any system, the rest concerns how to make use of that code. I love projects where most of my effort is spent writing "glue" code to integrate known-good libraries to create a new application. My work as an engineer then becomes finding that good code, and selecting from among the candidates the code that best meets my needs and will be easiest to integrate. > > That said, what I love most is writing code from scratch, implementing something important that has never existed before. But that's not what gets products out the door: Professionally, I must hold my own code authoring skills and desires at bay, and use the rest of my skills to assess what's best for the system and product as a whole. It's kind of schizophrenic: I ship better systems sooner when I keep my own coding skills as a last resort, and prioritize integrating the work of others. But I always yearn for projects where I get to do something truly new and innovative. > > I view the FPGA as an ideal canvas for an embedded engineer, where I can work more closely with the EEs while simultaneously using using my system design and integration skills to further reduce the software needed to produce a product. > > The first FPGA skill I'd most like to have to be productive would be the ability to write glue between the FPGA and devices in the outside world, and between IP cores within the FPGA. I became very excited when I saw that Migen included the Wishbone interface as a pre-integrated macro: There are many cores over at OpenCores that use the Wishbone interface, but getting them to talk to each other required a level of skill that was way beyond my knowledge. > > Why would I need to know how to implement a Wishbone interface? What I need to know is how to design and test a system that uses components integrated with standard interfaces, and those skills are readily transferred from my embedded experience, where I have had to test and integrate many hardware systems and components. I like using logic analyzers and o'scopes far more than I like using software debuggers! > > I suppose this isn't really an HDL issue: It's much higher than that. I suppose it's more of a tool-level issue with language implications. > > When I finally do start to write my own complex circuits from scratch (something I greatly look forward to, and expect to have lots of fun doing), I'll want to use a language (or languages) that are appropriate to the task at hand, much as I use assembler, C and Python to create embedded software. > > However, identifying the most appropriate language can be difficult, and may even be a needless effort if the highest-level languages can "play nice" at the lowest level. Permit me to share a case in point: > > Several years ago I had to develop an instrument based on a complex sensor system that used inherently noisy sensors from which we needed to extract "soft" real-time performance. The statistics alone were quite daunting, and I decided to prototype the system in Python so I could use Numpy and related libraries to quickly develop the algorithms I needed. The prototype was a success, but it had one interesting characteristic: It met our performance sped when running on the desktop, and was only a factor of 2-3 slower than our minimum required performance spec when running on our intended target system. Surprising performance for a prototype in an interpreted language. > > I profiled the code and found that the Python numeric and statistical libraries were wicked-fast: They were more than capable of doing all the math needed well within our timing requirements. That meant the slow performance was due to the code I had written to interface to the sensors and prepare their data for use by the fast libraries, and passing data between those libraries. I first moved as much functionality as possible into the libraries, which yielded an immediate 25% speed improvement. Next, I stumbled across Psyco (these were pre-PyPy days), and used it to more than double the performance of the non-library code I had written. > > That left me tantalizingly close to meeting my timing requirements, but I was still missing them by 30%. I had never before considered using Python to implement a near-real-time embedded system. Not only that, I also had never shipped Linux in a delivered system: I had always been driven to minimize system power, which meant small processors, which in turn meant running no code that didn't directly support the end result, which in turn meant often running with no OS, or at most an RTOS. For this system I had a much more generous power envelope due to the generous maximum weight my battery-powered instrument could have. If I went with a Linux platform, I'd need to double my power budget, which meant doubling the current rating and capacity of the batteries: Switching from NiMh to Li-Ion permitted me to get the power I needed with only a modest increase in system weight. > > But how to obtain that last bit of performance I needed? I was already using the fastest CPU I could get that met our packaging requirements (no fan, no extensive or expensive passive heatsinks, which at the time meant a ULV Celeron). A final round of profiling indicated my interface code was spending lots of time reading and converting data: Each data element was causing individual library calls. Implementing larger buffers helped, but still didn't get me close to the time I needed: Servicing the hardware was the problem. The final speed boost came when the buffers were moved from software to hardware, so each interface access returned 1000x more data, a design change that also improved the timing between the individual sensors, which in turn simplified the statistics needed to process the data. > > Sorry for the length of that story, but that experience has affected all the work I've done since, and also affects my expectations when it comes to FPGA development. I want my FPGA development to integrate nicely into the above process, to be a powerful tool that permits me not just to engineer a fully-specified circuit into a fully-specified target, but also to experiment with integrating different components, to move solutions or parts of the problem from one domain to another, to quickly make use of the work of others, and be able to create my own when needed. > > The key element that must be present to facilitate such flexibility is the ability to implement common aspects of interfaces across multiple domains, to switch interfaces while preserving functionality, to be able to work with gates, busses, processors, instructions, libraries, languages and whatever else is needed to permit a given slice of functionality to be implemented and accessed in its best and most effective domain. I want everything from simple FIFOs to complex processor interfaces to be available to me in multiple domains without having to create them from scratch each time. > > And THAT is what initially attracted me to MyHDL: If I can use Python to move software into hardware, and interface with it from the Python domain, why, I could become a system development god! And the advent of PyPy makes that approach even more tractable. > > I soon learned that such power is not the goal of MyHDL: The goal of MyHDL is to use Python to make a incrementally better flavor Verilog or VHDL, a relatively small but significant evolutionary step compared to the transformative change I was dreaming of. > > I desire to climb up and skip between levels of abstraction and implementation domains that are almost orthogonal to what MyHDL pursues. MyHDL may prove to be a piece of the final process, but it does not, and perhaps cannot, encompass that process on its own. > > Hence my excitement with Migen: Even in its embryonic, incomplete, and flawed initial state, it clearly seeks to bridge abstractions in ways alien to MyHDL, to make interfaces an integral part of the package and process, rather than something to code to at a low level. In this respect, MyHDL feels more like C, and Migen aims to be Python with its powerful libraries being made available to higher-level code. > > Again, I'm only mapping from domains I know to a domain I hope to enter and in which I hope become proficient and productive. I'm certain my mapping has flaws and gaps owing to my lack of general HDL knowledge. But I suspect this very lack may enable me to conceive of and pursue a path that may yield a greater whole. > > MyHDL and Migen both seem to me to be stepping stones in a stream I wish to see crossed by a 4-lane bridge. And arguments between and about them seem like arguments about C vs. Python: It's about the solution path and the power of the process, not about any particular tool! It's about how tools interact and combine, not about how or why they differ or overlap. > > > And if Python/PyPy doesn't evolve more quickly, it may get devoured by Julia, an interpreted/JIT language which abstracts into areas Python presently supports badly (such as multiprocessing and coroutines), with speeds PyPy has yet to attain. It may be that the concepts embodied in both MyHDL and Migen could eventually see more effective and more flexible implementations in the Julia ecosystem. Python, too, is just one tool in a procession of tools over time. > > > -BobC > > > On 04/22/2012 03:25 AM, Jan Decaluwe wrote: >> I still want to take the time to clarify my position >> on the many issues raised in this post. >> >> On 03/17/2012 09:20 PM, Bob Cunningham wrote: >>> On 03/16/2012 02:03 PM, Jan Decaluwe wrote: >>>> My conclusion is that Migen found an easy target in you, as a >>>> newbie, to confuse you. It made you think it can be used for >>>> serious design work. >>> Sorry, Jan. If I have to be "confused" to play with my FPGA, then so >>> be it. I'm very "serious" about being able to play with my FPGA! >>> >>> Your statement has an obvious implicit context: To me, you are >>> shouting, "MyHDL is for Serious Designers Only! Newbies and >>> Pragmatists should Go Away!" >>> >>> If that's what you are saying, then please be direct: Don't attack >>> Migen for being what it was intentionally created to be, or for being >>> something MyHDL is not intended to be. Are you upset about Migen >>> existing, or that there is an audience MyHDL can't help as well as >>> Migen may be able to? >> I am disturbed by the suggestion that my critique on Migen is >> based on anything else than a purely technical assessment. >> >> Let me be clear. I don't like Mr. Bourdeauducq's attitude for >> one bit. But do you think that would be a reason for me to >> ignore any good idea that he might come up with? Of course >> not. I am not a masochist. >> >> It is quite simple. During my career, I have found that when >> you think you have seen the worst in HDL-based design, it >> always gets worse. To date, Migen is the worst that I have >> seen. But to understand why I am saying this, you have to >> be prepared to follow my technical arguments and to >> engage in technical discussions. I have made a few starts, >> but I certainly was not done yet. However, I see close to >> zero enthousiasm to continue such discussions. >> >> I am therefore frustrated by the fact that I hear all kinds >> of opinions and suggestions to "merge" but that whenever things >> get a little technical then the "I am a beginner" umbrella opens. >> >> Migen is not my problem. It will disappear in the milky mist >> of HDL history, just like the many HDLs based on the same >> flawed paradigm. I am addressing it simply because misleading >> posts about it appear on this newsgroup. >> >> What I am really targetting instead is the conventional wisdom in >> mainstream HDL design, which often has it all wrong. >> >>> If you'd rather beginners like myself go >>> elsewhere, just say so. >> MyHDL welcomes beginners. It is the first item on "Why MyHDL": >> >> http://myhdl.org/doku.php/why#you_are_new_to_digital_hardware_design >> >> In fact, I put most of my hopes on beginners, as they have not >> yet been brainwashed by the conventional wisdom. >> >>> Remember, Migen wasn't created to be useful to beginners: It was >>> created by an experienced FPGA designer concerned about practicality >>> and productivity for a specific Open Source hardware project. It >> I am not impressed by "arguments from experience". The conventional >> wisdom that I am targetting was created by experienced designers, >> mostly Verilog ones. Experience can lead to conservatism and >> can become a hindrance to clear thinking. >> >>> simply happened that some things became a bit clearer to me after >>> looking at Migen and the problems it was created to address. >> Was it because of Migen or simply because you were spending more >> time on the problem? >> >>> Whatever Migen leaves out may have been what was getting in my way! >> I find that strange. I understand that you have a lot of >> experience with embedded software. Therefore, you must know >> procedural software techniques very well. That is exactly what >> Migen leaves out. What it leaves is low-level concurrency at the >> statement level, which must be new to you. And now you suggest >> that the obstacle is exactly that what you are most familiar >> with. Beats me. >> >>> I'm actually quite lazy: What is the *least* I need to know to make >>> useful digital designs *now*? >> No secrets here. The first sentence of "Why MyHDL" warns you: >> "There's a lot to learn and it will be hard work". Therefore, if >> you are intellectually lazy (not prepared to learn new things even >> when they will save you lots of time and effort later on), MyHDL >> or HDL-based design is not for you. >> >> MyHDL is for those who are lazy in the good engineering sense, >> wanting to accomplish more with less effort eventually. >> >>> I'm a beginner: Though I'd love to someday be able to design >>> circuits like Shakespeare wrote sonnets, I'd be more than happy today >>> if I were able to work at the level of "Green Eggs and Ham", a true >>> masterpiece written with an absolute minimum of linguistic >>> complexity. >> Come on, let's keep some perspective here. It's not *that* difficult >> or complex either. And there is a cookbook that shows you the way. >> >>>> When Migen claims that the "event-driven" paradigm is too general, >>>> what it really dumps is procedural support in your HDL descriptions >>>> - the interesting stuff. >>> What's "interesting" to you can be a frustrating block for a newbie. >>> I hope to one day also be very interested in those aspects of MyHDL, >>> but it seems to have little to do with what I want to get done today, >> I don't understand. Your specification seems very extensive and >> ambitious. It would seem that you have a big need for raising the >> abstaction level as high as possible, and for an easy path to >> strong verification. >> >>> which is to find the simplest path to get basic circuits out of my >>> mind and in to my FPGA. Then use them as building-blocks in my hobby >>> projects. >> There is a broad concensus about the "building blocks" paradigm >> in hardware design. That is really not the issue. The issue is >> what the abstraction level of the building blocks should be. >> >>> I am a MyHDL fan. Unfortunately, I simply remain unable to use MyHDL >>> to accomplish my own immediate hobby needs. That does not indicate >>> any flaw in MyHDL, merely the extent my own limitations. Do not be >>> surprised that I am interested in any tool that helps me circumvent >>> those limitations! >>> >>> I actually *like* how Migen slices and dices the process of FPGA >>> design: The parts that are left out are ones I doubt newbies like me >>> would notice, much less care about, until confronted with unusually >>> challenging designs. I suspect Sebastien would agree with much of >>> your analysis, the proper response being: "So What?" >> Suppose that I teach a class to newbies in embedded software design >> based on assembler. Would any of the newbies, except for the >> rare genius, miss the capabilities of say, C? Does this prove that >> teaching assembler was a good choice? >> >>> It's not about theoretical power or completeness: It's about barriers >>> to entry. It's not about what I can do in 5 years, but about what I >>> can do in 5 weeks. Migen is primarily about pragmatism and >>> productivity, making powerful circuits quickly and easily, and far >>> less about expert capabilities, theoretical purity or even >>> consistency. >> Again, I find this strange. I understand that you have not been >> successful with MyHDL. However, as I understand it you have not >> been successful with Migen either. So what is your defense based >> upon? Of course, we are about 5 weeks further now :-) >> >> More to the point. >> >> Barriers to entry - ok, but what is the task? I told you that I >> believe the main problem in HDL-based design is verification, and >> how MyHDL (unlike Migen) helps you by the fact that you >> can use the same modelling paradigm for high-level models and >> test benches as for synthesizable logic. >> >> You seemed surprized, which I found suprizing in turn. Is >> it so different in software? Really, getting those gates into >> an FPGA is the easy part. The difficult part is getting >> them to work properly. >> >> You will have noticed that Mr. Bourdeauducq made an error in >> the first simple "one-liner" circuit that I presented to him, >> as if he wanted to prove my point. Of course, the reason is >> not incompetence, but simply that he did not verify his >> design. >> >> There is a pattern however. Mr. Bourdeauducq cried foul because >> I didn't accept his "simple patches". What he ignored, and >> continued to ignore despite my insistence is that they broke >> MyHDL. Perhaps Mr. Bourdeaducq considers verification a "detail". >> >> Well, I don't. Verification is the problem. The reason why I >> think the abstraction level of synthesizable logic should >> be as high as possible, is because that leaves more time >> for verification. >> >>> I seek tools that will help me do what I want to get done today, and >>> so far Migen seems like it will be most useful. Tomorrow will likely >>> require additional tools, and I *absolutely* expect (and want) MyHDL >>> to the first of those tools. It is not an either-or proposition: I >>> want Migen *and* MyHDL. I realize MyHDL will take more time to >>> master, and I'm willing to commit that time. But I also want to >>> create something sooner, rather than later. And I believe that 100% >>> of what I learn using Migen will later prove useful with MyHDL. I >>> believe using Migen will keep me motivated toward learning MyHDL. >> Sounds good, but I think it is cheap talk. >> >> Most of Migen's technical choices, starting with its basic paradigm, >> are almost the opposite of MyHDL's. As a result, verification is not >> addressed, and it forces you to think at an artificially low level >> for synthesizable logic. What good can one learn from that? >> >>> Right next to me I have a Spartan 3E-500 that contains nothing of my >>> own design. That must change! >> Perhaps you are too ambitious. >> >> In your shoes, I would start as follows: >> >> * isolate a simple function out of your spec >> * try to concentrate on what it does, instead of how it should be implemented >> * write that behavior in a (clocked) MyHDL process or processes >> * also describe it in high-level python, and use that in a unit-test to verify >> * experiment with synthesis to see whether it works and the result is acceptable >> * iterate with the synthesizable description as necessary >> >> > > ------------------------------------------------------------------------------ > For Developers, A Lot Can Happen In A Second. > Boundary is the first to Know...and Tell You. > Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! > http://p.sf.net/sfu/Boundary-d2dvs2 > _______________________________________________ > myhdl-list mailing list > myh...@li... > https://lists.sourceforge.net/lists/listinfo/myhdl-list |