Controlix / Blog: Recent posts

I recently had to switch my revision control system here at SF from Subversion to Git, because of the ongoing problems with SF's SVN support. I always missed using SVN, which is nice, clean and simple to use when compared to Gi (read: fewer things go wrong per unit commit). Well, today I had a situation come up where I could not get Git to recognize a directory I wanted to commit, which resulted in my wasting a couple of hours and I never did find the solution to the problem. I eventually got frustrated enough that I started looking at other project-hosting websites, and I found one that seemed cool: CloudForge (cloudeforge.com). Just for the hell of it, I picked up a free 30-day trial membership and put together a new project for Controlix: ... read more

I have gotten the GHDL runtime support and its associated dependencies to build, link and run. NuttX is crashing somewhere in pthread-land (probably because I'm not yet running any threads |->), but the grotesque amount of link-time bugs I was dealing with, mostly from libgnat, are finally dealt with. Now it is time to try linking in some of the old VHDL controls from the original SVN repository and see if I can get the GHDL runtime to work correctly....

I recently found a nice alternative to FreeRTOS in the NuttX project (www.nuttx.org), and have decided to give it a go so I can get better POSIX compatibility for the various language simulation libraries. NuttX's POSIX support is much more advanced compared to FreeRTOS+POSIX, although it isn't perfect. NuttX will also eliminate the need for the BareMetal OS starter layer. It is definitely the best open source RTOS out there, at least as far as I have seen... Anyway, we will have to see how things go. I have it configured and integrated into the sourcetree and build system, and everything is compiling and the linking is at least not worse than it was before with FreeRTOS (which is still not too great and a huge PITA that I need to fix)....

I found the FreeRTOS multicore extension, which isn't very attractive and is quite old. When looking around for an alternative, I remebered that I had initially decided about a year and a half ago, after some soul-searching after my father died in fall 2017, to use BareMetal OS for this purpose, and I think I am going to go back to it. For those who do not know, BMOS is an exokernel, designed to run on amd64-compatible processors. An exokernel only provides an exteremely simple HAL on top of raw hardware. This comes down to basically the trampolines set which usually comes wth most RTOSes - bootloader, hardware config, per-processor threading, and ultra-minimal drivers for UART, ethernet and raw disk sector I/O. I plan to use BMOS to run one instance of FreeRTOS per processor core, and the UART/keyboard driver code will give me a start in developing the interactive console before I get around to writing driver code in RTL in the upper-level simulation layers of Controlix. I think that BMOS has everything I need for this, so wish me luck....

I made the decision today to toss the virtual circuit out of the Controlix stack. I will instead bind the various RTL simulation codebases directly on top of the RTOS and implement a simple IPC-based method of interfacing the various signaling primitives to each other and to the underlying RTOS APIs (and even to the hardware, if I can work mutexes around pointer and DMA accessesors for the individual languages). Now I don't need to worry about writing any compiler code for the languages to the VC, and can compile directly to native assembly which will result in faster code running. Should be a win, if the locking issues don't get too hairy.,,. I guess I have to say that open source and compilation is a more fundamental theoretic basis than the virtual machine concept. I might end up starting a new SF.net project for the VC at some point, but otherwise it is going to be ignored from now on.

The SystemC-on-RTOS thing has been executing on top of POSIX for a while now, so I finally have been able to play with the whole concept a bit more fundamentally than I was able to do previously. FreeRTOS doesn't come with multicore support out of the box; there is a working implementation out there somewhere, supposedly - I haven't been able to find it with the cursory amount of searching I've done so far, which means that I have to test in single-threaded mode on my slow-as-shit old Linux/Ubuntu development box, and it is therefore running extremely slowly during testing. Nevertheless, I have finally been able to do some non-trivial runs using GDB, which has shown me the next problem: testing execution in userspace using a single statically-compiled binary blob crashes in libstdc++, somewhere in memory allocation code. I still haven't figured out how to fix the bug and debugging it using GDB is slow and a real PITA in general, although I also took a while off coding to ruminate. I guess need to get that code running bare-metal in an emulator or on test hardware to get a faster build-debug cycle (or to maybe get lucky and find that it is a multithreading issue in the RTOS code). I need to track down that FreeRTOS multicore support code... anyway, that's the zeitgeist over here.... read more

A few small updates:

• Almost there with 'Hello, World' in SystemC - the whole tree builds, just finishing up fixing some link-time errors.

• There is something wrong with SF.net's Subversion support, which many people have been complaining about for over a year now. I couldn't check in or out, so I have moved the repository over to Git, which I don't prefer but do know how to use, and everything appears to be working smoothly now.... read more

Lots of changes to the source tree over the past week, mostly cleanups; but, the important one is this: FreeRTOS is out, RT-Thread OS is in. FreeRTOS was taking too long for me to get the whole thing working - I had spent months on trying to get it and the associated FreeRTOS+POSIX pthread support library to build and link properly, and I never did get it to run with a "Hello, World" type of example. RT-Thread was fully working with the "Hello, World" example running properly with the Linux/POSIX simulation target in less than a week! It is a good thing I came across it while searching Wikipedia for a list of open source RTOSes which supported pthreads. Hats off to the developers - the only issue is that the dev community for it is almost all Chinese, which makes it a bit difficult at times to find documentation, lists of bugs, etc. Still, the source code and build system is so clean, neat and organized compared to FreeRTOS that I didn't need to refer to much other than it to get everything up and running. Happy!

It is with a heavy heart that I have to make an important announcement: Controlix is no longer going to be written in SpecC, and I am moving everything over to the SystemC language. The SpecC compiler simply isn't up to snuff. It only supports 32 bit executables, has too many bugs, is currently close-to-unmaintained, and the authors (I have corresponded with them) are not willing to put time into maintaining it in the future and have moved on to other projects. The thing is basically dead, and I now feel that I cannot justify the time and energy it would take to work with it, both from my perspective as well as needing to "sell" Controlix as a viable project to the open source community in general. The Acellera SystemC reference compiler (actually it is simply a simulation library and a set of C++ classes) is currently maintained with ongoing releases, doesn't need a whole compiler front-end like the SpecC compiler did, and SystemC is supported by a ton of commercial products from big name companies. Also, I can continue my process of building from existing OS code in C, starting with existing procedural code and "morphing" it into parallelizable RTL-style designs smoothly as time goes on.... read more

I have been coding a lot the last week, but other than coding, I have also been thinking about the ways in which to make use of the Virtual Circuit. Obviously, one way is to just write directly in the VC's native intermediate assembly language, but who wants to do that for a lot of code? What is needed is to be able to take all the major HDLs (Verilog, VHDL, SystemC and (for me) SpecC) and be able to compile them into VC IR assembly. For that, I need to use Icarus Verilog.... read more

Lots of work on the source tree over the last two weeks:

• The Plasma RTOS is out and FreeRTOS is in. FreeRTOS is much, much better designed overall than Plasma, which basically was just "some RTOS" (most of them do basically the same things overall). Should be a major win as far as the overall design of the Controlix source system.

• The BareMetalOS exokernel is also out. FreeRTOS comes with code to handle the boot-and-initialization process and well as the low-level interfaces to filesystems, both of which Plasma basically lacked and which BMOS was going to provide.... read more

OK, so I ended up having to switch my OS back to Windows because DragonflyBSD couldn't support the Android SDK properly (I am also trying to work with Freenet on Android as a side project), and therefore I needed to install Cygwin, and because of that I decided to give building the SpecC reference compiler another go. This time it worked! I had very few of the previous bugs to deal with on cygwin, so I am a happy camper indeed. A lucky break for me....

I was browsing through some compiler-related stuff on Wikipedia yesterday, and I came across this:

https://en.wikipedia.org/wiki/ROSE_(compiler_framework)

This is a very well-written compiler-to-compiler framework which supports several "meta-compiler" techniques, the important one of which here is this:

https://en.wikipedia.org/wiki/Automatic_parallelization

Using auto-parallelization allows for extracting fine-grained multithreading out of standard code. The package in Rose which does this produced code which can be fed into OpenMP, and it will even suggest fixes for existing manually-parallelized OpenMP code! This means that you can mix code with explicit spinlocks and regular code, which in turn means that spinlocked code (such as operating system code) can be generated for existing operating systems (e.g. Linux) without having to rewrite much if any of the existing code (aside from having to use OpenMP locks specifically).... read more

It's been a while, and the project is still alive:

• No more SpecC. Sorry, I switched my OS to Dragonfly BSD and would have had to rebuild the SpecC build system for the fourth time, so I am now planning on writing the VC (the core of the OS) in plain C. I still think SpecC is the best RTL out there and the acual compiler code is fine, but I can't countenance inflicting the horrible, broken build process on other developers, and it isn't strictly necessary.... read more

Sorry about jumping the gun on the previous post folks - it turned out that the build wasn't even close to working. It should work now - just tested it with 'make' in the top level directory and it finishes without errors. Dunno if it'll run correctly - that is the next job of work!

As of now, all the Controlix system infrastructure is comitted to SVN and builds from a single invocation of "make all" in the top level directory. I haven't tried to run it yet, and there is no linkage of the "Hello, World!" test code (just the standard rtos_test mini-shell that comes with the Plasma RTOS), but everything else builds and links without errors. The BareMetal OS filesystem image should be created with the correct binaries, suitable (with possible use of a 'raw' FS conversion utility if you are not using QEMU) for loading and running in your favorite emulator.... read more

As I have mentioned before, one potential issue with the overall security-robustness of using SpecC (or any other HDL which makes use of testbenching) is that several "trampolines", written in native code, will be needed. The trampolines won't be very large, of course, and it should be possible to test out nearly all of the potential bugs - in any given code release, of course, and then there are the issues of supporting lots of different hardware platforms, with a concomitant increase in total code size.... read more

"Hello, world!" is building and executing in SpecC on Linux/POSIX. Now I need to hook up the threading from the Plasma RTOS to enable the SpecC simulation library to run, hook up the little mini-libc from BareMetalOS to enable console I/O, and I should be good to go to boot Controlix on bare metal! See you all in few when I have it working....

I found the bug with the build errors in GCC/G++ headers, Aparrently this happens all the time, because SpecC is strictly enforcing (extremely old) ANSI C compatibility. Since the GCC headers don't seem to be built around any sort of ANSI C compatibility, each Linux distro has to have it's own alternate header files. This is allowed for in the SpecC build system, but the last version they included support for was Ubuntu8. I now have to add Ubuntu16 support myself. Oh well, at least I know what the problem was - I was preparing to compile all the libc-using code in src/native with straight GCC and then link it, which will cause problems later when I need to optimize things better.

I have mentioned the virtual circuit system that Controlix will be based on before, but it has been a while. However, I finally think that I have it's design nailed down tight, so I am releasing it. That means bringing the design into Controlix at SF.net and starting to think about coding the VC up in SpecC. More work... anyway, I have added the VC-related notes from my Facebook account here as a Wiki page, and updated the front page with the appropriate links. Take a look and see if you can find any mistakes - it should cleanly and clearly reflect the "true nature" of hardware design, all the way from masks and layout to network programmability with switching and routing.

The current code (ultra-minimal though it is) is now added to the SVN repository 'controlix-new'. It is a mess, but the boot "hello world" type example and the first part of the code of the console server are in and building, producing executables. I spent most of today trying to work around a bug in SpecC - I ended up having to remove the standard '#include <stdio.h>' that one usually adds to the top of any C sourcefile that calls printf(), which my code does. It compiled and linked fine, but I hope it doesn't come back to bite me later....... read more</stdio.h>

Controlix controls will intercommunicate using the standard bus. This will be implemented differently on different targets - virtual memory + DMA on bare metal PC hardware, virtual ethernet on Linux/POSIX, etc. Above that layer (layer 2), however, will need to be some TCP/IP support, although hopefully a bare minimum set. This must be implemented by controlix itself and not the individual controls, however, and so a new native (C and/or assembly) trampoline for the bus needs to be implemented. I will start with the ultra-minimal TCP stack in the MIPS Plasma RTOS which I love to play with, and see if I need to replace it later on. This does make me a little more nervous, because the various types of trampoline code won't be written in a verifiable language and thus will represent a (potential) security hassle. Hopefully I won't have to implement too many protocols - I know DHCP and HTTP will be needed, at least.

Another couple of YEARS without progress, but I'm still not giving up on Controlix. The latest justification for an update is that I've finally managed to get a working build of SpecC, and since I can also now write a back-end translator for Icarus for SpecC (and thus keep compatibility with Verilog and VHDL), I have decided to change the core language Controlix is written in (again!) to SpecC.

SpecC has many advantages here:... read more

I've been spending the past few hours reading up on and playing with Icarus, commonly known as Icarus Verilog, which is an open source Verilog compiler/simulator package much like GHDL is for VHDL. There are some differences: Icarus is written in C++ instead of Ada, and it compiles to an intermediate language called VPP which is then executed/simulated by a standalone utility, 'vpp'.

Good design throughout, but the really nice part is new: Icarus/VPP now supports VHDL-to-VPP compilation! The VHDL support isn't finished yet, but I am going to try to leverage what is there and see if I can replace GHDL with Icarus for use with Controlix, as the primary compiler, because it would make porting Controlix to bare-metal hardware easier. The reason for this is that porting the 'vpp' utility would be all that would be needed and it is also C++ code, much easier to statically link than Ada.... read more

Most system level languages use some kind of macro language along with the core compiler, to enable variables in the build environment to be passed to the compiler, basic logic (in many cases) to be handled before compilation, and many other seemingly minor but very necessary uses. Controlix currently has none of that, and as a result a simple thing I was needing today (Passing a build date string to a shell 'Version' command) is basically impossible.... read more