Use of existing C drivers

[Carl van Denzen]

There are a lot of drivers for hardware (usb, LiquidCrystal, etc.) available in C.
I suggest to investigate the possibility to create some sort of framework to make these drivers usable in HaikuVM. With framework I mean: the directories where to place the .c, .h, .cpp, .java (native calls) files and the creation of the makefiles, etc.

Another possibility is to translate the drivers manually (line by line) from C to Java. And maybe there is something in between (half Java, half C).

Considering using the pure C code versus pure Java:

There are some (arguable) disadvantages:
- more than strictly necessary C code is introduced in HaikuVM:
|- for a Java programmer it is a foreign language
|- the resulting avr code is bigger
|- compilation is more difficult (include files) than compilation of Java
- provisions must be taken to prevent that all drivers are always linked into the executable

The advantages are:
+ there are a lot of C drivers available
+ For new hardware the C drivers will be available quite soon.
+ Edits/improvements of the c code are easily available in HaikuVM
+ speed: for some devices this might be important (usb, wifi)

I am prepared to do the first test with LiquidCrystal. The structure of the directories in HaikuVM is not yet very clear to me, so I did not see how to set up the framework. Hope you can make up some nice structure for it.

[genom2]

First of all - and before I forget - I want to thank you Carl very much for your support and patience!

Back to the topic, I see two choices:

1) JNI
This is close to the sort of framework you are thinking of. But is something in between Java and C. Because you have to declare and use native methods in Java and at the same time you have to write and use some wrapper C code in (JNI) C files.

If you want to use any set of arbitrary C-functions the Arduino world offers, it should be possible with HaikuVM. The standard way to access other C-functions with Java is thru JNI. HaikuVM supports JNI and there are descriptions on the HaikuVM page of how to use it. Four steps to accomplish this:
1.1) Put all needed (Arduino) functions into a C library and put it in your project directory. Lets call the C library libleonardo.a. Here http://www.ashleymills.com/node/327 is a nice (and colorfull) description of how that can be done.
1.2) Refer this C library in the HaikuVM.properties CLIBS variable. In your case 'leonardo.CLIBS = -lleonardo'. (Btw. this should give no need to make a special HaikuVM makefile.)
1.3) Pick the C-functions you want with Java native Methods (using JNI) in your Java program and edit your JNI C files according to the description on the HaikuVM page.
1.4) Let haiku.bat do the rest.

Ok, this is the theory. I don't have experience with this library thing but - as always - I will try to help you where needed. And may be it's wise to start with a smaller step to get famillar with JNI and HaikuVM first.

2) Write in Java (whithout using C)
When it comes to LiquidCrystal I see a chance to write the whole thing in Java. (Because no interrupts are needed.) Thankfully you already did a big part of the work. From here I see three problems:
2.1) Pins
Currently the pins of HaikuVM are only valid for duemilanove. We have to define them for leonardo too.
2.2) write(byte value)
This function is ment to be virtual, overwriting Print.write(byte value). But currently Print.write(byte value) is static. I have to change HaikuVM slightly to support this.
2.3) delayMicroseconds(ms)
Currently this method is implemented too sloppy. If its not working for LiquidCrystal I have to fix this (anyway).

So, where do you want to go from here?

[Carl van Denzen]

My patience is so big, because HaikuVM is so interesting.

1) JNI: sounds good. I already have some experience with JNI (in fact I used JNA), but that was on a Windows platform. I do not yet see why you use another signature and how you managed to do this. But that is not important for me: I will gladly use the proposed signatures.
The JNI thing will be useful for virtual serial.

2) Write in Java. This will be useful for the liquidCrystal.
2.1) Pins: I would like to start with it. If you can do the structural setup (i.e. create a java file in the right place), I can translate the hardware/arduino/variants/leonardo/pins_arduino.h into Java.
2.2) write(byte value): I already noticed this static method and wrote my own print method (that doesn't work by the way).
2.3) delayMicroseconds (or the Thread.sleep() method): that is a mystery for me that I will leave to you to repair.

I would be very pleased if I could access svn to share (and backup) my contributions to this project. I would to start off in a branch. Can you create one for me? I do have experience with svn.

Right now I start with the pins_arduino.h on the leonardo.

[genom2]

1) JNI
My intention is to keep HaikuVM JNI consistent with the output of javah. I will fix obvious differences for version 1.0.3.

2.1) Pins
With (leonardo) pins_arduino.h I see two problems I want to solve:
a) HaikuVM stores arrays in RAM space. Their arrays are stored in PROGMEM. This makes sense because it doesn't waste valuable RAM.
b) They store variable/register addresses in arrays. This can't be done with Java.
I have to think about a and b. Do you have any ideas or preferences?

3) SVN access
In general I'm open to this, but it's not an easy decision for me.
I need to get much more source code from you to have a look on.
And, I need some time.
I really hope you can follow me in this.

[Carl van Denzen]

1) JNI. I wait.
2.1a) Pins in RAM space. I wrote some code to see what it would do. See attached files. There seems to be a problem in the call from digitalWrite to AVRRegister.digitalWrite. Just before the call the program is still alive (checked by turning the LED on), but it looks like the thread never arrives at AVRRegister.digitalWrite.
2.1b) Register adresses in arrays. I worked on one of my ideas with enums, see LeonardoLib.java. Probably we want something more scalable (for all AVR registers). You have experience with annotations (NativeCVariable), maybe you can think of something (e.g. make a call like PORTB.digitalWrite(bit), PORTB.digitalRead(bit), PORTB.xor(byte)).
3) I understand

[genom2]

I needed some time to analyse and understand your inputs.

2.1a)
Your big example is confusing for me. (Please remember, I'm not able to test it with a Leonardo.) Please bring it down to the minimum code size needed to reproduce the error. If possible do it in a Duemilanove and Leonorado agnostic piece of code.

2.1b)
In principle, enum is a good idea and thank you for your try! But when I look into the bytecode produced, it's horrible for a microcontroller. In short:
- Much bytecode (flash mem) is needed for enums. And additional PROGMEM (flash mem) for a string representation for each enum value.
- But what is worse, much RAM is needed. An object for each enum value. Not only a number as you might have expected.

Conclusion: enum is not the way HaikuVM should go.

1) JNI
Meanwhile I made some progress on including and accessing an arduino clib. I have a positive test for Duemilanove using Serial.println(..) from this clib to have an output with Java/HaikuVM. So I have some hope that this will work for Leonardo too.

[Carl van Denzen]

2.1a) Big example is confusing. The error I detected might well have been triggered by a lack of RAM (enum objects and stack?). Because enums are not the way to go, I will not spend any more time on this example.
1) JNI. If it works for other Arduino's, it probably will work on a Leonardo (I don't see any reason yet :-) why it shouldn't). I assume you let me know when I can test it.

[genom2]

1) JNI
Now I have something for you to test with leonardo. It's all in SVN
- http://svn.code.sf.net/p/haiku-vm/code/trunk
revision
- 354
And planed to be in the next ZIP for release 1.0.3 (coming soon).

The simple Java file 'arduino.tutorial.HelloWorldJava.java' for test is in project 'examples'. For this to work with the leonardo you have to use the new configuration 'leonardo.UsingCLIB' (see in 'HaikuVM.properties'). This configuration takes the clib 'libleonardo.a' (located somewhere in project 'lib') to link with and 'haiku.avr.lib.arduino.HaikuMicroKernelLib' as MicroKernel. This MicroKernel declares and uses the native Java methods: init(), write(..), read() and available() to inizialize the chip, setting baudrate to 57600 and perform basic IO. This native Java methods are defined in JNI file 'tutorials/ArduinoUsingCLIB.cpp' of project 'myCProject' (you migth have the need to copy this to your project) and are just wrappers for the C functions in clib.

(A word of caution: All files and file locations mentioned here are only valid for revision 354 and might change for final release 1.0.3.)

2.1)
BTW and because a test was missing I wrote (mainly copied from the internet) an enum test 'haikuvm.bench.EnumTest1.java' to be found in project 'haikuBench' and had to fix a bug in HaikuVM to get it working. (Might be a reason for your 'LeonardoLib.java' using enum to failed.)

[Carl van Denzen]

It looks like libleonardo is not in subversion, see attached file.

[genom2]

Uuups, you are right I missed the lib*.a files.
Now I learned that *.a were excluded from subversion by default. Now I forced subversion to include them and they should be there for you.

[Carl van Denzen]

OK, I found the lib. But... virtual serial is too difficult to start with. So I changed the example to something simple as "DDRC=128;PORTC=128".
After some analysis it looks like there is a problem with the haikumagic.
Putting PORTC=128 in the file HaikuMicroKernelLib.java (and compiling/exporting/haikufying) I am quite sure that my main method is not called.

JVM.c I added DDRC=128; // pin 13 yellow led output

My main method in HelloWorldJava is as simple as this:
DDRC=128;PORTC=128; // this will put the yellow led on

Situation 1, file HaikuMicroKernelLib.java:
PORTC=128;
haiku.vm.HaikuMagic.main(args); // Will call your main method (by
Then the led will go on (as expected)

Situation 2, file HaikuMicroKernelLib.java:
haiku.vm.HaikuMagic.main(args); // Will call your main method (by
PORTC=128;
Then the led will NOT go on. I would expect that it would turn on, not
because of PORTC=128 in this HaikuMicroKernelLib.java, but because of PORTC=128
in HelloWorldJava.java. But it doesn't, so my guess is that my main method
(in HelloWorldJava) is not called.

Can you give me any hints how to investigate this? Or do you want me to provide more information or files?

[genom2]

I have only few ideas. Please try this:

1) Situation 1, file HaikuMicroKernelLib.java:
Replace
- PORTC=128;
by endless blinking:

while(true) {
  PORTC |= 128;
  for(int i=0; i<5000; i++);
  PORTC &= ~128;
  for(int i=0; i<5000; i++);
}

To see if it's a timing problem.

2) Situation 1, file HaikuMicroKernelLib.java:
Replace
- haiku.vm.HaikuMagic.main(args);
direct by
- System.out.println("Hello World Java");
To see if it's a stack problem.

3) general
Change/add configuration to:
- leonardo.UsingCLIB.MemorySize = 1200
To see if it's a heap problem.

[genom2]

I surely was over optimistic with leonardo.This, because duemilanove.UsingCLIB is running like a charm. Now I introduced a JNI function
- haiku.avr.lib.arduino.HaikuMicroKernelLib.debug(marker)
to enable better debugging. (Please re-build bootstrap.jar to use it.)

The implementation of this JNI function is added to
- tutorials/ArduinoUsingCLIB.cpp

I used it three times with markers:
0 -> direct after calling Serial.begin(57600) in ArduinoUsingCLIB.cpp
10 -> before calling haiku.vm.HaikuMagic.main(args) in HaikuMicroKernelLib.java
20 -> before calling System.println(..) in HelloWorldJava.java

All three files are in SVN revision 356.

With 'duemilanove.UsingCLIB' I get this:

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
10 10 10 10 10 10 10 10 10 10
10 10 10 10 10 10 10 10 10 10
20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20
Hello World Java

It also might help you for deeper debugging.

I'm looking forward to your foundings.

[Carl van Denzen]

The serial connection is too difficult (is over USB). So I would like to stick to the led at pin 13 for debugging. Right now I am going to try your suggestions from yesterday.

[genom2]

This is fine.

In case you are curious, I explain how I made the clibs:
- libduemilanove_atmega328.a
- libleonardo.a

I downloaded Arduino IDE 1.0.2, selected board 'Arduino Duemilanove w/ ATmega328' and made the sketch 'Blink'. Arduino IDE 1.0.2 is using a temporary directory 'build*.tmp' in %TEMP% for building the sketch. Herein I found the clib core.a. I just copied it into HaikuVM and renamed it to 'libduemilanove_atmega328.a'. This worked fine for me, including 'Serial' usage.

Then I selected board 'Arduino Leonardo' and made the sketch 'Blink' again. I just copied the freshly made 'core.a' into HaikuVM and renamed it to libleonardo.a. Hoping this will work fine for you, including 'Serial' usage which now - for Leonardo - refers to serial USB.

The pure C function debug(..) in ArduinoUsingCLIB.cpp looks like this:

void debug(int marker) {
    for(int i=0; i<2; i++) {
        for(int j=0; j<9; j++) {
            Serial.print(marker); Serial.print(' ');
        }
        Serial.println(marker);
    }
}

Using 'Serial' from the clib directly.

[Carl van Denzen]

Thank you for this explanation, but you (we?) are going too fast. It looks like even the BlinkSimple example in the trunk head revision doesn't work.
When I go back to 1.0.2 and remove the sei() from JVM.c BlinkSimple.java works nicely.

Could it be possible that indeed the main method is not called correctly? You changed something with arrays (YL.. into YL.._ with underscore). But I cannot understand how that would break the calling of the main method without any error message. And I can neither explain why it does work in your Arduino duemilanove.
Best thing to do is probably (via a binary method) checking which revision is still working and where it goes wrong. I will keep you informed.

[genom2]

Symptoms are sounding strange. And yes, binary method sounds good to find the cause. To help me thinking about this, I need your observations from suggestions 1, 2 and 3.

I have at least run about 40 JUnits (see junits.BaseTests.java and junits.SyncTests.java, ...) tested with configuration 'duemilanove' and 'duemilanove.UsingCLIB'. They all passed the test.

You supposed 'the main method is not called correctly', so did you try my suggestion 2)? Which main(..) do you mean:
a1) HaikuMicroKernelLib.main(..)
or
a2) HelloWorldJava.main(..)

BTW you told 'BlinkSimple.java' isn't running any more, which configuration do you refer to, either 'leonardo' or 'leonardo.UsingCLIB' or both cases?

And remember in case of 'BlinkSimple.java' only one main(..) is involved
b) BlinkSimple.main(..)
if and only if you use the suggested configuration with
- leonardo.MicroKernel = undef
(Yes, I changed it stupidly from "null" to "undef". Sorry for this silent change. Made it as a result of our discussion in the past about the difference between null and "null".)

Out of this thoughts I have a fourth suggestion:

4) Situation 1, file HaikuMicroKernelLib.java:
Replace
- haiku.vm.HaikuMagic.main(args);
direct by
- arduino.tutorial.HelloWorldJava.main(args);
Surpassing the - so called - HaikuVM magic and call the wanted main(..) in this explicit way. To see if 'the main method is not called correctly'.

[Carl van Denzen]

It is going too fast. I even cannot get the BlinkSimple.java working with the original version 1.0.2). Eclipse still has some features I do not understand.
I checked out revision 312 (=1.0.2). I changed JVM.c (commented out sei() again), but the Java part doesn't work. The export of haikutools.jar (I guess the source is haikufier) runs into an error because comm.jar is not found.
It feels like being back at the beginning. I did not save every file I changed. Everything I change in JVM.c before the call to invoke() works as I expect, but after that everything seems to crash (even with sei() commented out).
Could you please make a svn branch, originating from the 1.0.2 tag (that once worked for me). Can you put into this branch:
- the edits you made in JVM.c: #ifdef USBCON ...)
- in HaikuVM.properties: the leonardo section.
- a BlinkSimple.java with only a main() {DDRC=128;PORTC=128}
And can you have a look at the comm.jar problem in the project haikufier. In small steps we can implement the libleonardo.a.

[genom2]

'I even cannot get the BlinkSimple.java working with the original version 1.0.2'
In our thread 'Support for Arduino Leonardo' you gave me at 2013-04-30 this (successful) configuration in an appended file 'HaikuVM.properties':

leonardo.Extends = avr
leonardo.Target = atmega32u4
leonardo.MemorySize = 1540
leonardo.Clock = 16000000
#leonardo.MicroKernel = haiku.avr.lib.arduino.HaikuMicroKernelEx
leonardo.MicroKernel = haiku.avr.lib.simple010.HaikuMicroKernel
leonardo.Port = COM5
leonardo.Upload = avrdude -v -patmega32u4 -cavr109 -P $(HAIKU_PORT) -b57600 -U flash:w:$(HAIKU_OUTPUT)

Did you try this? This can save us a lot of branching (see below).

---

Ok, I will slow down.

'... make a svn branch, originating from the 1.0.2 tag ...'
I will do so in './branches/HaikuVM-1.0.2-Leonardo', but it will take me some time to be consistent and check every thing. If I'm ready (may be at the end of this week) I will send you the valid revision number.

'... comm.jar problem ...'
It's only a experiment for the future. You may delete it and delete files depending on it. I will do so in './branches/HaikuVM-1.0.2-Leonardo'.

[genom2]

step01:
BlinkSimple.java should work out-of-the-box.
At './branches/HaikuVM-1.0.2-Leonardo-step01'.

(Or alternative './branches/HaikuVM-1.0.2-Leonardo' (in revision 363) is ready to use for step 1.) I fullfilled all your requests.

I recommend to use 'bin:haiku.script.launch'|right-mouse-click|'Run as'|'haiku.script' as an external tools launch, to pick 'examples:leonardo.tutorial.BlinkSimple.java' to make and start.
=> To see if a basic run is possible.

step02:
At './branches/HaikuVM-1.0.2-Leonardo-step02'.
- Added arduino libs (but without using them yet).
- Added cpp support.
- haiku.script starts with configuration step02.
=> To see if it runs with added cpp support.

step03:
At './branches/HaikuVM-1.0.2-Leonardo-step03'.
- haiku.script starts with configuration step03
- HaikuVM (the JVM of Haiku) is from the trunk
=> To see if it runs with newest VM.

step04:
At './branches/HaikuVM-1.0.2-Leonardo-step04'.
- haiku.script starts with configuration step04
- took from trunk:
-- configuration leonardo.UsingCLIB
-- ArduinoUsingCLIB.cpp
-- HaikuMicroKernelLib.java
-- HelloWorldJava.java
- BlinkSimple.java should still work out-of-the-box.
- I expect HelloWorldJAva.java to run with USB too.
=> To see if initialization and io works with clib.

[Carl van Denzen]

Thank you, I think this is going to work (in the end...). Eclipse is still difficult for me, I cannot figure out what you mean with "I recommend to use 'bin:haiku(..)". I work on Windows 7, I tried in Eclipse menu Run/Execute and then different things. The menu "Run/Execute/Run as" doesn't give me any possibilities ("none applicable"). I noticed a file with a probable typo (lanunch instead of launch (?) in file .metadata.plugins. I cannot reproduce what file it was, because I renamed it.
Do you have a Windows 7 system available to test this?

[genom2]

'bin:haiku.script.launch'|right-mouse-click|'Run as'|'haiku.script'
is a shortcut for:
- open project 'bin' in "Package Explorer"
- move your mouse over file 'haiku.script.launch'
- right-mouse-click and a menu will popup
- move your mouse over item 'Run As'
- pick 'haiku.script' to launch.

Then
- this script will ask you for a Java file to make and start
- navigate to directory 'examples' and pick file 'leonardo.tutorial.BlinkSimple.java'

Remark: After this first time the script is also available through: Eclipse menu 'Run'|'External Tool'

[Carl van Denzen]

Yes, steps 01 to 04 work. I have run every step twice: once with the led On (the default in BlinkSimple.java) and once with PORTC=0. I verified that the led was off then.

I don't see the differences between the steps (e.g. I do not see a cpp or c++ in step 2).
The UsingCLIB does not compile, I will post the output. Here I do see c++ compilation.

Waiting for new instructions...

[genom2]

step02 only supports cpp without having a chance to use them simply because no cpp file was given in this branch.

step04 introduces the first cpp file.

In your attached file for HelloWorld.java I find line:
'Building file: ../../tutorials/ArduinoUsingCLIB.cpp'
This shows me a cpp compilation (as you said).
This should be the case for (well-behaved) BlinkSimple.java as well.

I guess here is the main problem: I asked for HelloWorldJava.java (sorry for the typo above) but you used HelloWorld.java!!
- HelloWorld.java still uses serial line code for duemilanove (and consequently fails on target Leonardo).
- HelloWorldJava.java should use the Arduino IDE CLIB functions for Leonardo (and hopefully works on target Leonardo).

[Carl van Denzen]

Sorry, I didn't read your recipe good enough (HelloWorldJava.java).
The results for HelloWorldJava.java are not as expected. One of the problems is that I don't know what port is going to be used by the serial line on my laptop (usb CDC, see http://arduino.cc/en/Guide/ArduinoLeonardo). When I download the program, I do not see any devices (in Windows configuration/printers and faxes. The exact names can be different, I have a Dutch Windows. When I reset the Leonardo, I see a COM port becoming available.
Maybe the display is simpler for this test:
http://www.dfrobot.com/wiki/index.php?title=Arduino_LCD_KeyPad_Shield_(SKU:_DFR0009)

I noticed a strange behaviour. Trying to debug HelloWorldJava.java, I introduced the famous code DDRC=128;PORTC=128;.
This doesn't seem to work, unless I put a
import static haiku.avr.AVRConstants.*;
in the file.
Can you explain why I don't see compile errors, but DDRC and PORTC do not work (without the import statement)?