Incorrect read long string from Serial

[TimReset]

Good day!

I upload arduino.tutorial.Echo to Arduino Nano, connect with it and write string longer 3 symbols and get result not all symbols.

Example:
Write: Hello!

Result:
Please start typing
0: H
1: e
2: l
3: !

Missing symbol "l".

I tried read from serial with System.in - result same.

I write same code on C++:

void loop() {
if (Serial.available() > 0) {
Serial.println(Serial.readString());
}
}

It's work correct.

I read source code of function Serial.readString() - it's contains "timedRead()":

int Stream::timedRead()
{
int c;
_startMillis = millis();
do {
c = read();
if (c >= 0) return c;
} while(millis() - _startMillis < _timeout);
return -1; // -1 indicates timeout
}

I wrote similar function on Java, but it's not has effect.

Do have any idea why symbols disappear?

[genom2]

Hello TimReset,

thank you for sharing your findings. You gave only little information to me. So I tried to fill up the gaps:

a) I use HaikuVM 1.3.0
b) on Windows XP
c) with my Duemilanove.
d) I do haikufy like this:
C:\haikuVM\bin\haiku -v --Config duemilanove C:\haikuVM\examples\src\main\java\arduino\tutorial\Echo.java
e) I start PUTTY with 57600 Baud to connect with my Duemilanove.

1)
Then I typed "abcd!" and got:
0: a
1: b
2: c
3: d
4: !
Every thing works like expected!

2)
Then I typed "abcd!" as fast as I can. Again, every thing works like expected!

3)
Then I cut "abcd!" and past it into PUTTY and got:
0: a
1: b
2: c
4: !
This looks like your finding. Only the first 3 characters and the last are echoed. I was very surprised to see this behavior even down to 300 Baud!?

4)
Ok, HaikuVM is slow. And may be input interferes with to much output. To test this idea I changed line
Serial.println(i+": "+(char)c);
into
Serial.print((char)c);
of Echo.java. This gave me the expected result:
abcd!

5)
But, on very long cut&past chunks of text I found missing characters again. Here I had to lower the baud rate of Echo.java to 19200. This does the trick and together with 4) finally everything worked like expected.

6)
Your processing C code can not be compared with arduino/tutorial/Echo.java. Because behind the processing scenes Serial.read() and Serial.write() are interrupt driven functions. Whereas arduino/tutorial/Echo.java does it by direct polling of IO registers.

7a)
I wrote https://sourceforge.net/p/haiku-vm/code/HEAD/tree/trunk/examples/src/main/java/duemilanove/tutorial/FastEcho.java for 57600 Baud and for very fast polling.
You have to download it because it's not included in HaikuVM 1.3.0. I do haikufy like this:
C:\haikuvm\bin\haiku -v --Config duemilanove C:\haikuVM\examples\src\main\java\duemilanove\tutorial\FastEcho.java
Which gave me perfect results with 57600 baud.

7b)
I wrote https://sourceforge.net/p/haiku-vm/code/HEAD/tree/trunk/examples/src/main/java/processing/examples/_04_Communication/Echo.java for 57600 Baud to link with the interrupt driven processing Serial.read() and Serial.write().
You have to download it because it's not included in HaikuVM 1.3.0. I do haikufy like this:
C:\haikuvm\bin\haiku -v --Config arduinoIDEUpload C:\haikuVM\examples\src\main\java\processing\examples_04_Communication\Echo.java
Not perfect on very big chunks of text but much much better than arduino/tutorial/Echo.java.

This is my picture of your findings:
First, characters disappear because HaikuVM 1.3.0 is a bit to slow for a burst of characters in 56700 Baud.
Second, writing polling IO in a tight loop or using interrupt based IO will do the job.
Third, I have no idea why arduino/tutorial/Echo.java exhibits this bad behavior even down to 300 Baud.

Kind regards
Bob Genom

[TimReset]

Great thanks for detail research!

Forgot to say - I connected to Arduino and send data through Arduino IDE "Serial Monitor".

I saw what you wrote. I test paragraph 7a and it's work! And I wrote method with it's approach:

    static String readString() {
        final int maxLen = 100;
        char[] chars = new char[maxLen];
        int pos = 0;
        while (pos < maxLen) {

            int startMillis = 0;
            while (((UCSR0A & (1 << RXC0)) == 0) && (startMillis < 1000)) {
//use increment but not currentTime() for speed
                startMillis++;
            };
            if ((UCSR0A & (1 << RXC0)) == 0) {
                break;
            }
            chars[pos] = (char) UDR0;
            pos++;
        }
        return new String(chars, 0, pos);
    }

It's working for 100 length of string max and speed 57600 Baud.
I tested for maxLen = 500 but it's work for length of string by 79 characters max. For greater string the program hanging :( And I don't know why.

[genom2]

I have an explanation, that your finding exhibits down to 300 Baud. The reason is: The behavior is independent from the baud rate! For this you have to assume, that the AVR has a hardware buffer of size 2.

This is what happens if a burst of characters e.g. "abcd!" comes in:

IN    ACTION
a
    read a
    write 0
b   into AVR hardware buffer
    write :
    write \s
c   into AVR hardware buffer
    write a
    write \r
d   lost because hardware buffer overflow
    write \n

    read b
!   into AVR hardware buffer
    write 1
    write :
    write \s
    write b
    write \r
    write \n

    read c
    write 2
    write :
    write \s
    write c
    write \r
    write \n

    read !
    write 3
    write :
    write \s
    write !
    write \r
    write \n

It is not a perfect explanation but very close, I think.

Back to your last problem:
"For greater string the program hanging :( And I don't know why."

I guess the reason is heap fragmentation. After a while of allocating and collecting this

char[] chars = new char[500];
together with
new String(chars, 0, pos)

There is no contiguous 500 char space on the heap.

Try to make it static:

static char[] chars = new char[500];

[TimReset]

I tried use static:

final static int maxLen = 500;
final static char[] chars = new char[maxLen];

It's not work.