amforth-devel

[Amforth] uploading files

[Robert E. <epp...@so...>]

Slowly progressing towards using amforth on an arduino uno board.
The basic system is up, runs fine with screen /dev/ttyACM0
but I cannot upload files.

amforth-upload.py:
First I had to replace 'python2' in the sheebang line of
amforth-upload.py by 'python2.7' 'python2.6' or 'python'
to have it run on debian/wheezy.

But now if i try:
	amforth-upload.py -v -t /dev/ttyACM0 lib/misc.frt
	including file: './lib/misc.frt'
	sending line: \ some definitions that may be useful

	amforth 4.6 ATmega328P
	>

Then it hangs... (independent of python version)

The second last output line very often starts with something like
'amamforth' repeating a couple of characters of 'amforth', usually two.

What might be the problem?

I could not find ascii-xfr to use as a temporary workaround.
So how can I upload files to amforth now?
Robert

Re: [Amforth] uploading files

[Matthias T. <mt...@we...>]

Hello Robert,

I cannot test with an recent arduino (the ttyACM device tells
me this), so only one remark.

> Slowly progressing towards using amforth on an arduino uno board.
> The basic system is up, runs fine with screen /dev/ttyACM0
> but I cannot upload files.

the python script requires that no other process accesses the
serial line. You cannot watch it working.

Another tool to use is am4up with minicom (a small
C program in the tools directory). See at the sources on
how to use it. It implements the same communication
logic as the python script but *within* minicom. The downside
is that it does not honor the include commands. Nobody's
perfect and patches are very welcome.

> amforth-upload.py:
> First I had to replace 'python2' in the sheebang line of
> amforth-upload.py by 'python2.7' 'python2.6' or 'python'
> to have it run on debian/wheezy.

Blame the debian people.

Matthias

Re: [Amforth] uploading files

[Robert E. <epp...@so...>]

Hi Matthias

Matthias Trute <mt...@we...> writes:

> I cannot test with an recent arduino
Did others use it on the uno?

>> runs fine with screen /dev/ttyACM0
>> but I cannot upload files.

> the python script requires that no other process accesses the
> serial line. You cannot watch it working.

Yes, I assumed that and always terminated screen before trying to
upload.


> Another tool to use is am4up with minicom (a small
> C program in the tools directory). See at the sources on
> how to use it.
Ah, thank you. Another thing to try.

> The downside is that it does not honor the include commands.
No problem. I think putting the include logic into the upload
tool might be questionable design anyway.

Thank you,
Robert

Re: [Amforth] uploading files

[Erich W. <ew....@na...>]

On 01/13/2012 07:59 PM, Robert Epprecht wrote:
> Matthias Trute<mt...@we...>  writes:

>> Another tool to use is am4up with minicom (a small
>
>> The downside is that it does not honor the include commands.
> No problem. I think putting the include logic into the upload
> tool might be questionable design anyway.


ACK! I added these lines into the makefile

TARGET=main
CONSOLE=/dev/ttyUSB0
...

upload_file:
         cat $(TARGET).fs | unfold_fs > $(TARGET).fs.unfold
         trim_fs $(TARGET).fs.unfold  > $(TARGET).fs.upload
upload: upload_file
         amforth-upload.py  -t ${CONSOLE} $(TARGET).fs.upload


to produce the following commands

$ make -n upload
cat main.fs | unfold_fs > main.fs.unfold
trim_fs main.fs.unfold  > main.fs.upload
amforth-upload.py  -t /dev/ttyUSB0 main.fs.upload

unfold_fs treats all include directives adding comment lines
(begin end of file X) and the content of file X into the
output stream. trim_fs removes all comments of form
\ ... line
or
  ... ( comment ) ...
and it compresses consecutive spaces, possibly in undesired places.


Please find the two scripts below for anyone's use
Comments and patches are, of course, welcome.

Cheers,
Erich


# ----------------------------------------------------------------
$ cat ~/bin/unfold_fs
#!/usr/bin/perl -w
# 2007-12-08 EW
#
# filter, read all input, expand lines like
#   "^.include filename$"
# with the content found in filename
#
# this is written unfold amforth projects before
# transferring them to the target controller.

use strict;
my $Prog=$0;
my $file="";

my $pattern='^[#]*include\s+(\S+)';
my $comment='\ ---';

my $verbose=0;

while (<>)
{
     if (m/$pattern/)
     {
	$file = "$1";
	print STDERR "  $file\n" if ($verbose);
	if (not -r $file)
	{
	    print STDERR "$Prog: file $file not found!\n";
	    exit 2;
	}
	print "$comment include begin $file\n";
	system("$Prog $file") == 0 or  die "failed.\n";
	print "$comment include end $file\n";
     }
     else
     {
	print $_;
     }
}

# ----------------------------------------------------------------
$ cat ~/bin/trim_fs
#!/usr/bin/perl -w
# 2007-12-08 EW
# trim_fs
#
# filter, remove comments, leading and trailing whitespace and empty lines
#
# this is written to trim unfolded amforth projects

use strict;
my $Prog=$0;

while (<>)
{
     chomp;
     s/\\[ \t].*$//;		# remove line comments
     s/\( .* \)//;	# remove inline comments

     #s/^\s+//;		# delete leading whitespace / well no, keep indendation
     s/\s+$//;		# delete trailing whitespace
     s/\b\s+/ /g;	# compress white space after first word
     next if /^$/;	# delete empty lines

     print "$_\n";
}

# ----------------------------------------------------------------

Re: [Amforth] uploading files

[Robert E. <epp...@so...>]

Matthias Trute <mt...@we...> writes:

> Another tool to use is am4up with minicom (a small
> C program in the tools directory). See at the sources on
> how to use it.

Did that, but still no luck. It does not work with my UNO.

Robert

Re: [Amforth] uploading files

[Erich W. <ew....@na...>]

Hi,

On 01/13/2012 02:12 PM, Robert Epprecht wrote:
>
> But now if i try:
> 	amforth-upload.py -v -t /dev/ttyACM0 lib/misc.frt
> 	including file: './lib/misc.frt'
> 	sending line: \ some definitions that may be useful
>
> 	amforth 4.6 ATmega328P
> 	>
>
> Then it hangs... (independent of python version)

I can confirm this behaviour.

I think its interesting to note the timing stuff:

1.
install .hex files on controller with programmer
The controller is put into reset state and programmed.
At the end, the reset is released, so

2.
the controller goes out of reset and starts the amforth
system. The last thing it does is printing out

   amforth 4.6 ATmega328P
   >

The first line is the output of ver. The second line is
the ok prompt. Nothing new here.

HOWEVER, at this point in time you are NOT connected
with amforth-upload.py. (You may be connected with minicom,
then you see this output).

3.
Now (assuming no minicom or other terminal is connected)
amforth-upload.py opens the serial port and reads the above
characters *again*. They are not sent, when amforth-upload.py
opens the serial port. I suspect, that there is a fifo-buffer
in the usb-serial stuff not being cleared.

One key difference between the duemilanove and the uno is its
connection from serial (controller) to the USB plug of the board.

4.
amforth-upload.py is confused from this point on, because it
sent some character, a '\' say, and it does not read back this
backslash (the echoed response from amforth). The program will
stall at this point.

Matthias has suggested to add "dest.flush()"
after line 103 of this snippet
     102         for o in list(string):
     103                 dest.write(o);
     104                 dest.flush();    #<--- added here
     105                 if o == "\t":
     106                         o = " "
     107
     108                 while True:


But I never came around to test this. Do you want to try that?

If this does not help, maybe add some code, that will read all
available data and discard it *before* sending the first character.


Cheers,
Erich

Re: [Amforth] uploading files on arduino UNO

[Erich W. <ew....@na...>]

Hi,

I just remembered that I received a package of stuff the other
day and got out a brand new uno ...

>
> 2.
> the controller goes out of reset and starts the amforth
> system. The last thing it does is printing out
>
> amforth 4.6 ATmega328P
>  >
>
> The first line is the output of ver. The second line is
> the ok prompt. Nothing new here.
>
> HOWEVER, at this point in time you are NOT connected
> with amforth-upload.py. (You may be connected with minicom,
> then you see this output).
>
> 3.
> Now (assuming no minicom or other terminal is connected)
> amforth-upload.py opens the serial port and reads the above
> characters *again*. They are not sent, when amforth-upload.py
> opens the serial port. I suspect, that there is a fifo-buffer
> in the usb-serial stuff not being cleared.

This may be wrong. I observe that connecting to the Uno with
minicom will *always* produce this output.

amforth 4.6 ATmega328P Forthduino
 >

Whereas a duemilanove will not.

As a quick test I added 37 statements (I didn't grok how to write a
for(i=0; i<37; i++) { read one char; } statement in python ;-)

         i = tty_dev.read(1)

between line
	tty_dev = file(tty_dev_name,"r+",0)

and lines
	if len(args)<1:
		if verbose:
			print >>sys.stderr, "processing stdin"
		write_file_flow(sys.stdin,tty_dev)
	else:

And *drum roll*
$ make marker CONSOLE=/dev/ttyACM0
amforth-upload.py  -t /dev/ttyACM0 lib/misc.frt
: erase
  ok  0 fill
  ok;
  ok
 > : .(
  ok   [char] ) word count type
word ?? -13 16
 >

Well almost. It fails on something else now.


> One key difference between the duemilanove and the uno is its
> connection from serial (controller) to the USB plug of the board.

So it seems to me, connecting to the uno will cause it to reset.
Strange concept.


Cheers,
Erich

Re: [Amforth] uploading files on arduino UNO

[Matthias T. <mt...@we...>]

Hi,

> This may be wrong. I observe that connecting to the Uno with
> minicom will *always* produce this output.
> 
> amforth 4.6 ATmega328P Forthduino
>  >

>From the arduino website

"When the Uno is connected to either a computer running Mac OS X or
Linux, it resets each time a connection is made to it from software (via
USB)."

That means, the upload utility needs to flush the input buffer
everytime it starts. Strange hardware design.

Doable, but not this evening ;)

Matthias

Re: [Amforth] uploading files

[Robert E. <epp...@so...>]

Erich Waelde <ew....@na...> writes:

> 4.
> amforth-upload.py is confused from this point on, because it
> sent some character, a '\' say, and it does not read back this
> backslash (the echoed response from amforth). The program will
> stall at this point.
>
> Matthias has suggested to add "dest.flush()"
> after line 103 of this snippet
>      102         for o in list(string):
>      103                 dest.write(o);
>      104                 dest.flush();    #<--- added here
>      105                 if o == "\t":
>      106                         o = " "
>      107
>      108                 while True:
>
>
> But I never came around to test this. Do you want to try that?

did not help.
I got 'amfoamforth 4.6 ATmega328P' this time, keeps changing.

Robert

Re: [Amforth] uploading files on arduino UNO

[Robert E. <epp...@so...>]

Erich Waelde <ew....@na...> writes:

>> One key difference between the duemilanove and the uno is its
>> connection from serial (controller) to the USB plug of the board.

> So it seems to me, connecting to the uno will cause it to reset.
correct

> Strange concept.

As far as I know all arduinos do that on linux and maybe osx.
If it does not show up under some conditions (like a given 
arduino model) i think it might be a timing hazard.

Robert

Re: [Amforth] uploading files on arduino UNO

[Erich W. <ew....@na...>]

Hello,

On 01/14/2012 08:12 AM, Robert Epprecht wrote:
> Erich Waelde<ew....@na...>  writes:
>
>>> One key difference between the duemilanove and the uno is its
>>> connection from serial (controller) to the USB plug of the board.
>
>> So it seems to me, connecting to the uno will cause it to reset.
> correct
>
>> Strange concept.
>
> As far as I know all arduinos do that on linux and maybe osx.
> If it does not show up under some conditions (like a given
> arduino model) i think it might be a timing hazard.

No, this feature has been introduced at some point as
"Automatic (Software) Reset" it seems. It is definitely absent on
duemilanove boards.


One way to eliminate this behaviour is to cut the trace (Leiterbahn)
between the two pads labeled "RESET EN".


As for the amforth-upload.py script, I assume that reading characters
until the ok-prompt is seen *before* sending bytes would help as well.
However, the automatic software reset implies that I cannot connect
to an already running controller (something I routinely do) without
causing a reset. I can see, why they did this, and I'm grateful that
cutting a trace will solve this for me. YMMV.

Cheers,
Erich



http://store.nkcelectronics.com/arduino-diecimila.html

	|Automatic (Software) Reset
	|
	|Rather than requiring a physical press of the reset button before
	|an upload, the Arduino Uno is designed in a way that allows it to
	|be reset by software running on a connected computer. One of the
	|hardware flow control lines (DTR) of the ATmega8U2 is connected to
	|the reset line of the ATmega328 via a 100 nanofarad capacitor.
	|When this line is asserted (taken low), the reset line drops long
	|enough to reset the chip. The Arduino software uses this
	|capability to allow you to upload code by simply pressing the
	|upload button in the Arduino environment. This means that the
	|bootloader can have a shorter timeout, as the lowering of DTR can
	|be well-coordinated with the start of the upload.
	|
	|This setup has other implications. When the Uno is connected to
	|either a computer running Mac OS X or Linux, it resets each time a
	|connection is made to it from software (via USB). For the
	|following half-second or so, the bootloader is running on the Uno.
	|While it is programmed to ignore malformed data (i.e. anything
	|besides an upload of new code), it will intercept the first few
	|bytes of data sent to the board after a connection is opened. If a
	|sketch running on the board receives one-time configuration or
	|other data when it first starts, make sure that the software with
	|which it communicates waits a second after opening the connection
	|and before sending this data.
	|
	|The Uno contains a trace that can be cut to disable the
	|auto-reset. The pads on either side of the trace can be soldered
	|together to re-enable it. It's labeled "RESET-EN". You may also be
	|able to disable the auto-reset by connecting a 110 ohm resistor
	|from 5V to the reset line; see this forum thread for details.

Re: [Amforth] uploading files on arduino UNO

[Robert E. <epp...@so...>]

Erich Waelde <ew....@na...> writes:

> 	|Automatic (Software) Reset

> 	|You may also be able to disable the auto-reset by connecting
> 	|a 110 ohm resistor from 5V to the reset line

Yes, I have heard of this trick and tried it. I was not able to get
the desired result on my uno board yet. Will do more experiments...

Robert

Re: [Amforth] uploading files on arduino UNO

[Matthias T. <mt...@we...>]

Hi all,

> 
> As for the amforth-upload.py script, I assume that reading characters
> until the ok-prompt is seen *before* sending bytes would help as well.

I just added a code line that *could* help. It simply sends an empty
line to the conroller and waits for the ok and the prompt before
staring the actual upload (just like pressing enter upon serial line
connect). Let me know if it helps.

It is helpful anyway to ensure a proper input state.

Matthias

Re: [Amforth] uploading files on arduino UNO

[Robert E. <epp...@so...>]

Erich Waelde <ew....@na...> writes:
> On 01/14/2012 08:12 AM, Robert Epprecht wrote:
>>>> One key difference between the duemilanove and the uno is its
>>>> connection from serial (controller) to the USB plug of the board.
>>
>>> So it seems to me, connecting to the uno will cause it to reset.
>>> Strange concept.
>>
>> As far as I know all arduinos do that on linux and maybe osx.
>> If it does not show up under some conditions (like a given
>> arduino model) i think it might be a timing hazard.
>
> No, this feature has been introduced at some point as
> "Automatic (Software) Reset" it seems. It is definitely absent on
> duemilanove boards.

Maybe I'm confusing something, but my duemilanove board running an
arduino sketch *does* restart the sketch when connecting to it
through serial (just double tested). So I had the impression that
this was a reset.

Robert

Re: [Amforth] uploading files on arduino UNO

[Torsten S. <tsa...@gm...>]

Yes, the DTR line triggers the reset. In the UNO schematic the pin on the (r3) 16u2 is labeled CTS. Could this be the reason?

Cheers,
Torsten

Am 14.01.2012 um 20:44 schrieb Robert Epprecht:

> Erich Waelde <ew....@na...> writes:
>> On 01/14/2012 08:12 AM, Robert Epprecht wrote:
>>>>> One key difference between the duemilanove and the uno is its
>>>>> connection from serial (controller) to the USB plug of the board.
>>> 
>>>> So it seems to me, connecting to the uno will cause it to reset.
>>>> Strange concept.
>>> 
>>> As far as I know all arduinos do that on linux and maybe osx.
>>> If it does not show up under some conditions (like a given
>>> arduino model) i think it might be a timing hazard.
>> 
>> No, this feature has been introduced at some point as
>> "Automatic (Software) Reset" it seems. It is definitely absent on
>> duemilanove boards.
> 
> Maybe I'm confusing something, but my duemilanove board running an
> arduino sketch *does* restart the sketch when connecting to it
> through serial (just double tested). So I had the impression that
> this was a reset.
> 
> Robert
> 
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Mar 27 - Feb 2
> Save $400 by Jan. 27
> Register now!
> http://p.sf.net/sfu/rsa-sfdev2dev2
> _______________________________________________
> Amforth-devel mailing list for http://amforth.sf.net/
> Amf...@li...
> https://lists.sourceforge.net/lists/listinfo/amforth-devel

Re: [Amforth] uploading files on arduino UNO

[Robert E. <epp...@so...>]

Matthias Trute <mt...@we...> writes:

>> As for the amforth-upload.py script, I assume that reading characters
>> until the ok-prompt is seen *before* sending bytes would help as well.
>
> I just added a code line that *could* help. It simply sends an empty
> line to the conroller and waits for the ok and the prompt before
> staring the actual upload (just like pressing enter upon serial line
> connect). Let me know if it helps.

Thanks a lot. Alas it is not working yet:

dada@i3:~/AMFORTH/amforth-svn/amforth/trunk$ ./tools/amforth-upload.py -v \
	-t /dev/ttyACM0 lib/misc.frt

skipping empty line

including file: './lib/misc.frt'
sending line: \ some definitions that may be useful

Then it hangs.

Trying to guess what the script does (i don't talk python) it looks
as if the empty line is filtered out and not sent to the avr.

Robert

Re: [Amforth] uploading files on arduino UNO

[Matthias T. <mt...@we...>]

Robert,

>>> As for the amforth-upload.py script, I assume that reading characters
>>> until the ok-prompt is seen *before* sending bytes would help as well.
>>
>> I just added a code line that *could* help. It simply sends an empty
>> line to the conroller and waits for the ok and the prompt before
>> staring the actual upload (just like pressing enter upon serial line
>> connect). Let me know if it helps.
> 
> Thanks a lot. Alas it is not working yet:

I've got an major update for the amforth-shell.py
script, maybe you could give it a try as well.

amforth-shell.py -u <file-to-upload.frt> etc



Matthias

Re: [Amforth] uploading files on arduino UNO

[Robert E. <epp...@so...>]

Matthias Trute <mt...@we...> writes:

> I've got an major update for the amforth-shell.py
> script, maybe you could give it a try as well.
>
> amforth-shell.py -u <file-to-upload.frt> etc

As I already told you, this works fine with arduino uno now.

suggestion:
Lines starting with '\' are sent as emty lines now.
I think it would be worth suppressing them.

Robert

Re: [Amforth] uploading files on arduino UNO

[Robert E. <epp...@so...>]

Matthias Trute <mt...@we...> writes:

> I've got an major update for the amforth-shell.py script
oh, thank you

> maybe you could give it a try as well.
> amforth-shell.py -u <file-to-upload.frt> etc

svn r1140

[with truncated path names]
amforth-shell.py -v -t /dev/ttyACM0 -u some-file
Traceback (most recent call last):
  File "amforth-shell.py", line 11, in <module>
    import fdpexpect
ImportError: No module named fdpexpect

Is this a debian python installation problem?

Robert

Re: [Amforth] uploading files on arduino UNO

[Matthias T. <mt...@we...>]

Robert,

>     import fdpexpect
> ImportError: No module named fdpexpect
> 
> Is this a debian python installation problem?

Most probably. My ubuntu 11.10 has the
package

> dpkg -l | grep python | grep exp
ii  python-pexpect                                2.3-1ubuntu

installed automatically, IIRC.

HTH

Matthias

Re: [Amforth] uploading files on arduino UNO

[Robert E. <epp...@so...>]

Matthias Trute <mt...@we...> writes:

>> ImportError: No module named fdpexpect

> ii  python-pexpect                                2.3-1ubuntu

Yes, that was the problem here.

Uploading a very short first testfile to the uno with amforth-shell.py went fine now :)

Thanks a lot,
Robert