amforth-devel

[Amforth] Wish to FORGET

[Enoch <ix...@ho...>]

Hello Matthias & all,

Here's something beyond me in Amforth, please help:

I wrote a table driven CRC-8 routine for short messages using any
generating polynomial, see http://pastebin.com/1YtVdMeg
(I didn't like the slow approach of lib/hardware/1wire-crc8.frt).

The Flash look-up table, crc8tab, is created at compile time -- first
forming 256 PAD bytes and then packing them into 128 Flash words. DOES>
provides the run-time lookup convenience.

My question is simple:

How can I reclaim the Flash memory of the crc8_msb_table word itself?

(1) Our Tools Ext don't include FORGET.

(2) I could use Core Ext MARKER if we had a deferred facility of
sort. I.e., create a deferred table and fill it using a transient
crc8_msb_table (which MARKER would afterwards help to remove).

Among Forth main strengths is the ability to change itself at run-time
(aka "reflection"). Can we apply it without the penalty of accumulating
redundant code?

Thanks, Enoch.

Re: [Amforth] Wish to FORGET

[Matthias T. <mt...@we...>]

Hi Enoch,


> How can I reclaim the Flash memory of the crc8_msb_table word itself?
> 
> (1) Our Tools Ext don't include FORGET.
> 
> (2) I could use Core Ext MARKER if we had a deferred facility of
> sort. I.e., create a deferred table and fill it using a transient
> crc8_msb_table (which MARKER would afterwards help to remove).

There is no simple way. I did not actually test it, but I'd
start as follows. First define the dictionary header for the data
(create), allocate the flash room for it (dp 128 + to dp), now the
marker and finally all the words that you want to dispose after filling
the data flash area (use !i, not , ). Hmm. sounds not only difficult, it
is indeed.

> Among Forth main strengths is the ability to change itself at run-time
> (aka "reflection"). Can we apply it without the penalty of accumulating
> redundant code?

The major problem (and the reason why I dropped FORGET) is
fragmentation. The current memory "management" is the trivial
append/remove approach. The last time FORGET worked for the
dictionary was until the introduction of wordlists. A flash
memory manager that can deal with fragmentation is unlikly
to earn its own size (your CRC table could be pre-computed
on the host side as well and only the resulting table goes
to the controller. Just like the magic numbers in lib/sinus.frt).

Matthias
PS: how long last the pastebin files? They look good.

Re: [Amforth] Wish to FORGET

[Enoch <ix...@ho...>]

Hello Matthias,

Matthias Trute <mt...@we...> writes:

> Hi Enoch,
>
>
>> How can I reclaim the Flash memory of the crc8_msb_table word itself?
>> 
>> (1) Our Tools Ext don't include FORGET.
>> 
>> (2) I could use Core Ext MARKER if we had a deferred facility of
>> sort. I.e., create a deferred table and fill it using a transient
>> crc8_msb_table (which MARKER would afterwards help to remove).
>
> There is no simple way. I did not actually test it, but I'd
> start as follows. First define the dictionary header for the data
> (create), allocate the flash room for it (dp 128 + to dp), now the
> marker and finally all the words that you want to dispose after filling
> the data flash area (use !i, not , ). Hmm. sounds not only difficult, it
> is indeed.
>
>> Among Forth main strengths is the ability to change itself at run-time
>> (aka "reflection"). Can we apply it without the penalty of accumulating
>> redundant code?
>
> The major problem (and the reason why I dropped FORGET) is
> fragmentation. The current memory "management" is the trivial
> append/remove approach. The last time FORGET worked for the
> dictionary was until the introduction of wordlists. A flash
> memory manager that can deal with fragmentation is unlikly
> to earn its own size (your CRC table could be pre-computed
> on the host side as well and only the resulting table goes
> to the controller. Just like the magic numbers in lib/sinus.frt).

Indeed, your suggested use of MARKER is what I anticipated but I have to
agree with you, it would be too difficult/dangerous for an average
programmer to try.

Hence, let us revisit FORGET. I agree with you again that with Flash
memory "de-fragmemetation" is not an option. However, there is a
solution if you agree to modify the dictionary structure just a bit. I
believe that you are familiar with the various "dynamic storage
allocation" algorithms (Knuth, Vol 1, Fundamental Algorithms, 2.5
Dynamic Storage Allocation). What if we introduce one bit for each
definition which means "FREE" or "IN-USE". FORGET would become: (1) Mark
the definition as FREE (2) if possible, merge this newly freed Flash
space with adjacent Flash free definitions. For the purpose of this
algorithm the "unused dictionary space" would be just another free
definition space on our linked list.  When compiling, we can compile
into the largest free definition space or choose another strategy.

By the way, I don't agree with Forth-RC1 comment on FORGET which says as
follows: "This word is obsolescent and is included as a concession to
existing implementations."

Because:

(1) In Flash restricted situations what other solution there is for the
program to change itself while running...

(2) How else can one implement "Dynamic Software Updating"
<http://en.wikipedia.org/wiki/Dynamic_Software_Updating>.

As I am not a Forth expert (yet), did any other flash based Forth
introduce FORGET?

Regards, Enoch.

>
> Matthias
> PS: how long last the pastebin files? They look good.

Thanks, 4 more weeks but I promised to improve upon it.



>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb

Re: [Amforth] Wish to FORGET

[Matthias T. <mt...@we...>]

Hi,

> Indeed, your suggested use of MARKER is what I anticipated but I have to
> agree with you, it would be too difficult/dangerous for an average
> programmer to try.

I wrote "difficult" not "too difficult" A subtle difference.

> Hence, let us revisit FORGET.
> I agree with you again that with Flash memory "de-fragmemetation" is
> not an option. However, there is a solution if you agree to modify
> the dictionary structure just a bit. I believe that you are familiar
> with the various "dynamic storage allocation" algorithms (Knuth, Vol
> 1, Fundamental Algorithms, 2.5 Dynamic Storage Allocation). What if
> we introduce one bit for each definition which means "FREE" or
> "IN-USE". FORGET would become: (1) Mark the definition as FREE (2) if
> possible, merge this newly freed Flash space with adjacent Flash free
> definitions. For the purpose of this algorithm the "unused dictionary
> space" would be just another free definition space on our linked
> list.  When compiling, we can compile into the largest free
> definition space or choose another strategy.

That may work. The only problem I have with it is: A colon
definition does not know how much space it needs in advance.
So how do you decide, which fragment do you use with a particular
definition? And what do you do, if such an limit is reached?

> By the way, I don't agree with Forth-RC1 comment on FORGET which
> says as follows: "This word is obsolescent and is included as a
> concession to existing implementations."

I have absolutely no problem that FORGET will be forgotten.

> (2) How else can one implement "Dynamic Software Updating" 
> <http://en.wikipedia.org/wiki/Dynamic_Software_Updating>.

Do you really have non-disruptive software upgrades in mind?
That is so much far beyond the scope of my little toy,  that
I even decline any ambitions in this direction.

> As I am not a Forth expert (yet), did any other flash based Forth 
> introduce FORGET?

I assume, most of them do not have word lists and hence do not have
the problems arising from them ;) There are a few members
of this list, that may tell better.

Matthias

Re: [Amforth] Wish to FORGET

[Enoch <ix...@ho...>]

Matthias Trute <mt...@we...> writes:

> Hi,
>
>> Indeed, your suggested use of MARKER is what I anticipated but I have to
>> agree with you, it would be too difficult/dangerous for an average
>> programmer to try.
>
> I wrote "difficult" not "too difficult" A subtle difference.
>
>> Hence, let us revisit FORGET.
>> I agree with you again that with Flash memory "de-fragmemetation" is
>> not an option. However, there is a solution if you agree to modify
>> the dictionary structure just a bit. I believe that you are familiar
>> with the various "dynamic storage allocation" algorithms (Knuth, Vol
>> 1, Fundamental Algorithms, 2.5 Dynamic Storage Allocation). What if
>> we introduce one bit for each definition which means "FREE" or
>> "IN-USE". FORGET would become: (1) Mark the definition as FREE (2) if
>> possible, merge this newly freed Flash space with adjacent Flash free
>> definitions. For the purpose of this algorithm the "unused dictionary
>> space" would be just another free definition space on our linked
>> list.  When compiling, we can compile into the largest free
>> definition space or choose another strategy.
>
> That may work. The only problem I have with it is: A colon
> definition does not know how much space it needs in advance.
> So how do you decide, which fragment do you use with a particular
> definition? And what do you do, if such an limit is reached?
>
>> By the way, I don't agree with Forth-RC1 comment on FORGET which
>> says as follows: "This word is obsolescent and is included as a
>> concession to existing implementations."
>
> I have absolutely no problem that FORGET will be forgotten.
>
>> (2) How else can one implement "Dynamic Software Updating" 
>> <http://en.wikipedia.org/wiki/Dynamic_Software_Updating>.
>
> Do you really have non-disruptive software upgrades in mind?
> That is so much far beyond the scope of my little toy,  that
> I even decline any ambitions in this direction.

Matthias, 

Please don't call your work a toy! Before landing here I even considered
a commercial package such as SwiftX for AVR and I found it, IMHO,
inferior.

Regarding the "forget" question. The simplest approach is to select each
new deinition the largest available free space fragement to compile
into. If the Forth programmer is well trained he/she factors his/her
code into small pieces and reutilization of "forgetten" definitions
would be high.

Anyway, that's not a high priority issue for me now, it's part of my
exploration of my new "ecosystem" :-)

Regards, Enoch.





>
>> As I am not a Forth expert (yet), did any other flash based Forth 
>> introduce FORGET?
>
> I assume, most of them do not have word lists and hence do not have
> the problems arising from them ;) There are a few members
> of this list, that may tell better.
>
> Matthias
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb

Re: [Amforth] Wish to FORGET

[Enoch <ix...@ho...>]

Hello Matthias,

Matthias Trute <mt...@we...> writes:

> Hi,
>
>> Indeed, your suggested use of MARKER is what I anticipated but I have to
>> agree with you, it would be too difficult/dangerous for an average
>> programmer to try.
>
> I wrote "difficult" not "too difficult" A subtle difference.
>
>> Hence, let us revisit FORGET.
>> I agree with you again that with Flash memory "de-fragmemetation" is
>> not an option. However, there is a solution if you agree to modify
>> the dictionary structure just a bit. I believe that you are familiar
>> with the various "dynamic storage allocation" algorithms (Knuth, Vol
>> 1, Fundamental Algorithms, 2.5 Dynamic Storage Allocation). What if
>> we introduce one bit for each definition which means "FREE" or
>> "IN-USE". FORGET would become: (1) Mark the definition as FREE (2) if
>> possible, merge this newly freed Flash space with adjacent Flash free
>> definitions. For the purpose of this algorithm the "unused dictionary
>> space" would be just another free definition space on our linked
>> list.  When compiling, we can compile into the largest free
>> definition space or choose another strategy.
>
> That may work. The only problem I have with it is: A colon
> definition does not know how much space it needs in advance.
> So how do you decide, which fragment do you use with a particular
> definition? And what do you do, if such an limit is reached?
>
>> By the way, I don't agree with Forth-RC1 comment on FORGET which
>> says as follows: "This word is obsolescent and is included as a
>> concession to existing implementations."
>
> I have absolutely no problem that FORGET will be forgotten.

Here's an interesting article by one of Forth greats:
"Proposal: un-obsolete FORGET"
http://www.forth200x.org/unobsolete-forget.txt

The author sees FORGET as a useful tool for hot system upgrade. He
writes:

"""
Typical use
===========

FORGET is useful when a human redefines an existing
word in the command line, finds a bug, and wants to
correct it:
"""

By the way, there are "marketing reasons" for Amforth to acquire
FORGET-fullness :-)

Regards, Enoch.


>
>> (2) How else can one implement "Dynamic Software Updating" 
>> <http://en.wikipedia.org/wiki/Dynamic_Software_Updating>.
>
> Do you really have non-disruptive software upgrades in mind?
> That is so much far beyond the scope of my little toy,  that
> I even decline any ambitions in this direction.
>
>> As I am not a Forth expert (yet), did any other flash based Forth 
>> introduce FORGET?
>
> I assume, most of them do not have word lists and hence do not have
> the problems arising from them ;) There are a few members
> of this list, that may tell better.
>
> Matthias
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb

Re: [Amforth] Wish to FORGET

[Matthias T. <mt...@we...>]

Hi Enoch,

>> I have absolutely no problem that FORGET will be forgotten.
> 
> Here's an interesting article by one of Forth greats:
> "Proposal: un-obsolete FORGET"
> http://www.forth200x.org/unobsolete-forget.txt
> 
> The author sees FORGET as a useful tool for hot system upgrade.

Compare FIND with MARKER

Find name, then delete name from the dictionary along with all words
added to the dictionary after name.

vs.

Remove the definition of name and all subsequent definitions.

As far as I can tell, there is no difference. Both remove not
only the word affected but all what's defined afterwards (which
somehow contradicts what you want to do, IIRC).

> FORGET is useful when a human redefines an existing
> word in the command line, finds a bug, and wants to
> correct it:

FORGET has only one advantage: It does not need planning.

My problem with FORGET arises from the sentence

"If the Search-Order word set is present, FORGET searches the
compilation word list."

And the other word lists? That is IMHO the source for
fragmentation.

> By the way, there are "marketing reasons" for Amforth to acquire
> FORGET-fullness :-)

LOL. The most ultimate reason...


Matthias

PS: did you note the dates? Michael wrote his statement in 2009,
the forth2012RC was published 3 years later and FORGET is not
re-animated there.

PPS: I'm not fully and finally against FORGET, I simply won't implement
it. And probably never include it into amforth.

Re: [Amforth] Wish to FORGET

[Enoch <ix...@ho...>]

Matthias Trute <mt...@we...> writes:

> PPS: I'm not fully and finally against FORGET, I simply won't implement
> it. And probably never include it into amforth.

Hello Matthias,

That's ok for now :-) 

I solved the problem of removing table generators with marker,
communicating their output to unaffected code via PAD.

BTW, how do you like my latest amforth-shell patch? I use appl_defs.frt
as a project wide configuration file including, for example, a DEBUG
flag that introduces various tests to the code. In this context, I am
waiting for your new [if] [else] [then] implementation. 

Thanks + Regards, Enoch.

Re: [Amforth] Wish to FORGET

[Erich W. <ew....@na...>]

Enoch,

On 03/03/2013 05:22 AM, Enoch wrote:
> I wrote a table driven CRC-8 routine for short messages using any
> generating polynomial, seehttp://pastebin.com/1YtVdMeg
> (I didn't like the slow approach of lib/hardware/1wire-crc8.frt).

If you want me to look at your code, you better include it in
the message. pastebin creates an external dependency, which does
imho not fit with mailing lists.

Erich

Re: [Amforth] Wish to FORGET

[Enoch <ix...@ho...>]

Hi Erich,

While my CRC-8 code may be of interest I did not want it to become the
focus of conversation and this is why I posted the code as a
"pastebin". My focus was on Forth ability to modify the program
structure during program execution and indeed Matthias has addressed
this very topic.

I intend to contribute my CRC-8 code once it is completed with
crc8_lsb_table construction. It is my opinion that full length code
should not be posted on any mailing list. It is what pasebin was
invented for <http://en.wikipedia.org/wiki/Pastebin> (*). If you have
another "pastebin" site to suggest I am open to hear.

The code at <http://pastebin.com/1YtVdMeg> will expire in 4 weeks. I
hope to submit the final version for review beforehand. You are
*welcome* to comment if you find my code style or approach deficient.

Regards, Enoch.

(*) Nautilus, GNOME file manager, has an extension that makes pastebin
uploads (very) easy.


Erich Waelde <ew....@na...> writes:

> Enoch,
>
> On 03/03/2013 05:22 AM, Enoch wrote:
>> I wrote a table driven CRC-8 routine for short messages using any
>> generating polynomial, seehttp://pastebin.com/1YtVdMeg
>> (I didn't like the slow approach of lib/hardware/1wire-crc8.frt).
>
> If you want me to look at your code, you better include it in
> the message. pastebin creates an external dependency, which does
> imho not fit with mailing lists.
>
> Erich
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb

Re: [Amforth] Wish to FORGET

[Erich W. <ew....@na...>]

Hi Enoch,

On 03/04/2013 04:03 AM, Enoch wrote:

> The code at<http://pastebin.com/1YtVdMeg>  will expire in 4 weeks.

See? In 4 weeks the code is gone. If you had added it to the message
in the first place, then anyone could find it even in years, if
the mail archive is still available on the net or locally on my or
your hard disk --- even for offline searching and viewing.

So call me a mailing list die hard --- that's fine. And I you would
please be so kind as to add your lines to the message in the future.
I'm not afraid of a lengthy message. Thanks!

Cheers,
Erich

Re: [Amforth] Wish to FORGET

[Enoch <ix...@ho...>]

Erich Waelde <ew....@na...> writes:

> Hi Enoch,
>
> On 03/04/2013 04:03 AM, Enoch wrote:
>
>> The code at<http://pastebin.com/1YtVdMeg>  will expire in 4 weeks.
>
> See? In 4 weeks the code is gone. If you had added it to the message
> in the first place, then anyone could find it even in years, if
> the mail archive is still available on the net or locally on my or
> your hard disk --- even for offline searching and viewing.

Hi Erich,

What if I wanted it to self expire as the code wasn't "fully baked".  If
my code is taken by Matthias then it would no doubt reach posterity :-)
but I am not there yet. Besides, pastebin makes downloading easier and
guarantees the integrity of the downloaded code. Mail system, if you are
not careful with safe line length, would do bad things to your code.

> So call me a mailing list die hard --- that's fine. And I you would
> please be so kind as to add your lines to the message in the future.
> I'm not afraid of a lengthy message. Thanks!

I take a solemn oath to submit the final crc8.frt for peer review!

Regards, Enoch.

> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb