Undocumentated limitations of FOR/NEXT cause unexpected operations.

[Tony Bolton]

Using GCB Great Cow BASIC (0.98.01 2017-10-27) on Linux Mint 18.3

There appear to be a considerable range of numeric values of FOR/NEXT loop parameters that produce unexpected operations but none get a mention in the documentation. A specific example is

For i = 1 to 252 step 5

This initially produces the expected steps of i=1,6,11,16... etc, but then spans the entire range again
with i= 0,5,10,15 etc.

For i = 252 to 1 step 5

This loop spans the entire range 4 times.
The unexpected overflow problems can be dispelled if the loop termination test is restructured to not perform any operations on the loop counter outside the expected range of the loop. A test like this

'For incrementating loops
If end_value-loop_value >= step_value then 
add step_value to loop_value and branch back to loop start
'For decrementating loops
If loop_value-end_value >= step_value then 
subtract step_value from loop_value and branch back to loop start

For completely different reasons this loop is never executed but there is no mention in the documentation of the limitation.

bot = 1
top = 252
For i = top to bot step 5

It appears that there is no runtime support for decrementing loops so this bit of assembler ignores the code.

    lds SysCalcTempA,TOP
    lds SysCalcTempB,BOT
    cp  SysCalcTempA,SysCalcTempB
    brlo    SysForLoopEnd1

This is a sample program showing the effects.

; ----- Configuration

  #chip mega8, 1

; ----- Define Hardware settings
'      Setup Hardware I2C
    HI2CMode Master
    #define HI2C_DATA
    #define HI2C_BAUD_RATE 50

  '''Set up LCD
    #define LCD_IO 10
    #define LCD_I2C_Address_1 0x7E
    #define line1 = 128     ;go to start of line 1
    #define line2 = 192     ;go to start of line 2
    CLS

    bot = 1
    top = 252
    Do forever
    LCDcmd(line1) : Print "Test increment"
'       This loop runs the range twice
    for ipwr = bot to top step 5
        LCDcmd(line2) : Print ipwr : LCDspace 2
        wait 400 ms
    next
    wait 1 s

    LCDCmd(line1) : Print "Test decrement"
'       This loop runs 4 times.
    for ipwr = 252 to 1 step 5
        LCDcmd(line2) : Print ipwr : LCDspace 2
        wait 400 ms
    next
    wait 1 s

    LCDCmd(line1)
    Print "Test runtime" : LCDspace 4
    ' This never runs
    for ipwr = top to bot step 5
        LCDcmd(line2) : Print ipwr : LCDspace 2
        wait 400 ms
    next
    wait 1 s
    loop

[Anobium]

Not undocumented but a consquence of the nature of the commands.

For i = 1 to 252 step 5 .. 1, 6, 11 .. 246, 251, 1(or 256 if word). Therefore the range needs to be addressed, or your loop needs to mange the overun.

Simply use 1 to 251.

[Tony Bolton]

I have checked the source code for the FOR loop and and this documentation was helpfully provided...

'Then to this: (27/8/2010)
            'V = SV - ST
            'If SV not const or EV not const, if V > EV then goto SysForLoopEnd(n)
            'SysForLoop(n):
            'V += ST
            '...
            'if V <= EV then goto SysForLoop(n)
            'SysForLoopEnd(n):

This is JUNK. It only works as expected, that is the value of V (loop variable) is on or between the limits (SV - start value, EV - end value) for the special case where the difference between the limits (EV-SV) is an integral multiple of the step size (ST). For all other cases the final pass will have a value of V higher than EV. This is not what a programmer would expect because no other language would dream of doing so.

In case the big numbers in my first example reduced appreciation of the issue, here are some little ones.

    bot = 1
    top = 4
    Do forever
    LCDcmd(line1) : Print "Test increment"
    for ipwr = bot to top step 2
        LCDcmd(line2) : Print ipwr : LCDspace 2
        wait 1 s
    next
    loop

On the first pass ipwr = 1, this is easy.
On the second pass ipwr = 3, this is reassuring.
On the third unexpected pass ipwr = 5, this is wrong.

Don't try and tell a programmer they should have used some other number, there are many valid applications where the loop parameters are derived from independent data or measurements and the programmer has little control over what is used. They just expect a defined result for any set of legal numbers.

To be really helpful have a defined result for the special case of step=0, presently it produces an infinite loop. This is theoretically correct but disruptive in real life, a more practical result would be a single pass with V = SV.

[Anobium]

Point taken - I will update the help to clarify this constraint.

[Moto Geek]

@Tony Bolton, this is an interesting issue. I have not yet bumped into it, but it seems it could happen.

'Then to this: (27/8/2010)
            'V = SV - ST
            'If SV not const or EV not const, if V > EV then goto SysForLoopEnd(n)
        'SysForLoop(n):
        'V += ST
        '...
        'if V <= EV then goto SysForLoop(n)
        'SysForLoopEnd(n):

This is JUNK.

Is there a better way to do this that would produce the results you are expecting? What would you suggest to make this work as anticipated?

[Tony Bolton]

I suggest the folowing steps...

(1) Start with the loop termination algorithm I outlined in the frst post of this thread.

(2) Try to add step=0 handling, should be reasonably easy.

(3) Try to add runtime step decrement handling, this will have a more difficult effort/size/complexity compromise.

(4) Conduct a rigorous gamut of testing, especially considering the extremes of the range of each variable type to identify over/under flows.

[Anobium]

I have added this to the list to investigate.

The issue is not as simple as it looks, as you say. The loop termination will required to handle the overflow and the underflow conditions that may happen. Also, the compiler does assume certain conditions which I can document when I get a moment.

[Anobium]

Just added to the Help in the FOR-NEXT section.

---

Handling a FOR-NEXT Overflow

When using FOR-NEXT with Great Cow BASIC you may need to handle an overflow situation.  
An overflow will happen when the next step in your increment exceeds the variable type.  

Consider. For = 250 to 255 STEP 3.  The steps will be 250, 253 and 0 etc.  
As the third in this sequence has overflowed to 0 not 256 the For-Next loop will continue and you may have unexpected outcomes from your program.  

To resolve this. Do not use a FOR-NEXT but a construct that will ensure an overflow is controlled.  

The following code shows how to use a DO-LOOP to increment (the constant __INC) from the base ( __MIN) to a maximum value ( __MAX).  
This program tests for an overflow condition and will exit, and, the program tests to ensure the next value is less than the maximum.  

Consider. __MIN = 250, __MAX = 255 and IINC=3.  
The steps will be 250 and 253. There is no overflow and the incremented value is always less then the maximum   

---

#chip  lgt8fx328p
#option Explicit

'USART settings for USART1
#define USART_BAUD_RATE 9600
#define USART_TX_BLOCKING
#define USART_DELAY OFF

'Wait for terminal to select.. or, give time for me to get to terminal....
wait 2 s


'******************************************** START OF CODE ***********************
#DEFINE __MIN 250
#DEFINE __MAX 255
#DEFINE __INC 3
'This variable needs to be of the correct type Byte, Word etc to handle the maximum value
dim __nextremetest as byte

__nextremetest = __MIN

do
    'Code to show results. Remove from target code
        hserprint __nextremetest
        HSerPrintCRLF
        wait 100 ms
    'Do Stuff
    '...
    'Do Stuff

    __nextremetest = __nextremetest  + __INC
    'Test of overflow
    if c then exit do

    'Now test for maximum has not been exceeded
    if __nextremetest > __MAX then exit do
loop

---

This code is efficient in terms coding.  
And, this method should be used there is a risk that an overflow situation may happen.

[jackjames]

" if c then exit do"
What is c ????????

[Anobium]

C is the Carry Flag. Do not use status.C register as using C is managed by the compiler and works across all chip families.

So, testing C when certain operations are happening is useful.

There are many functions that use C. See http://gcbasic.sourceforge.net/help/_rotate.html you will see another use of C

[jackjames]

THANKS !

[Anobium]

Testers Needed.

I have updated the compiler and I have rewritten the FOR-NEXT handler.

I have used the insights provided by Tony and with some adaption I have rewritten. The new handler will now support more options and is intended to be reliable.

I have tested many use case as follows, but, I need YOU to test.

From	To	Step	Comments
Constant_Low	Constant__Hi	Not present	Assumes step 1 - counts up
Constant_Hi	Constant__Lo	Not present	counts up i.e. 240 to 8 passing through 0
Constant_Low	Constant__Hi	Step Constant = 1	counts up
Constant_Hi	Constant__Lo	Step Constant = 1	counts up i.e. 240 to 8 passing through 0
Constant	Constant	Step Constant = 0	loops. never exits
Variable	Variable	Not Present	Assumes step 1 - counts up
Variable	Variable	Positive Integer	counts step up - handling overflows
Variable	Variable	Negative Integer	counts step down- handling overflows
Variable	Variable	Step Integer=0	Loops, until Step is specified
Variable	Variable	Positive Constant	creates system integer and counts step up - handling overflows
Variable	Variable	Negative Constant	creates system integer and counts step down - handling overflows

If this is not tested. I just wasted a weeks work... so, please let me know when you can find some time to look at this huge change,

[Anobium]

Sign up, Sign up, Sign up.

To test. Install RC42 and then I will upload the new test compiler and the language file. I have a series of test programs - these may help but I think it is best to create test scenarios that you typically use.

Sign up, Sign up, Sign up.

[Anobium]

The following test program now works as expected. This is totally based on Tony's post. I changed to serial to automate the testing.

Evan

#define NewNextForhandler
#chip 16f88
#option Explicit

    'USART settings for USART1
    #define USART_BAUD_RATE 9600
    #define USART_TX_BLOCKING
    #define USART_DELAY OFF

    dim ipwr, bot, top as Byte

    bot = 1
    top = 252

    HSerPrint "Test increment"
    HSerPrintCRLF
'       This loop runs the range twice in original code
    for ipwr = bot to top step 5
        HSerPrint ipwr
        HSerPrint ", "
        wait 10 ms
    next
    wait 1 s

    HSerPrintCRLF
    HserPrint "Test decrement"
    HSerPrintCRLF
'       This loop runs 4 times in original code
    for ipwr = 252 to 1 step 5
        HSerPrint ipwr
        HSerPrint ", "
        wait 10 ms
    next
    wait 1 s

    HSerPrintCRLF 2
    HSerPrint "Test runtime"
    HSerPrintCRLF
    ' This never runs in original code
    for ipwr = top to bot step 5
        HSerPrint ipwr
        HSerPrint ", "
        wait 10 ms
    next
    wait 1 s

[Domenic Cirone]

Sign me up, I will some testing on arduino nano.

[Anobium]

Domenic - thank you. I will post instructions soon.

[Anobium]

Testers

Please test. You must use RC42 as the baseline and then apply the EXE and MSG files to your GreatCowBASIC folder. This will update to the new FOR-NEXT handler.

To enable/test the new handler you must add #define NewNextForhandler to your program. This will then ensure you can test the new handler, and, you can comment out to test/compare against the old handler.

The patch zip is here. The zip contains many of the programs used to test. Please add to this collection.

This is the code that being used as the basis. The compiler is simply doing this. So, the 'proof code' program in the zip let us see what is happening without using the new handler.

#chip 16f88
#option Explicit

    'USART settings for USART1
    #define USART_BAUD_RATE 9600
    #define USART_TX_BLOCKING
    #define USART_DELAY OFF

    dim LoopVar, StartValue, EndValue, ccount as Integer
    dim StepValue as Integer

    //Set the value here.
    StartValue = -12
    EndValue = -14
    StepValue = -2

    If StepValue.15 = 0 then
      //handle a positive step value

      'Starting code
      LoopVar = StartValue
      posloop:

        'do stuff - your code
        HSerPrint LoopVar
        HSerPrintCRLF


        if EndValue-LoopVar >= StepValue then
          LoopVar = LoopVar + StepValue
          goto posloop
        end if

    else

      //handle a negative step value

      'Starting code
      LoopVar = StartValue
      negloop:

        'do stuff - your code
        HSerPrint LoopVar
        HSerPrintCRLF

        'while the  * -1 looks odd, it is not.  This caters for negative steps and then permits a Simple addition on the next lo
        if LoopVar - EndValue >= (StepValue * -1) then
          LoopVar = LoopVar + StepValue
          goto negloop
        end if

    end if

You will see more warnings when using the new handler, and, you will be shown how to mask these warnings. These warnings are intended to inform users when specific constraints may influence the operation of the new handler. You can 1) Improve the warnings by editing the MSG file 2) Ask for more warnings to be added to the final complier 3) Recommend we remove warnings via this thread. :-)

---

Have fun.... post results.

[Anobium]

I have ran the static tests on all the demos.

There will be issues that need to be sorted but they are minor. Take the following example from the SPIRAM program. Where the EndValue is 0x1F (d31)-2, so, this is an end value of 29. Old code the the last LoopValue to 30 and the new code set the last LoopValue to 28. Clearly, the new code is correct but this will mean the user code should be revised.

There may be many cases where this happens.

'#define NewNextForhandler

#chip 16f88
#option Explicit

    'USART settings for USART1
    #define USART_BAUD_RATE 9600
    #define USART_TX_BLOCKING
    #define USART_DELAY OFF

    DIM PFMAddress as LONG

    for PFMAddress = 0x0000 to 0x001F - 2 step 2
       'do stuff
        HSerPrint PFMAddress
        HSerPrintCRLF
    Next

END

OLD 0,2,4.. 26,28,30
New 0,2,4.. 26,28

[Anobium]

New compiler exe uploaded. Dated 17/3/2021 Use ForNextPatch002.zip

Add new modifier to the FOR-NEXT. #IGNOREWARNING

#IGNOREWARNING will override the warnings for an explicit instance of FOR-NEXT. This should be used in libraries to ensure the user program controls the warnings.

Example shown below from epd_epd2in13d.h library

  SET EPD_DC ON
  for EPD_Ind_raw=1 to GLCD_Height #IGNOREWARNING
    for EPD_Ind_col=1 to GLCD_Width #IGNOREWARNING
      'Replaced with macro for speed              SendData_EPD2in13D(GLCDBackGround)
      EPD2in13D_Data = GLCDBackGround

      SendData_EPD2in13D_Macro
    next
  next
  SendCommand_EPD2in13D(DATA_TRANSMISSION_2)

The user will get warnings but the library is assumed to be tested!

[stan cartwright]

I didn't want to upset anyone with my comments.
It's true that I have never had a problem with for next step.
If the for next range is not exactly divisible by the step value then it will not reach the end value.
This seems a user problem not a compiler problem. Sorry if sloppy programming upset anyone.
I realise and appreciate the work the gcb team has done, even if it's for something I don't use.
I wonder if the overflows in the glcd lib still exist. There are so many displays supported now.
Say you draw a square. erase it and redraw it 1 pixel down and repeat in a constant loop you will find different results on different displays.
Some times the square will slowly go off the bottom of the screen then after a while will reappear at the top and do the same thing again.
A different display would behave differently but the point is there's no checking for plot-draw that's greater than the screen resolution. Last time I looked it seemed it was checking but there's lots of code in glcd to examine. An excellent bit of coding with universal commands.
I really got to get my head together to do animated graphics and a test of lots of gcb commands.
I guess I've been lucky so far. I did report the box fill overflow which was sorted so I'm not a time waster.
I wanted to do an "Asteroids" game but modifying the draw function to check each segment being off screen would be too slow.

[Anobium]

I give up.

[Anobium]

This discussion closed in March 2021

I can revert all the changes back in about 2 seconds. It will revert back to the broken FOR-NEXT loop.

Get involved to test and resolve the issue, or, I will revert.