Menu

Big (arbitrary precision) Integers

Wanderer
2015-03-21
2015-04-20
  • Wanderer

    Wanderer - 2015-03-21

    In the Windows version (other versions not tested) I found some bugs concerning big integers (arbitrary precision) which occurred either only within the *.bas interpreter, or only in the virtual machine for bytecode.

    Test program:

    a& = 230000000000000
    b& = 70000000000000
    Print a&, b&
    a& = 98765432109876543210
    Print a&
    Input x$
    End
    

    Output from Interpreter (as expected):
    230000000000000 70000000000000
    98765432109876543210

    Output from Bytecode:
    -2147483648
    -2147483648
    98765432109876543210

    This bug seems to affect only a certain range of numbers; making the first ones larger by some digits produced a correct output.

    Another test with larger numbers produced a correct output with bytecode; but there occurred instead a different bug in the *.bas interpreter. I could not isolate this bug in a "minimal" program since my own "radix" function is necessary to check the output. This function converts a rational number a&/b& into a string. When digits% is initialized to a number larger than 308, the *.bas interpreter outputs only the integer part; the fractional part returns zero. The bytecode file produces correct output. - The numerical range where this error occurs makes me think that the interpreter mistakes at some point a "big integer" for a "float"; but I could not find out where exactly.

    Test program:

    num& = 11
    denom& = 10
    digits% = 310
    For i% = 1 To digits%
      mul denom&, 10
      mul num&, 11
    Next
    Print num&
    Print denom&
    Print @bnx_radix$(num&,denom&,10,digits%)
    Input x$
    End
    
    Function bn_radix$(a&,base%)
      'Radix$ function for Big Integers
      Local temp2$, sign$
      If base%>64 Or base%<2 
        Return ""
      EndIf
    
      sign$=""
      If a&<0 Then
        sign$="-"
        a& = -a&
      EndIf
      repeat
        temp2$ = @bn_makedigit$(a& Mod base%) + temp2$
        div a&, base%
      Until a&=0
      Return sign$+temp2$
    EndFunction
    
    Function bnx_radix$(a&,b&,base%,maxlen%)
      'Returns a&/b& in base base%, max. maxlen% digits.
    
      Local temp$, n&
    
      If b&=0
        Return ""
      EndIf
      If Sgn(a&)<>Sgn(b&)
        temp$="-"
      EndIf
      a& = Abs(a&)
      b& = Abs(b&)
    
      n& = a&/b&      !Integer part
      temp$ = temp$ + @bn_radix$(n&,base%)+"."
    
      If temp$="."
        Return ""      !invalid base%
      endif
      n& = a& Mod b&  !Residue
      repeat
        mul n&,base%
        temp$ = temp$+@bn_makedigit$(n&/b&)
        n& = n& Mod b&
      Until (Len(temp$)>=maxlen%) Or (n&=0)
      Return temp$
    EndFunction
    
    Function bn_makedigit$(dig%)
      If (dig%<10)
        Return Chr$(dig%+48)
          Else 
            If (dig%<36)
            Return Chr$(dig%+55)
            Else 
              If (dig%<62)
              Return Chr$(dig%+61)
              Else 
                If (dig%=62)
                Return "@"
                Else 
                  If (dig%=63)  !63-27=36
                  Return "$"
                  Else
                    Return "!"   !signals an error
              EndIf
            EndIf
          EndIf
        EndIf
      EndIf
    EndFunction
    

    Output for digits%=305 (from both Bytecode and Interpreter, as expected):

    46361946174185179276750011640582332180628489327466740940324624399125580047136127
    91040099862472389532234624164471691266852378419643448380736167177813270380964069
    10455319635140412247283439812346355032556095748060313880788898473283281584926217
    4235725185103959623351550554679376495444807144093743931113105818652023236389561
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    0000000000000000000000000000000000000000000000000000000000000000000
    4636194617418.517927675001164058233218062848932746674094032462439912558004713612
    79104009986247238953223462416447169126685237841964344838073616717781327038096406
    91045531963514041224728343981234635503255609574806031388078889847328328158492621
    74235725185103959623351550554679376495444807144093743931113105818
    

    Output for digits%=310:
    From Bytecode (as expected):

    74666377932986973076998661247274251800223988346778460951802210841035737921713205
    36097991229510408065559184563123303502178423968619970051659404601540050081246442
    93357396825589985328372452652181928243481917763208616108149328880207457825339522
    42283777678617780130039056838166826396788163536344155384969680519972699424377518
    8611
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000000000000000000000000000000000000000000000000000000000
    7466637793298.697307699866124727425180022398834677846095180221084103573792171320
    53609799122951040806555918456312330350217842396861997005165940460154005008124644
    29335739682558998532837245265218192824348191776320861610814932888020745782533952
    2422837776786177801300390568381668263967881635363441553849696805199726
    

    From the Interpreter (*.bas file):

    74666377932986973076998661247274251800223988346778460951802210841035737921713205
    36097991229510408065559184563123303502178423968619970051659404601540050081246442
    93357396825589985328372452652181928243481917763208616108149328880207457825339522
    42283777678617780130039056838166826396788163536344155384969680519972699424377518
    8611
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000000000000000000000000000000000000000000000000000000000
    7466637793298.000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    0000000000000000000000000000000000000000000000000000000000000000000000
    
     
  • Markus Hoffmann

    Markus Hoffmann - 2015-03-27

    Bug #1: confirmed. There is a bug in the compiler, which mistreats constant numbers up to 19 digits as normal 32 bit int. This bug is fixed now. The next version 1.23-17 should have that fixed. Now, only numbers up to 999999999 are treated as 32bit ints. This is not 100% correct but should work in most cases. It can cause problems with big integer constants like 0xffffffff and should also be fixed. This bug is only in the bytecode compiler.

     

    Last edit: Markus Hoffmann 2015-03-27
    • Markus Hoffmann

      Markus Hoffmann - 2015-04-01

      I have even fixed this issue now. Now Integers up to 32bits (0xffffffff) can can be treated as normal ints. Bigger values will be treated as arbitrari int values (slower).

       
  • Markus Hoffmann

    Markus Hoffmann - 2015-03-27

    Problem #2: I could isolate the problem to following lines:

    n&=699424377518861100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
    b&=1000000000000000000000

    f%=n&/b&
    f&=n&/b&
    print f%
    print f&
    dump
    quit

     

    Last edit: Markus Hoffmann 2015-03-27
  • Markus Hoffmann

    Markus Hoffmann - 2015-03-31

    Here is a workaround: Do not use "/" with big integers. The result would be a big rational number (which is not implemented). Since you want the integer part of the division anyways, please use "DIV" instaed. So it should be f%=n& DIV b&

    Following lines in your code will fix the problem:

    ...
    repeat
      mul n&,base%
      f%=n& DIV b&
      temp$ = temp$+@bn_makedigit$(f%)
      n& = n& Mod b&
    Until (Len(temp$)>=maxlen%) Or (n&=0)
    Return temp$
    

    EndFunction

     

    Last edit: Markus Hoffmann 2015-03-31
    • Wanderer

      Wanderer - 2015-03-31

      Good, that will help to get around it for now.

       
  • Wanderer

    Wanderer - 2015-04-06

    There seems still to be a bug in the interpreter. In the two big integer divisions in the program above, I replaced the "/" by "DIV" and got the following output:

    74666377932986973076998661247274251800223988346778460951802210841035737921713205
    36097991229510408065559184563123303502178423968619970051659404601540050081246442
    93357396825589985328372452652181928243481917763208616108149328880207457825339522
    42283777678617780130039056838166826396788163536344155384969680519972699424377518
    8611
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000000000000000000000000000000000000000000000000000000000
    7466637793298.111111111111111111111111111111111111111111111111111111111111111111
    11111111111111111111111111111111111111111111111111111111111111111111111111111111
    11111111111111111111111111111111111111111111111111111111111111111111111111111111
    1111111111111111111111111111111111111111111111111111111111111111111111
    

    Also, did you notice that the bytecode produced correct output even the first time (using the "/")? So the difference seems to be not so much between "/" and "DIV", but between the bytecode compiler and the interpreter.

    PS: The manual is not clear about the use of DIV. "a% DIV b%" is not documented, and "DIV var, n" is documented as identical with "var = var/n" (Page 234 according to the page numbering, which is page 240 of the PDF file.) - Page 41 (resp.47) says that the backslash "\" is used for integer division (no mention of DIV in this context), and this operator is not listed under the operators supported for arbitrary precision integers (page 38, resp.45). On the other hand, I did not find any hint in the manual that DIV is intended to mean an integer division. Finally, since I found by testing that "a%/b%" also returns the integer part of the result (instead of rounding to the nearest integer, as usual), I supposed that X11-Basic makes no difference at all between the different operators for division, when applied to integers. So I think it would be useful to clarify this in the manual.

     

    Last edit: Wanderer 2015-04-06
  • Markus Hoffmann

    Markus Hoffmann - 2015-04-07

    Yes, I noticed that the compiler did it right. But there are two different parsers for the expressions:
    The parser for the interpreter tries to estimate what type of expression it evaluates. E.g. The result of a division of two integers is usually a rational number and should be stored in a floating point variable. If the result is assigned to an integer variable, the result is converted into an int later.

    With big integers this gives a problem, because the result of a division of two big integers may or may not fit into a float. Since there is no way to store arbitrary precision rational numbers in X11-Basic, this can be a problem. Either the parser detects this conflict and complains or it ignors it and converts the result into a float and later back into a big integer. The bytecode parser does this (which happens to work for you, since the result is always small). The interpreter will now complain. The DIV operator (as a complement to the MOD operator) always produces results of the same type as the operators, this means, if you use DIV with integer numbers it will return an integer (without intermediate conversion into a float). For big integers this has the advantage that there is never a loss of precision.

    There is also a reason why I have used an intermediate integer variable f% in the workaround. This was to avoid conversion into float within the parameter passing to a subroutine. Of course this should be improved, but I hoped, that the workaround would do for your specific problem for the moment until I fixed the thing in a more clever way.

     
    • Wanderer

      Wanderer - 2015-04-11

      The above output (last post) was produced using DIV, not /. Also, the subroutine "makedigit" requires an integer as a parameter. So why should the result of the division be converted into a float? - Also, even then the result would be wrong: the interpreter returned all ones as decimals.
      - But I will test again using the additional variable, as you suggest.

       
  • Wanderer

    Wanderer - 2015-04-12

    I have now tested big integer division in an isolated program. It is obviously this operation which causes the errors. - And also MOD, see next post. (Tested only with the Windows version.)

    Summary of results:

    Bytecode returns always the integer part for both DIV and /. Considers \ as a syntax error (which causes a Segmentation fault at the end of the program).

    The interpreter returns always the integer part for DIV, /, and \, when the result is assigned to a big integer variable.
    When the result is assigned to a normal integer variable, the interpreter has a rather inconsistent and faulty behavior:
    DIV returns always 1.
    \ returns always 0.
    / rounds the result to the nearest integer (as expected) if the operands are within Float range, and returns an invalid value otherwise.

    So for now, big integer division seems safe only if DIV or / is used and the result is assigned to a big integer variable. All other uses lead to errors either in the interpreter or in the bytecode, or in both.

    My suggestions:
    - Allow \ for big integers also in the bytecode compiler.
    - Change the behavior of / so that it rounds to the nearest integer (as usual in other programming languages), instead of truncating the result. Otherwise there would be no sense in having several different operators for division.
    - Fix the behavior of DIV and \ in the interpreter. (There are situations, such as in my first test program above, where the programmer knows that the result will be within normal integer range; so it should be allowed to assign the result of a big integer division to a normal integer.)
    - Make the use of DIV more consistent. If DIV is intended to mean integer division, then "DIV a%, b%" should also be equivalent to "a% = a% \ b%" (not to "a% = a% / b%"), and "DIV a, b" (float) should throw a "Type mismatch" error (or convert a and b to integers before dividing, and return only the integer part in a). - On the other hand, if you want to retain "DIV a, b" as meaning "a = a / b", then I would discard the use of DIV for integer divisions and use only \ for that purpose. This would help to avoid confusions.

    Here is my test program:

    denom& = 1
    digits% = 100
    For i% = 1 To digits%
      mul denom&, 10
    Next
    num& = denom&
    mul num&, 4
    dec num&
    Print num&
    Print denom&
    Print digits%, " digits"
    bigQ& = num& div denom&
    q% = num& div denom&
    Print "DIV = "; bigQ&; " (Big Integer variable)"
    Print "DIV = "; q%; " (Integer variable)"
    bigQ& = num& / denom&
    q% = num& / denom&
    Print "/ = "; bigQ&; " (Big Integer variable)"
    Print "/ = "; q%; " (Integer variable)"
    bigQ& = num& \ denom&
    q% = num& \ denom&
    Print "\ = "; bigQ&; " (Big Integer variable)"
    Print "\ = "; q%; " (Integer variable)"
    Input x$
    End
    

    Output for 100 digits, in the interpreter:

    39999999999999999999999999999999999999999999999999999999999999999999999999999999
    999999999999999999999
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000000
    100      digits
    DIV = 3 (Big Integer variable)
    DIV = 1 (Integer variable)
    / = 3 (Big Integer variable)
    / = 4 (Integer variable)
    \ = 3 (Big Integer variable)
    \ = 0 (Integer variable)
    

    Output for 100 digits, bytecode:

    ERROR: ==> Syntax error NUM& \ DENOM
    ERROR: ==> Syntax error NUM& \ DENOM
    39999999999999999999999999999999999999999999999999999999999999999999999999999999
    999999999999999999999
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000000
    100
             digits
    DIV = 3 (Big Integer variable)
    DIV = 3 (Integer variable)
    / = 3 (Big Integer variable)
    / = 3 (Integer variable)
    ERROR at offset $17e: ** 1 - Segmentation fault
    PC negativ !
    ** PROGRAM-STOP
    

    (By the way, you can see here another small bug in the bytecode: "PRINT digits%," inserts a line feed after digits%.)

    Output for 310 digits (changing the number in Line 1), in the interpreter:

    39999999999999999999999999999999999999999999999999999999999999999999999999999999
    99999999999999999999999999999999999999999999999999999999999999999999999999999999
    99999999999999999999999999999999999999999999999999999999999999999999999999999999
    99999999999999999999999999999999999999999999999999999999999999999999999
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000
    310      digits
    DIV = 3 (Big Integer variable)
    DIV = 1 (Integer variable)
    / = 3 (Big Integer variable)
    / = -2147483648 (Integer variable)
    \ = 3 (Big Integer variable)
    \ = 0 (Integer variable)
    

    Output for 310 digits, bytecode:

    ERROR: ==> Syntax error NUM& \ DENOM
    ERROR: ==> Syntax error NUM& \ DENOM
    39999999999999999999999999999999999999999999999999999999999999999999999999999999
    99999999999999999999999999999999999999999999999999999999999999999999999999999999
    99999999999999999999999999999999999999999999999999999999999999999999999999999999
    99999999999999999999999999999999999999999999999999999999999999999999999
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000
    310
             digits
    DIV = 3 (Big Integer variable)
    DIV = 3 (Integer variable)
    / = 3 (Big Integer variable)
    / = 3 (Integer variable)
    ERROR at offset $185: ** 1 - Segmentation fault
    PC negativ !
    ** PROGRAM-STOP
    

    PS: It might be necessary to test also what happens when a big integer is divided by a "normal" integer. I did this test with MOD (see next post) and it made no difference - except in the interpreter when the result is assigned to a "normal" integer, which was wrong anyway. So this is perhaps already OK for those cases which work correctly when dividing by a big integer.

     
  • Wanderer

    Wanderer - 2015-04-12

    I tested also MOD for big integers and found a similar bug in the interpreter: When the result is assigned to a "normal" integer, the interpreter returns wrong results, both within and beyond float range, but the errors are different. - The bytecode returned all correct results.

    Test program:

    denom& = 1
    digits% = 100
    For i% = 1 To digits%
      mul denom&, 10
    Next
    num& = denom&
    mul num&, 4
    inc num&
    bigQ& = num& Mod denom&
    q% = num& Mod denom&
    Print num&
    Print denom&
    Print digits%, " digits"
    Print "MOD big int = "; bigQ&; " (Big Integer variable)"
    Print "MOD big int = "; q%; " (Integer variable)"
    test% = 10
    bigQ& = num& Mod test%
    q% = num& Mod test%
    Print "MOD int = "; bigQ&; " (Big Integer variable)"
    Print "MOD int = "; q%; " (Integer variable)"
    Input x$
    End
    

    Output from the interpreter, 100 digits:

    40000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000001
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000000
    100      digits
    MOD big int = 1 (Big Integer variable)
    MOD big int = 0 (Integer variable)
    MOD int = 1 (Big Integer variable)
    MOD int = 2 (Integer variable)
    

    Output from bytecode, 100 digits:

    40000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000001
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    000000000000000000000
    100
             digits
    MOD big int = 1 (Big Integer variable)
    MOD big int = 1 (Integer variable)
    MOD int = 1 (Big Integer variable)
    MOD int = 1 (Integer variable)
    

    Output from the interpreter, 310 digits:

    40000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000001
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000
    310      digits
    MOD big int = 1 (Big Integer variable)
    MOD big int = -2147483648 (Integer variable)
    MOD int = 1 (Big Integer variable)
    MOD int = -2147483648 (Integer variable)
    

    Output from bytecode, 310 digits:

    40000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000001
    10000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000000000000
    00000000000000000000000000000000000000000000000000000000000000000000000
    310
             digits
    MOD big int = 1 (Big Integer variable)
    MOD big int = 1 (Integer variable)
    MOD int = 1 (Big Integer variable)
    MOD int = 1 (Integer variable)
    
     
  • Markus Hoffmann

    Markus Hoffmann - 2015-04-13

    Thank you for the demonstration programs. Yes, it looks like, X11Basic has still a consitency problem with the division of integers. First of all, the "\" operator is not(!) implemented, though the manual at some point mentions it. So an expression with "\" should always give a syntax error (at the moment).

    Next thing there is the fundamental problem, how a result of a division should be internally stored. And here is also the difference between the compiler and the interpreter. The compiler determines the type of the result of an expression looking only at the right side (of "="). The interpreter looks also at the left side of "=". Usually it does not make any difference, except for treating results of divisions of big integers.

    Here is the tables what the result of "/" operator is:

    | % | flt | & | #
    

    ----+---+-----+---+------
    % | % | flt | & | #
    flt |flt| flt |flt| #
    & | & | flt | & | #
    # | # | # | # | #

    And this is what the " DIV " operator should do:

    | % | flt | & | #
    

    ----+---+-----+---+------
    % | % | % | & | %
    flt | % | % | & | %
    & | & | & | & | &
    # | % | % | & | %

    If you assign the result to an int variable, the result will be converted to int (or big int), so there will be no difference, except for, for "/" you loose precision if the result is too big and for "DIV" you loose the fractional part. The interpreter does know, if the result is assigned to an integer variable, but the compiler does not. So the programmer has to choose the right operator.

    And now we find, that there is a bug in the interpreter, because if it finds

    a%=a DIV b (or a%= a MOD b) it treats the right expression and all its intermediate results as small (floats), and not as it should as big integers.

    a&=a DIV b would be treated as big integers, but also this is inconsistent, because there could be parts of the expression better treated as floats.

    I always thought that knowing the variable type the result of an expression is assigned to would be better for evaluation of the expression, but now I see, that it is not. The right side of an assignment "=" should always treated independently of what type the final result must be finally converted to. The programmer needs to be aware of this and needs to choose wherethere he need precision or fraction. Unfortunately such, the interpreters parser needs complete redesign...

    BTW: To my eyes a "\" operator is unnecessary and redundant and therefore should not be implemented. The question if the results should be rounded or truncated should be explicitly answered by the programmer using ROUND() when needed.
    (The default is always to cut of the fractional part, see INT()).

     
    • Wanderer

      Wanderer - 2015-04-20

      Yes, this seems to be a tricky problem. Perhaps you could find out how other open source compilers (such as FreeBasic) solve it?
      In any case, since DIV and MOD are defined as integer operations, I would suggest to completely disallow their use with float variables and throw a "Type mismatch" error when this is attempted. This might spare you some headache. (Instead, INT(a) DIV INT(b) could be used.) - The same for complex numbers, except if you would wish to introduce "complex integers" as a new type of numbers.

      For the other cases, I would say the following (after superficial consideration; I have not yet thought through it very deeply):
      Since / is defined as a "non-integer" division, its result should always be assigned to a float, even if the operands are integers. This would require the creation of arbitrary precision floats for storing the results of divisions with arbitrary precision integers; otherwise you will have overflow problems. (I imagine this might be relatively easy if you could implement bit shift operations for arbitrary precision integers. You could then store an arbitrary precision float as two numbers, an integer (&) and a (binary) exponent, and convert all operations with such numbers internally to (&) integers by scaling (shifting) them appropriately. Only that such operations might require an additional parameter indicating the desired precision of the result.)
      - Assigning a%/b% or a&/b& to an integer does not make much sense, in my opinion, since there exists already DIV for this purpose.

      As for the bugs reported above, I do not see them much related to this question. The interpreter already yields correct results if they are assigned to a & variable; and since this is the "normal" assignment (according to your table), the problem seems rather to reside in the way the interpreter converts this long integer to a normal integer. You can see from the results of my test program that a long integer of value 3 was converted to a normal integer of value 1, 4, 0, and -2147483648 respectively, depending on the operator used and on the numerical range of the operands. Since the long integer variable returns always the correct result (3 - allowing that you truncate the result of / instead of rounding it), the interpreter is obviously already capable of dividing correctly; but it is not capable - in this case - of converting a long integer correctly to a normal integer.
      (This does not apply to an ordinary conversion such as a% = b&, this works; but the conversion seems to fail when it concerns the result of a division.)
      - A suspicion: might it be that when processing a% = b& DIV c&, the interpreter attempts to convert b& and c& to normal integers before dividing, or something like that?

       
  • Markus Hoffmann

    Markus Hoffmann - 2015-04-13

    MOD is also a bit tricky: How to determine, which version to use?

    Version 1:
    | % | flt | & | #
    ----+---+-----+---+------
    % | % | flt | % | #
    flt |flt| flt |flt| #
    & | & | flt | & | #
    # | # | # | # | #

    or
    Version 2:
    | % | flt | & | #
    ----+---+-----+---+------
    % | % | flt | % | #
    flt |flt| flt | & | #
    & | & | & | & | &
    # | # | # | & | #

     

    Last edit: Markus Hoffmann 2015-04-13
  • Markus Hoffmann

    Markus Hoffmann - 2015-04-13

    If we had a way to store arbitrary precision rational numbers (suffix §) it could be like:

     | % | flt | &   | # | §
    

    ----+---+-----+-----+------
    % | % | flt | % | # | §
    flt |flt| flt | flt | # | §
    & | & | § | & | ? | §
    # | # | # | # | # | ?
    § | § | § | § | ? | &

    Even here arbitrary precision rational complex numbers (suffix ?) cannot be stored.

     

    Last edit: Markus Hoffmann 2015-04-16

Anonymous
Anonymous

Add attachments
Cancel





Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.