#910 cast(string, variable) can cause fbc to segfault (infinite recursion)

[Jeff Marshall]

As indicated in fb.net forum topic fb 1.07 tests

The following code cause fbc to segfault:

sub proc1( byval s as string )
end sub 
sub proc2( byref s as const string )
    proc1( cast(string, s) )
end sub

Initial investigation reveals an inifinite recursion when fbc tries to figure out how to assign the cast(string, s) to the temorary variable to pass to the byval parameter in proc1(). Maybe related to rtlStrAssign().

Occurs in fbc 1.06 and 1.07, but not in fbc 1.05.

[Jeff Marshall]

Bug is also demonstrated in this example:

dim x as const string = "test"
dim y as string
y = cast(string, x )

Further investigation highlights a design issue with fbc compiler.
1) where type compatibility is fully checked at parser level, there is typically no concern.
2) Internally, fbc does not check all type compatibility at parser level, and in some cases it is not possible to do so. Instead, fbc relies on AST generation to either succeed and continue, or error and report back to the parser that AST could not be generated.
3) In this case, i.e. rtlStrAssign() we generally expect the internal AST generation to succeed without any report of error. The issue is that same internal routines are used potentially for both a) try and generate AST, automatically applying conversions and only report an error if invalid, and b) generate AST and ignore some conversions (like CONST) as it no longer matters.

[Jeff Marshall]

Immediate issue fixed in [d561d04145a47b24a14fedb2db48b642191d5e56]

Follow-up work is noted in TODO notes added to compiler source.