#3716 Segmentation fault when calculating the taylor series of a specific function

[frcl]

The following code evaluated in Maxima 5.44.0 leads to a segmentation fault

taylor(sin(atan(1/(sqrt(3) + 2)) - x), x, 0, 1);

I first reported this in the sage bug tracker where Github user DaveWitteMorris pointed out that more generally

taylor(sin(atan(1/(sqrt(a) + b)) - x), x, 0, 1);

crashes for all integers a and b where a < b^2 and a is not a square number.

[Robert Dodier]

Looks like a stack overflow. Looks like it gets into a loop, EXPAND -> TRY-EXPANSION -> TRIG-CONST -> PSCOSA+B -> EXPAND ... . I guess the first place to look is what PSCOSA+B is trying to do.