An infinite recursive loop (resulting in StackOverflow) occurs while attempting to optimize an expression such as
//tbody/row[position() mod $grp-size=1]
Where $grp-size is a variable whose type has not been declared.
What happens is as follows:
(a) because $grp-size is possibly untyped atomic, the type-checker adds an UntypedAtomicConverter around the $grp-size variable reference to convert untyped atomic values to doubles
(b) the expression UntypedAtomicConverter($grp-size) is recognized as being constant for the expression, so it is extracted and replaced with a variable. The expression now reads
let $zzz := UntypedAtomicConverter($grp-size)
return //tbody/row[position() mod $zzz=1]
(c) the resulting expression is re-optimized. The static type of $zzz is AnyAtomicType; this does not exclude the possibility of it being UntypedAtomic, so steps (a) and (b) are repeated ad infinitum.
It is not clear to me why the problem has escaped detection for so long; it does not appear to be new.
Because there is no way of representing the fact that the return type of UntypedAtomicConverter is "AnyAtomic but not UntypedAtomic", the solution adopted is to prevent an UntypedAtomicConverter expression being loop-lifted.
A patch is being placed in Subversion.