> HIGHEST PRIORITY:
> [1064863 part 1] user-defined closure() function --
> ExpressionTool patch
> I put this first not because I am asking you to supply this function,
> but because I had to made changes to ExpressionTool in order to make
> closures truly evaluate lazily. In the past there were many
> in which closures were forcibly evaluated (particularly the ones
> returned from extension function calls.)
> This patch is very critical, because it is at the core of Saxon and
> even if I do all the rest with user extensions and using the extension
> function factory API, etc., I'll still run into this problem.
> You may feel like this is a patch against the most touchy area in your
> code, but I can tell you that I have been using this for over a year
> now and I tend to do some of the weirdest XSLT things anyone might
> consider and I never once ran into problems because of this patch.
> Multiple times when I had problems I would disable my patches first
> to check if they would go away, and it never was my patch
> that made the
> So, I ask you to accept at a minimum the patch for the ExpressionTool
> or do anything that would prevent a Closure returned from a function
> from being eagerly evaluated.
> If you don't actually supply the closure() function, there are no
> dependencies on the other patches.
There are going to be a number of changes in the way delayed evaluation
works in 8.2. In particular, there will be two kinds of Closure, one which
(like the present Closure) remembers the values in the sequence the first
time it is evaluated, and the other discards them. Static analysis of
variable references is used to decide which kind to use on any given
The other change is that it will be possible to evaluate a Closure in "push"
mode as well as "pull" mode (i.e. it will have a process() method as well as
an iterate() method. So as a minimum, this means there are some logistical
issues in integrating and testing your patch.
However, I'm still not happy about the principle. Your patch changes the
effect of eagerEvaluate so that in some circumstances it does a lazy
evaluation. This presupposes that the caller can handle this: but in some
cases the reason eagerEvaluate rather than lazyEvaluate is called is that
the caller can't handle it.
Probably the most important case where this was true in 8.1 was in
evaluating functions. This was done using eagerEvaluate rather than
lazyEvaluate because on rare occasions lazyEvaluate didn't work, and I
didn't know why. It turned out to be an interaction with tail-call
optimization, which is now fixed, so in 8.2 functions will be called using
If we want to extend the use of lazy evaluation, then changing eagerEvaluate
so that it doesn't do what it says is not the right approach.
But I'm not sure it's a good idea to call user-written Java functions using
lazy evaluation. You're looking at it from a position where you trust the
author of the functions to know what they're doing; I'm looking at it from a
position where I don't. I don't actually want the mechanism for calling Java
extension functions to be a way for highly-skilled system programmers to
interact with the deep internals of Saxon, which is the way you are trying
to take it: I would like the mechanism to be as simple, safe, and fool-proof
as possible in the hands of ordinary incompetent mortals. The fact that
extension functions can have side-effects is already dangerous enough,
without allowing them to return results that depend on state that can be
changed after the function returns control. So I think there's a real
difference in our perspectives on this: and in the end, it's me that has to
deal with the bug reports.