I've never been successful trying to allocate more stack space to the Java VM. The amount of heap space will make no difference to this error.
The most likely explanation for the different behaviour is simply that your application code is using some of the stack space that would otherwise be available to Saxon.
I can't really offer any other suggestions without seeing your code. How deeply nested do you think its calls are?
Michael Kay
(By the way, it's helpful if you can choose a subject line that describes the problem. If everyone titled their posts as you have done, it would become very difficult to find your way through the archives or through digest postings.)

From: Roberto Cisternino [mailto:roberto.cisternino@gmail.com]
Sent: 16 January 2009 06:59
To: saxon-help@lists.sourceforge.net
Subject: [saxon] Saxon B

I need help to understand why Saxon B is behaving differently depending if I use it from the command line or direclty from a java program.

Using the command-line the stylesheet works perfectly and the transformation generates as expected a good result.
Using Saxon from a Java program and the same stylesheet caused the below error:

javax.xml.transform.TransformerException: Too many nested apply-templates calls. The stylesheet may be looping.; SystemID: ...

For sure the source XML and the stylesheet are enought complex, but I am surprised I cannot obtain the same result using one of the two methods described.

I tuned the VM to provide more heap memory or stack, but the result was always the above error.

The error appears too early in the transformation, and this surprise me more and more as the transformation is enough long (less than half a minute) using the command-line.

What are the default memory settings for the Saxon B command-line ?

Any ideas ?

I will appreciate very much any suggestions

Thank you in advance

Roberto Cisternino