On 23/12/2010 15:23, Srinivas Ranganathan wrote:

Thank you very much for a quick reply.

 

There is a slight wrinkle.  Saxon 8.7 is bundled along with an oracle product and we write extensions ( by implementing some interfaces that oracle product calls in.) . Hence If I need to move to the new release, I need to convince oracle to upgrade the saxon version that is bundled along with the app. For that if I am confident that it is a saxon version issue, I have a strong case against upgrade request for oracle.


You probably have a strong case anyway. If you're paying for a supported application from Oracle, then they shouldn't be using unsupported components as part of their product. (Or any any rate, the responsibility for supporting those components is theirs alone).

 

If you can look at my code and comment if something comes out glaringly that would be great.  I understand that multi threading bugs can be hard to find.

 

Code –  Class XSLTTransform

 

templatesMap is a static  HashMap where we keep a key value pair. Key is the unique ID and value is the XSLT Template that is created from the xslt. Then we create a new transformer from the template. This whole thing is synchronized.

I can't see anything obviously wrong here. It would be more usual, I think, for everything to share the same TransformerFactory rather than creating a new one per stylesheet, but it should work correctly this way. Also, having a sychronized static method is usually bad for performance, but it should be safe.

It's possible the problem could becaused by this bug:

https://sourceforge.net/tracker/index.php?func=detail&aid=1878851&group_id=29872&atid=397617

Although the reported effect there is an AssertionError rather than an ArrayIndexOutOfBoundsException, it's never possible to be certain what will happen once internal data is corrupted, and your stack trace does suggest you were evaluating a Closure at the time, which is similar to the situation described in that bug. I checked the 8.7 source code and the bug is indeed present there (it was reported and fixed against 9.0).

As an alternative to upgrading, you could try to rebuild 8.7 with the source clearance of that bug, which is to make the method net.sf.saxon.expr.ComputedExpression.getSlotsUsed() synchronized.

Michael Kay
Saxonica


 

private static synchronized Transformer createNewTransformer( String xslt, String digestValue ) throws Exception {

 

        if (!templatesMap.containsKey(digestValue)) {

            Class transformerClass = Class.forName("net.sf.saxon.TransformerFactoryImpl");

            TransformerFactory transformerFactory = (TransformerFactory) transformerClass.newInstance();

            SAXSource transformSource = new SAXSource(new InputSource(new StringReader(xslt)));

            Templates templates = transformerFactory.newTemplates(transformSource);

            templatesMap.put(digestValue, templates);

        }

        return templatesMap.get(digestValue).newTransformer();

    }

 

Please look at the last line alone. This is just a internal specific implementation that calls the above method to get the transformer. It generates the unique key from  the xslt and creates a transformer. This code is multithreaded. Ie Multiple instances of the class can have the same method executed. Note: We are not using the same instance of the class. It’s just that we share the templates to create new transformer.

 

private Transformer getTransformer(DataDcl eventDataDcl) throws Exception {

 

        String xslt = eventDataDcl.getTextValue(EventConstants.COLUMN_XSLT);

        String key = "XSLTDigestKey";

        String digestValue = null;

        if (eventDataDcl.getRow().containsKey(key)) {

            digestValue = eventDataDcl.getTextValue(key);

        }

        if (digestValue == null) {

            digestValue = EncryptionHelperUtl.digest(xslt);

            eventDataDcl.getRow().put(key, digestValue);

        }

      

        return createNewTransformer(xslt, digestValue);

    }

 

Main code – Again called in multithreaded mode.

 

                Transformer transformer = getTransformer( eventDataDcl );

 

      StringWriter resultBuffer = new StringWriter();

      SAXSource inputSource = new SAXSource(new InputSource(new StringReader(sourceDocument)));

     Result result = new StreamResult(resultBuffer);

      for (Map.Entry parameterEntry : parameters.entrySet()) {

transformer.setParameter((String)parameterEntry.getKey(), parameterEntry.getValue());

    }

      transformer.transform(inputSource, result);

 

 

The last line is where we get the exception.  Note: I have also removed all the templates and had each thread compile the XSLT and apply on the xml. Ie no static hashmap. I had even more problems. Saxon was not able to compile xslt in multiple threads(it gave other exception) and also I saw the same stacktrace as I see here.

 

At the same time I am taking steps to upgrade the version. 550 bugs is a huge number.  Please comment on the above code. It will be very helpful.

 

I really appreciate the help you have extended.

 

Thanks,

Srinivasan Ranganathan

 

 

From: Michael Kay [mailto:mike@saxonica.com]
Sent: Thursday, December 23, 2010 7:01 AM
To: saxon-help@lists.sourceforge.net; Srinivas Ranganathan
Subject: Re: [saxon] Saxon 8.7 issue

 

Saxon 8.7 is a very old release (Feb 2006); there have been five major releases since then and innumerable maintenance releases; during that time around 550 bugs have been fixed.

It's possible that there is a threading bug in your application code, or its possible that you are hitting one of those 550 Saxon bugs. I'm prepared to look at your code to see if there is anything obviously wrong, but multi-threading bugs can be hard to find. I think your best strategy is to move forward to a more recent Saxon release and see if the problem goes away.

Michael Kay
Saxonica

On 22/12/2010 20:56, Srinivas Ranganathan wrote:

Hi,

 

I have been trying to solve a issue that we are facing with saxon 8.7 for months without success.

 

How saxon is used in our App.

 

All the below logic runs in multiple threads. Each thread runs something and generates a xml.

We generate xml on the fly from run time variables.

We have a XSLT that is compiled into a template and new xslt transformer is created. ( The template is cached and new transformer is created for each thread /run. This logic alone is in synchronized block )

We apply the runtime xml with the xslt transformer, which results in another xml. We use that XML to pass to other applications in a SOA solution.

 

Now in the fourth step in multithreaded mode, we get a exception ( at random times ) in AIX and IBM Websphere JDK 5. We were unable to recreate the exception in Windows platform and IBM Websphere  JDK 5. The below exception occurs infrequently and with a probability of close to 2% of the time when our app is long running with about 6 threads. We have even caught this exception and retry the same logic again. Again the retry sometime succeeds and sometime fails. I have verified that when the failure happens, the input xml to the xslt transformer is valid and is the same to the one that succeeds. I have made all checks to make sure we are using it correctly and sending in valid arguments. It makes me believe that saxon transformer is not thread safe. I googled and checked various forms/discussions and do not find any issue related to this exception discussed. Can you kindly look into it and give your feedback on this issue.

 

Exception

 

java.lang.ArrayIndexOutOfBoundsException

                at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValue(TinyParentNodeImpl.java:64)

                at net.sf.saxon.tinytree.TinyParentNodeImpl.getStringValueCS(TinyParentNodeImpl.java:39)

                at net.sf.saxon.tinytree.TinyNodeImpl.getTypedValue(TinyNodeImpl.java:91)

                at net.sf.saxon.instruct.SimpleContentConstructor.evaluateItem(SimpleContentConstructor.java:213)

                at net.sf.saxon.instruct.ValueOf.evaluateItem(ValueOf.java:173)

                at net.sf.saxon.instruct.SimpleNodeConstructor.iterate(SimpleNodeConstructor.java:164)

                at net.sf.saxon.instruct.Choose.iterate(Choose.java:373)

                at net.sf.saxon.value.Closure.iterate(Closure.java:281)

                at net.sf.saxon.value.Value.getIterator(Value.java:222)

                at net.sf.saxon.expr.UserFunctionCall.iterate(UserFunctionCall.java:210)

                at net.sf.saxon.expr.ExpressionTool.eagerEvaluate(ExpressionTool.java:279)

                at net.sf.saxon.expr.UserFunctionCall.callFunction(UserFunctionCall.java:236)

                at net.sf.saxon.expr.UserFunctionCall.iterate(UserFunctionCall.java:206)

                at net.sf.saxon.instruct.CopyOf.processLeavingTail(CopyOf.java:106)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:328)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Template.expand(Template.java:98)

                at net.sf.saxon.instruct.Template.processLeavingTail(Template.java:82)

                at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:297)

                at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:169)

                at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:133)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:328)

                at net.sf.saxon.instruct.Template.expand(Template.java:98)

                at net.sf.saxon.instruct.Template.processLeavingTail(Template.java:82)

                at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:297)

                at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:169)

                at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:133)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.instruct.Template.expand(Template.java:98)

                at net.sf.saxon.instruct.Template.processLeavingTail(Template.java:82)

                at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:297)

                at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:169)

                at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:133)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.instruct.Template.expand(Template.java:98)

                at net.sf.saxon.instruct.Template.processLeavingTail(Template.java:82)

                at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:297)

                at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:169)

                at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:133)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.instruct.Template.expand(Template.java:98)

                at net.sf.saxon.instruct.Template.processLeavingTail(Template.java:82)

                at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:297)

                at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:169)

                at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:133)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:332)

                at net.sf.saxon.instruct.Instruction.process(Instruction.java:91)

                at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:164)

                at net.sf.saxon.instruct.Template.expand(Template.java:98)

                at net.sf.saxon.instruct.Template.processLeavingTail(Template.java:82)

                at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:297)

                at net.sf.saxon.instruct.ApplyTemplates$ApplyTemplatesPackage.processLeavingTail(ApplyTemplates.java:429)

                at net.sf.saxon.Controller.transformDocument(Controller.java:1518)

                at net.sf.saxon.Controller.transform(Controller.java:1346)

 

Thanks,

Srinivasan Ranganathan

 


CONFIDENTIALITE & RESPONSABILITE: Ce message, incluant ses fichiers joints, est transmis pour l'usage exclusif des personnes à qui il est destiné et peut contenir des renseignements confidentiels ou assujettis au secret professionnel. Il est strictement interdit d'en faire toute autre distribution, copie ou divulgation. Si vous n'êtes pas le destinataire visé ou que vous avez reçu ce message par erreur, veuillez nous en aviser immédiatement par réponse à ce courriel ou par téléphone au 514.989.3141 et en détruire toute copie (incluant les fichiers joints). Merci.
EquiSoft Inc décline toute responsabilité relativement aux opinions exprimées dans ce courriel ou aux conséquences de tout virus informatique qu’il pourrait transmettre.

CONFIDENTIALITY & LIABILITY: This message, including any attachments, is intended only for the use of the individuals to whom it is addressed and may contain information that is privileged or confidential. Any other distribution, copying or disclosure is strictly prohibited. If you are not the intended recipient or have received this message in error, please notify us immediately by reply e-mail or by phone at 514.989.3141 and permanently delete any copy of this message (including any attachments). Thank you.
EquiSoft Inc does not accept liability for the views expressed in the email or for the consequences of any computer viruses that may be transmitted with it.

 
 
------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and, 
should the need arise, upgrade to a full multi-node Oracle RAC database 
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
 
 
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
saxon-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/saxon-help 

 



CONFIDENTIALITE & RESPONSABILITE: Ce message, incluant ses fichiers joints, est transmis pour l'usage exclusif des personnes à qui il est destiné et peut contenir des renseignements confidentiels ou assujettis au secret professionnel. Il est strictement interdit d'en faire toute autre distribution, copie ou divulgation. Si vous n'êtes pas le destinataire visé ou que vous avez reçu ce message par erreur, veuillez nous en aviser immédiatement par réponse à ce courriel ou par téléphone au 514.989.3141 et en détruire toute copie (incluant les fichiers joints). Merci.
EquiSoft Inc décline toute responsabilité relativement aux opinions exprimées dans ce courriel ou aux conséquences de tout virus informatique qu’il pourrait transmettre.

CONFIDENTIALITY & LIABILITY: This message, including any attachments, is intended only for the use of the individuals to whom it is addressed and may contain information that is privileged or confidential. Any other distribution, copying or disclosure is strictly prohibited. If you are not the intended recipient or have received this message in error, please notify us immediately by reply e-mail or by phone at 514.989.3141 and permanently delete any copy of this message (including any attachments). Thank you.
EquiSoft Inc does not accept liability for the views expressed in the email or for the consequences of any computer viruses that may be transmitted with it.