RE: [Babeldoc-devel] Multithreading problems...

[McDonald, B. <Bru...@ba...>]

David,

Thank you - I will be working on this.  I would appreciate any help that we can get here.

THIS IS THE LAST BIG ISSUE BEFORE BABELDOC 1.2 GOES FINAL.  

PLEASE HELP.

regards,
Bruce.

-----Original Message-----
From: David Kinnvall [mailto:dav...@al...]
Sent: Tuesday, November 04, 2003 9:25 AM
To: bab...@li...
Subject: Re: [Babeldoc-devel] Multithreading problems...


Hi everybody!

David Kinnvall (me) wrote at a previous time:
[stuff about my multithreading problems and the lack of datapoints]

I now have a very small setup that triggers the problem, suggesting
that there is nothing really particularly strange about my setup,
but rather that it should be reproducible (debuggable) for others.

Interesting datapoint: The *first* thread seems to always survive.
It is the remaining one(s) that crashes, sooner or later.

Relevant config snippets to (hopefully) be able to reproduce:

Setup a pipeline called whatever and a directory scanner that
feeds it, using the configs outlined below. Adjust as necessary
for your environment(s).

feeder/config.properties:
=========================
# Scanner feeder implementation
scanner.type=asynchronous
#scanner.type=synchronous
scanner.queue=disk
scanner.queueDir=scanner/queue
scanner.queueName=scanner
scanner.poolSize=3

pipeline/whatever/whatever.properties:
======================================
entryStage=entry

entry.stageType=Null
entry.nextStage=enricher
entry.tracked=false

enricher.stageType=XpathExtract
enricher.nextStage=writer
enricher.XPath.foo=/foo/@name

writer.stageType=FileWriter
writer.nextStage=null
writer.outputFile=output/whatever/${ticket.getValue()}_${document.get("foo")}.xml

Input document(s) - create a couple of these, named <anything>.xml:
===================================================================
<?xml version="1.0" encoding="iso-8859-1" standalone="no" ?>
<foo name="bar1"/>

Now, armed with an active Babeldoc instance, using config as
above, give a bunch (say 4) of input xml documents of the above
form to the scanner. For me it (almost) every time bombs for the
second and third threads in the enricher stage. You might have to
retry a few times, but for me it is very consistent, with an NPE
in VariableProcessor.mustExpand()

Important: You need to give multiple input documents to the
scanner at once, to trigger the problem *quickly* ...but...

However: It is certainly possible to trigger it with one single
document at a time as well! Just keep feeding the scanner+pipeline
one of the documents, watch it process it (successfully at least
once), and keep going, until a different thread from the first
gets to work on it. Boom.

The first thread seems to correctly get its configuration for
pipeline stages et al, but the subsequent configurations do
have a tendency to lack values, particularly, as Bruce noticed,
the ones of "complex" nature. A dump of the config data for an
XpathExtract stage, as configured in the snippet above, during
exception handling of one of the NPEs shows this (formatted)

com.babeldoc.core.option.ConfigData@11c2b67[
   name=enricher,value=<null>,children={
     stageType=com.babeldoc.core.option.ConfigData@659db7[
       name=stageType,value=XpathExtract,children=<null>
     ],
     XPath=com.babeldoc.core.option.ConfigData@1556d12[
       name=XPath,value=<null>,children={
         foo=com.babeldoc.core.option.ConfigData@edf389[
           name=foo,value=<null>,children=<null>
         ]
       }
     ],
     nextStage=com.babeldoc.core.option.ConfigData@16be68f[
       name=nextStage,value=writer,children=<null>
     ]
   }
]

from which we learn that what should have been
   enricher.XPath.foo=/foo/@name
has instead become
   enricher.XPath.foo=<null>
which wasn't expected by the pipeline stage. Hence the NPE.

As before, I need help in resolving this. I have so far been
utterly unable to find a solution on my own. I hope that the
additional data in the above text might be of some assistance
for others to have a go at the problem as well, or at least
verify that the problem exists outside of my particular setup.

Best regards,

David



-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?   SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
_______________________________________________
Babeldoc-devel mailing list
Bab...@li...
https://lists.sourceforge.net/lists/listinfo/babeldoc-devel