Menu
▾
▴
Re: [Opaltoolkit-users] Problem reloading opal
|
From: Anthony B. <ant...@ir...> - 2009-07-02 14:19:11
|
Hi,
We're using edu.sdsc.nbcr.opal.manager.DRMAAJobManager (slightly
modified to work with DRMAA.jar 6.1, but the changes are minor).
Issuing ant clean and ant install didn't solved the problem.
Here's a full log of a tomcat session with the following actions:
started tomcat, launched a job (which worked), stopped opal2, wait,
restarted opal2, launched a job
Jul 2, 2009 1:05:03 PM org.apache.catalina.core.AprLifecycleListener
lifecycleEvent <------- starting tomcat
INFO: The Apache Tomcat Native library which allows optimal performance
in production environments was not found on the java.library.path:
/softs/local/java/jdk1.5.0_12/jre/lib/amd64/server:/softs/local/java/jdk1.5.0_12/jre/lib/amd64:/softs/local/java/jdk1.5.0_12/jre/../lib/amd64:/usr/local/sge/lib/lx24-amd64:/usr/local/sge/lib/lx24-amd64
Jul 2, 2009 1:05:03 PM org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8383
Jul 2, 2009 1:05:03 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 372 ms
Jul 2, 2009 1:05:03 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Jul 2, 2009 1:05:03 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.27
Jul 2, 2009 1:05:03 PM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
Jul 2, 2009 1:05:03 PM org.apache.catalina.loader.WebappClassLoader
validateJarFile
init server...
*******engineType:0
*******engineType:0
*******engineType:0
Jul 2, 2009 1:05:15 PM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8383
Jul 2, 2009 1:05:15 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
Jul 2, 2009 1:05:15 PM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/12 config=null
Jul 2, 2009 1:05:15 PM org.apache.catalina.storeconfig.StoreLoader load
INFO: Find registry server-registry.xml at classpath resource
Jul 2, 2009 1:05:15 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 12435 ms
[http-8383-Processor23] ERROR manager.DRMAAJobManager - Can't set hard
limit - The hardRunDurationLimit attribute is not supported. <----
this is a successfull job launching
Exception in thread "HSQLDB Timer @609a18a0"
java.lang.NullPointerException <----- this is when I "Stop" opal2
at org.hsqldb.lib.HsqlTimer.nextTask(Unknown Source)
at org.hsqldb.lib.HsqlTimer$TaskRunner.run(Unknown Source)
at java.lang.Thread.run(Thread.java:595)
<----- Here I
"Start" opal2 after a moment (no log) and I launch a job (no log here
but a SOAP fault received by clients)
Luca Clementi a écrit :
> Anthony Bretaudeau wrote:
>> Hi,
>> Now that I use the stop opal webapp, wait a minute and then restart it I
>> no longer have this HSQLDB problem (although I still get this when
>> stopping:
>> Exception in thread "HSQLDB Timer @4e8890da"
>> java.lang.NullPointerException
>> at org.hsqldb.lib.HsqlTimer.nextTask(Unknown Source)
>> at org.hsqldb.lib.HsqlTimer$TaskRunner.run(Unknown Source)
>> at java.lang.Thread.run(Thread.java:595)
>> )
>>
>
> Anthony,
> you can ignore this error.
>
>> However, I still have a problem: services are usually not accessible
>> after restarting. I get this kind of error message from clients:
>>
>
> There is a problem in your configuration, but this is a very generic
> error message.
>
>
> What kind of jobmanager are you using?
> Can you try to issue one last, ant clean, ant install?
>
> Can you send me the $CATALINA_HOME/logs/catalina.out?
>
>
> Sincerely,
> Luca
>
>
>> Exception in thread "main" AxisFault
>> faultCode:
>> {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
>> faultSubcode:
>> faultString: java.lang.reflect.InvocationTargetException
>> faultActor:
>> faultNode:
>> faultDetail:
>> {http://xml.apache.org/axis/}hostname:genoweb3
>>
>> java.lang.reflect.InvocationTargetException
>> at
>> org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:221)
>>
>> at
>> org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:128)
>>
>> at
>> org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)
>>
>> at
>> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:604)
>>
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1750)
>>
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2906)
>>
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:624)
>>
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:116)
>>
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:486)
>>
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:810)
>>
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:740)
>>
>> at
>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:110)
>>
>> at
>> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1208)
>>
>> at
>> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:525)
>>
>> at javax.xml.parsers.SAXParser.parse(SAXParser.java:392)
>> at
>> org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227)
>>
>> at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)
>> at org.apache.axis.Message.getSOAPEnvelope(Message.java:424)
>> at
>> org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstandChecker.java:62)
>>
>> at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206)
>> at org.apache.axis.client.Call.invokeEngine(Call.java:2765)
>> at org.apache.axis.client.Call.invoke(Call.java:2748)
>> at org.apache.axis.client.Call.invoke(Call.java:2424)
>> at org.apache.axis.client.Call.invoke(Call.java:2347)
>> at org.apache.axis.client.Call.invoke(Call.java:1804)
>> at
>> edu.sdsc.nbcr.opal.AppServicePortTypeSoapBindingStub.launchJob(AppServicePortTypeSoapBindingStub.java:551)
>>
>> at
>> CustomOpalClient.CustomOpalClient.launchJob(CustomOpalClient.java:397)
>> at Test.main(Test.java:49)
>>
>> It seems like a server problem, maybe with axis? Is this a known
>> problem? Is there a solution (apart from restarting tomcat server)?
>> Thanks!
>>
>> Luca Clementi a écrit :
>>> Anthony Bretaudeau wrote:
>>>> Hi,
>>>> I've got a problem when reloading opal2 app from tomcat manager. After
>>>> doing so, registered webservices stop working and the only solution is
>>>> to restart tomcat (which is quite uncomfortable as we have other
>>>> apps in
>>>> production on the same tomcat server).
>>>> Looking at the log, I see some db related errors when reloading: see
>>>> below.
>>>> Can you help me?
>>>> Thanks!
>>> Anthony,
>>> we already saw this problem.
>>> It is caused by a a race condition of HSQL during the reload of the
>>> webapp, basically HSQL is restarted before the old HSQL is completely
>>> destroyed. It doesn't always happen, sometime the reload works well.
>>>
>>> If you use external database this problem won't appear.
>>>
>>> My suggestion is to stop the webapp, wait some seconds and then start
>>> the webapp again, this should avoid this race condition.
>>>
>>> Sincerely,
>>> Luca
>>>
>>>
|
