Re: [Opaltoolkit-users] problem launching opal jobs

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Luca,

On Thursday 25 June 2009 03:30:36 pm Luca Clementi wrote:
> but if you use fork, Opal will simply fork the job on the node where the
> tomcat is running.
>
> The problem you are experiencing is connected with the libdrmaa.so so if
> you use fork everything is fine.

I understand that, but given that the DRMAA stuff wasn't working correctly, it 
was nice to back off to a simpler setup and verify that that was indeed 
working before continuing to the more complicated setup.

> To update the LD_LIBRARY_PATH try running
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<your path to libdrmaa.so>

It wasn't clear to me whether this needs to be set when doing the 'ant 
install' stage, or when starting Tomcat so I did both.  I started with my 
working configuration with Fork, changed the opal.properties and the job's 
properties to use DRMAA, undeployed and deployed the application and verified 
that in each case I could see the libdrmaa via:

-bash-3.1$ ls $LD_LIBRARY_PATH
libattr.a    libdrmaa.so.0      libsite.a       libtorque.so.0.0.0
libcmds.a    libdrmaa.so.0.0.0  libtorque.a     libtorque.so.2
libdrmaa.a   liblog.a           libtorque.la    libtorque.so.2.0.0
libdrmaa.la  libnet.a           libtorque.so    pbs_sched.a
libdrmaa.so  libpbs.a           libtorque.so.0

I did this before executing any commands.

Here's what I see when I try to submit a job:

[root@agave opal-ws-2.0]# java edu.sdsc.nbcr.opal.GenericServiceClient -l 
http://agave.wustl.edu:8080/opal2/services/pdb2pqr -r 
launchJob -a "--ff=amber sample.pdb output.pqr" -f samples/sample.pdb

Reading command line arguments
Service URL: http://agave.wustl.edu:8080/opal2/services/pdb2pqr
Invoking operation: launchJob

Command line arguments: --ff=amber sample.pdb output.pqr
Making non-blocking invocation on Opal service -
Exception in thread "main" AxisFault
 faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
 faultSubcode: 
 faultString: java.lang.reflect.InvocationTargetException
 faultActor: 
 faultNode: 
 faultDetail: 
        {http://xml.apache.org/axis/}hostname:agave.wustl.edu

java.lang.reflect.InvocationTargetException
        at 
org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:221)
        at 
org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:128)
        at 
org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)
        at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown 
Source)
        at 
org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown 
Source)
        at 
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown 
Source)
        at 
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown 
Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
        at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
        at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
        at 
org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227)
        at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)
        at org.apache.axis.Message.getSOAPEnvelope(Message.java:424)
        at 
org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstandChecker.java:62)
        at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206)
        at org.apache.axis.client.Call.invokeEngine(Call.java:2765)
        at org.apache.axis.client.Call.invoke(Call.java:2748)
        at org.apache.axis.client.Call.invoke(Call.java:2424)
        at org.apache.axis.client.Call.invoke(Call.java:2347)
        at org.apache.axis.client.Call.invoke(Call.java:1804)
        at 
edu.sdsc.nbcr.opal.AppServicePortTypeSoapBindingStub.launchJob(AppServicePortTypeSoapBindingStub.java:551)
        at 
edu.sdsc.nbcr.opal.GenericServiceClient.main(GenericServiceClient.java:311)

and from the Tomcat logs:

2009-06-26 13:18:12,492 INFO  
edu.sdsc.nbcr.opal.AppServiceImpl.launchJob(AppServiceImpl.java:237) - called
2009-06-26 13:18:12,494 INFO  
edu.sdsc.nbcr.opal.AppServiceImpl.retrieveAppConfig(AppServiceImpl.java:1038) - 
called
2009-06-26 13:18:12,500 INFO  
edu.sdsc.nbcr.opal.AppServiceImpl.writeAppInput(AppServiceImpl.java:944) - 
called
2009-06-26 13:18:12,506 INFO  
edu.sdsc.nbcr.opal.util.Util.getRemoteDN(Util.java:62) - Client's DN: Unknown 
client
2009-06-26 13:18:12,508 INFO  
edu.sdsc.nbcr.opal.util.Util.getRemoteIP(Util.java:43) - Client's IP: 
127.0.0.1
2009-06-26 13:18:12,509 INFO  
edu.sdsc.nbcr.opal.state.HibernateUtil.saveJobInfoInDatabase(HibernateUtil.java:72) - 
called
2009-06-26 13:18:12,515 INFO  
edu.sdsc.nbcr.opal.AppServiceImpl.launchApp(AppServiceImpl.java:579) - Using 
job manager class: edu.sdsc.nbcr.opal.manager.DRMAAJobManager

I'm just taking a WAG here, but the client complains:

 faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException

and the server complains:

edu.sdsc.nbcr.opal.util.Util.getRemoteDN(Util.java:62) - Client's DN: Unknown 
client
2009-06-26 13:18:12,508 INFO  
edu.sdsc.nbcr.opal.util.Util.getRemoteIP(Util.java:43) - Client's IP: 
127.0.0.1

This makes me wonder if there is some authentication problem.

Is there a problem if I'm running the client as root?  If I'm running it from 
the same system that the server is running from?  

Malcolm

-- 
Malcolm Tobias
314.362.1594