FAQ

Frequently asked questions for UNICORE.

General

What is UNICORE?

UNICORE (Uniform Interface to Computing Resources) offers a ready-to-run Grid system including client and server software. UNICORE makes distributed computing and data resources available in a seamless and secure way through intranets and internet.

Where do I get UNICORE?

You can download UNICORE packages here: http://www.unicore.eu/download/unicore6/

Where do I get information about UNICORE?

You can find all the UNICORE documentation at the UNICORE web site: http://www.unicore.eu/documentation/ and at the wiki: http://sourceforge.net/apps/mediawiki/unicore/

Where do I get help or report bugs?

You can post support requests to the unicore-support mailing list: unicore-support@lists.sourceforge.net

You can post bug reports and feature request at UNICORE bug trackers

Alternatively, you can discuss bugs and feature requests on the unicore-devel list (you will have to subscribe): unicore-devel@lists.sourceforge.net

So, how do I start?

You can try out the UNICORE LiveCD or the public UNICORE Testgrid. You can also install a demo installation with the graphical installer of the Core Server Bundle. Please refer to UNICORE 6 in 30 minutes for further information.

What are the prerequisites for UNICORE?

In order to run the UNICORE server or client components, all you need is a Java Runtime Environment (JRE) 1.7 or higher, we recommend OpenJDK, but Oracle and IBM should work too. Since all components are platform independent, they will run under Linux, MAC and windows likewise.

Does UNICORE have a resource management system/batch system?

No. UNICORE is a Grid Middleware, it submits jobs to already installed resource management systems/batch systems.

Do I need a resource management system/batch system?

No. UNICORE can run without a resource management system/batch system, jobs are only forked then. But in production one normally wants a resource management system/batch system.

Is UNICORE compatible with the Globus Toolkit?

UNICORE and Globus have very different security models, basic services and interfaces, and are thus not directly interoperable. On the other hand, UNICORE can use GridFTP for data staging, and can retrieve certificates from a MyProxy CA.

Clients

What is the meaning of the various job states shown in the client?

UNICORE shows the following job states:

  • STAGINGIN - the server is staging in data from remote sites into the job directory
  • READY - job is ready to be started
  • QUEUED - job is waiting in the batch queue
  • RUNNING - job is running
  • STAGINGOUT - execution has finished, and the server is staging out data to remote sites
  • SUCCESSFUL - all finished, no errors occured
  • FAILED - errors occured in the execution and/or data staging phases
  • UNDEFINED - this state formally exists, but is not seen on clients

How to run URC on Mac OSX

Because of issues with Apple Java 6, it is required to install a new Java Version on your Machine. For the current version of URC (7.1.2) it is required to manipulate configuration files after installing Java; for future releases we plan to improve the situation.

After installing for example Oracle JDK 8 (1.8.0_40) there are two ways to get the URC working:

First solution (administrator privileges required):

Get the current path of your Java installation's configuration file, for example:
"/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Info.plist"
and manipulate this file:

...
<key>JVMCapabilities</key>
<array>
    <string>CommandLine</string>
</array>
...

...
<key>JVMCapabilities</key>
<array>
    <string>CommandLine</string>
    <string>JNI</string>
    <string>BundledApp</string>
</array>
...

Second solution (no additional privileges required):

Manipulate the content of the file "<URC_PATH>/Eclipse.app/Contents/Info.plist"

...
<key>Eclipse</key>
    <array>
        <string>-vm</string><string>/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/server/libjvm.dylib</string>
    </array>
...

and the content of the file "<URC_PATH>/UNICORE_Rich_Client.app/Contents/MacOS/UNICORE_Rich_Client.ini"

...
-vm
/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/server/libjvm.dylib
...

How to install UCC on Windows?

Please refer to [Windows_Issues].

URC - Slow start on Windows

Some Anit-Virus software is analysing software bevore execution, which significantly slows down the start of URC.
For Kaspersky Endpoint Security 10.2.2 we observed a delay of approx. 2 minutes.

Certificates

What kind of certificates do I need?

Each server component (only for the TSI it is optional) needs one X.509 private/public key pair signed by a CA (Certification Authority) as identity. Each user needs a signed X.509 private/public key pair which has to be loaded in the keystore of the client. Additionally, you need the certificates of the CAs.

For testing purposes, the UNICORE components come with demo certificates. Never use these in publicly available servers.

How do I export my certificate from Windows Internet Explorer?

The Internet Explorer saves certificates in the Registry. Click Start->Settings->Control Panel, then double-click Internet. On the Content tab, click Personal, click a certificate you want to export, and then click Export. In the wizard, check "export private key" and the option "include all certs in the path if possible". The resulting pfx file is actually a p12 file. When using it with UCC, set the storetype=pkcs12 option in the preferences file.

How do I export my certificate from Firefox?

In Firefox, click Edit -> Preferences, go to the Advanced tab and click Encryption, then View Certificates. Click a certificate you want to export, and then click Backup. The resulting file is a p12 file. When using it with UCC, set the storetype=pkcs12 option in the preferences file.

Where to put the CA (Certification Authority) certificates?

All network connections between the UNICORE components use client-authenticated SSL (Secure Sockets Layer), i.e. both sides of the connection check that they trust the other side. "Trust" means that the CA is checked.

If you use a single CA for all your certificates, the configuration is rather simple: Each server component needs to know the CA certificate, and additionally the CA has to be loaded in the client's keystore.

If you use multiple CA's, consider how the UNICORE components work together: The client communicates with the Gateway, so the Gateway has to know the user's CA and the client has to know the CA of the Gateway. The Gateway also communicates with the Registry and UNICORE/X, so the Registry and UNICORE/X should know the Gateway's CA and vice versa. Additionally, the UNICORE/X communicates with the XUUDB, so both components need to know each other's CA certificates.

Which certificates go into the XUUDB?

When adding a new user to the XUUDB, you need his signed certificate (public key). If you run the XUUDB in DN mode, the distinguished name (DN) of the user's public key will suffice.

How to obtain user certificates?

You can generate a certificate request within the URC: Open the Keystore view and select Generate Certification Request from the context menu (URC) or Actions menu (GPE client). The client will create a new private key, which is automatically stored in the keystore editor, and a certification request, which you are asked to save to disk. Send the certification request to a CA (Certification Authority).

You will get a signed certificate (public key) and a CA certificate in return. Store them on disk and click import them into the keystore.

How do I create my own certificates?

To set up your own certificate authority (CA) to issue user and server certificates, refer to [Create_Own_CA].

Connect problems, exceptions, errors

The following error messages are copied from UCC's output and log file, but similar error messages can be seen in the URC and in server logs, too; the solutions apply accordingly. When you encounter problems using UCC, try using the -v option and look in the ucc.log file in the current working directory.

What to do when a "Illegal key size" message appears?

When you see the exception messages containg the words "illegal key size",
similar to

FATAL AuthSSLProtocolSocketFactory - exception unwrapping private key -
java.security.InvalidKeyException: Illegal key size
java.io.IOException: exception unwrapping private key -
java.security.InvalidKeyException: Illegal key size
at org.bouncycastle.jce.provider.JDKPKCS12KeyStore.unwrapKey(Unknown Source)
at org.bouncycastle.jce.provider.JDKPKCS12KeyStore.engineLoad(Unknown Source)
at java.security.KeyStore.load(Unknown Source)
....

you have to update security files of your Java installation. Download the "Unlimited Strength Jurisdiction Policy Files" from http://java.sun.com/javase/downloads/index.jsp under topic Additional Resources 'Other Downloads': Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files and install it. Extract the files local_policy.jar and US_export_policy.jar and place them into the lib/security directory of your java installation.

What to do when a "Unexpected number of X509Data: for Signature" message appears?

When you see this message in a ucc.log file or in a log file of a server, you have to make sure that the key in your keystore has an alias. If in doubt, use the Portecle (http://portecle.sf.net) tool (or an equivalent tool) to check your keystore and assign an alias to the key entry.

What to do when a "PKIX path building failed" message appears?

When you see the following message in a logfile

Cannot contact registry
org.codehaus.xfire.XFireRuntimeException: Could not invoke service..
Nested exception is org.codehaus.xfire.fault.XFireFault: Couldn't send  message.
...
Caused by: javax.net.ssl.SSLHandshakeException:  sun.security.validator.ValidatorException :
PKIX path building failed:  sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target

you are missing the CA certificate of the Gateway you are trying to connect to in your truststore.

What to do when a "Unknown Certificate" message appears?

A client is connecting to your server that does not trust the CA of your server certificate.

What to do when a "bad certificate" message appears?

When you see the following message in a logfile

...
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert:  bad_certificate
...

the Gateway is not trusting you (i.e. the signer of your private key).
Make sure you are using the right user certificate and contact the server admin.

What to do when a "Given final block not properly padded" exception appears?

When you see the following message in a logfile

...
java.io.IOException: failed to decrypt safe contents entry:
javax.crypto.BadPaddingException: Given final block not properly padded
...

you have probably given the wrong password.

What to do when a "Invalid keystore format" exception appears?

When you see the following message in a logfile

...
java.io.IOException: Invalid keystore format
...

and if you are using a p12 file as keystore, use the storetype=pkcs12 option in UCC's preferences file. (You also should provide a seperate jks truststore which holds the CA certificates, use the truststore= and truststorePassword= options.__

If you are using a jks file, skip the storetype= option or set it to jks.

What to do when a "toDerInputStream rejects tag type 66" message appears?

When you see the following message in a logfile

...
java.io.IOException: toDerInputStream rejects tag type 66

you have probably exported your key from Internet Explorer, see How do I export my certificate from Windows Internet Explorer.

What to do when a "signature is required for <CreateTSR>" message appears?

When you see the following message in UCC's output

Can't create target system.
The root error was: org.codehaus.xfire.fault.XFireFault:
Authentication failed on &lt;TargetSystemFactoryService&gt;: signature is  required for &lt;CreateTSR&gt;

you are probably using a p12 file. Specify the alias of your private key in the preferences file, e.g.:

alias=My Alias

I am running Debian and keep getting "Network unreachable"

This is a problem with Debian and/or Java and a detailed explanation and a working workaround can be found at the following website: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=560142

When listing storages I get Access denied for my DN on resource Enumeration[295cd722-...]

This happens most likely when the server side is using an outdated XACML policy that does not allow users to list storages from StorageFactories.

The simplest solution is to ask your server administrator to update to the XACML 2 policies. They have been shipped with the UNICORE distribution since the 6.4.0 release.

In order to use the right policies, apply the following changes in uas.config on the server side:

#
# the XACML config file which contains the list of security policy files
#
uas.security.accesscontrol.pdp.config=conf/xacml2.config

#
# the XACML PDP implementation class
#
uas.security.accesscontrol.pdp=eu.unicore.uas.pdp.local.LocalHerasafPDP

The configuration properties should already be in that file, but you need to change them to the values given above.

When invoking UCC on Windows, I get a mysterious "Syntax Error" and nothing else

When you encounter this problem, there is a good chance that the path of your UCC is rather long. We have noticed that adding all required libraries to the CP (classpath) variable can lead to a very long string which the Windows BAT interpreter refuses to handle. Try to make your path shorter, i.e. move the UCC directory up in the file system hierarchy.

For example, if your original path to the UCC directory was

 C:\Users\user\UCC\ucc-commandline-client-6.5.0-all\ucc-command-line-client\

please move the last directory up one level, or, even better, put its contents into C:\Users\user\UCC. This will ensure that the class path does not grow too large. We are looking for a solution to overcome this limitation.

What to do when a service throws the security exception "Cannot set up certs for trusted CAs" / "Cannot locate policy or framework files!"

When you encounter this problem then your java version cannot access the policy files. This may happen when you are using IBM java. We suggest to switch to java of openjdk.

The Unicore Rich Client crashes when I try to open Help>Welcome.

If your Crash report looks similar to this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fdebc368617, pid=21587, tid=140597561623680
#
# JRE version: OpenJDK Runtime Environment (7.0_51) (build 1.7.0_51-b00)
# Java VM: OpenJDK 64-Bit Server VM (24.45-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libsoup-2.4.so.1+0x71151]  soup_session_feature_detach+0x11
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping,
# try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /tmp/jvm-27387/hs_error.log
#
# If you would like to submit a bug report, please include
# instructions on how to reproduce the bug and visit:
#   http://icedtea.classpath.org/bugzilla
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

you have to add the following options when starting Unicore from the command line prompt:
< -vmargs -Dorg.eclipse.swt.browser.DefaultType=mozilla>

(https://forums.opensuse.org/showthread.php/492348-eclipse-juno-crashs-on-OpenSuse-13-1/page3)
(This Problem was detected on openSUSE 13.1 (Bottle) (x86_64) and may be related to default settings)

Unicore Rich Client crashes after trying to gain access to the TestGrid.

This is most likely caused by the GTK-Layout you are using.
Open System Settings>Application Appearance> GTK ()
->Change the GTK2 theme to anything but oxygen-gtk
(
This may also be found under “Customize Look and Feel” or similar depending on the OS you are using)
(https://bugs.launchpad.net/ubuntu/+source/openjdk-7/+bug/1241532 #2)
(This Problem was detected on openSUSE 13.1 (Bottle) (x86_64) and may be related to default settings)

The problem may also occur when adding a Script job to a workflow and opening it for the first time.

Grid Layout

How many open ports in the firewall does UNICORE need?

Only the Gateway needs to be accessible from outside the firewall, using one port. The other components need not be accessible outside the firewall. It is possible to bypass the gateway for filetransfers, which will boost performance by a factor of 2, but will need a second open port to allow clients direct connections to the UNICORE/X server.

Another open port is required if you want to use the UFTP high-performance file transfer.
Please refer to the UFTP manual for details.

How many machines do I need for a UNICORE system?

You can install UNICORE on a single machine. For performance reasons, you might want to install the UNICORE/X and the Workflow engine on dedicated machines since they need the most resources (RAM and CPU). The XUUDB and Registry need only very little resources.

It depends on whether you want to use the workflow system or not.

The minimal setup without workflow requires only one machine for Gateway
and UNICORE/X, the TSI (and the optional UFTP server) runs on the login node. You need not use the XUUDB here,
use a map file instead.

With the workflow system, we recommend three machines. One hosts the Gateway, global Registry, and XUUDB, one the workflow system, and one the UNICORE/X server. The TSI (and the optional UFTP server) runs again on the login node of your cluster. For low load, two machines would be enough.

Can the UNICORE Gateway serve multiple UNICORE/X or registries?

Yes. If you use static initialisation, enter all VSites in the connections.properties file of the gateway.

Can I share the Registry and XUUDB between multiple UNICORE6 servers?

Yes. The XUUDB is accessed using web service calls, and is configured in the UNICORE server's main configuration file. For setting up a shared registry, see this guide.

May I have one UNICORE/X with multiple nodes?

Yes, if you run a batch system like Torque on your nodes. The VSite acts as "front end", so you can access your little cluster through UNICORE.

Then, if I send a job to a VSite, will UNICORE send the job to a free node?

The batch system will do that, yes. It's important to understand that UNICORE does not replace a local batch system.

Must the UNICORE components run as root?

The Gateway, UNICORE/X, and other services should run as normal user. The TSI and UFTPO must run as root since they do setuid to the actual user.

How to setup UNICORE servers behind a NAT router?

The important thing is to understand that the Gateway is the one that needs to be accessed from "outside". This means that all the UNICORE/X components must use the external Gateway address in the "container.baseurl" property (defined in the wsrflite.xml file). The Gateway config file gateway.properties however defines the internal address (host and port) that the Gateway uses. Note that the installers (both graphical and tgz) do not support this scenario, it has to be done manually.

Server Components

What Server Components are there?

A good starting point is the overview of the UNICORE architecture (clickable).

  • Gateway
  • UNICOREX
  • Registry
  • XUUDB
  • TSI
  • Unity
  • UFTPD
  • Workflow
  • Servorch

How to set up UNICORE server components on Windows?

Please refer to [Windows_Issues].

How can I install UNICORE on my cluster running SGE, LoadLeveler...?

The UNICORE core server package includes documentation on how to access your favourite batch system. See this guide.

My batch jobs fail with "Job was not completed (no exit code file found)"

This happens when UNICORE does not get valid information from the batch system (e.g. via "qstat" or similar command). Make sure that the user id used for this purpose can see all jobs in the qstat listing. Also, check if the GetStatusListing.pm Perl module is correct. This can be slightly system-dependent, as the qstat output depends on the chosen batch system and possibly its configuration.

[Category:Admins] [Category:Users]


Related

Wiki: CSC2012
Wiki: Create_Own_CA
Wiki: GridKa2010
Wiki: GridKa2011
Wiki: IWSGC2010
Wiki: Main_Page
Wiki: Windows_Issues

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks