Java-REST-client-r2317

Using e-Science Central REST API in Java

• Storage API client
  • Manipulating e-Science Central folders
  • Uploading data to e-Science Central
  • Downloading data from e-Science Central
  • Uploading document metadata
• Workflow API client
  • Running workflows
  • Monitoring workflow progress
  • Terminating workflows
• Datasets API client
• Configuring clients to access the server over HTTPS

To quickly start using the REST API of the system in Java we provide a Java client library. To use it, the relevant .jar file needs to be included within your project.
The exact process for doing this depends upon the IDE that you use for development, but broadly the process involves:

• Downloading the correct api-model.jar file
• Placing this file in the libraries for your project
• Including the api-model.jar file in any libraries you distribute with your code

Currently, the preferred way to get the api-model.jar file is to checkout sources from this repository and build the project from sources. Follow the building instructions page if you haven't got the source code compiled yet.
Once this has been done you can find the jar file under code/server/APIModel/target and include it in your project. The file name will have suffix with the current version of the system, e.g. api-model-3.1-SNAPSHOT.jar.

Now, it should be possible to import the relevant API client packages into your code:

import com.connexience.api.model.*;
import com.connexience.api.*;

The com.connexience.api package provides three classes you can use to access your e-Science Central server: StorageClient, WorkflowClient and DatasetsClient. The following sections will show details about how to use each of them to do the most common tasks.

Storage API client

The java code to connect to an e-Science Central server is shown below. In this code you should replace arguments passed to the StorageClient constructor with the values specific to your environment. Look at the javadoc attached to the constructor to have more information about the required arguments.

import com.connexience.api.*;
import com.connexience.api.model.*;

public class PrintUser
{
   public static void main(String[] args) throws Exception
   {
      // Create a new storage client which:
      //   connects to localhost on port 8080 over insecure HTTP connection
      //   using credentials myusername/mypassword.
      StorageClient client = new StorageClient("localhost", 8080, false, "myusername", "mypassword");

      // Get the current user object.
      EscUser currentUser = client.currentUser();

      // Print out the name of the current user.
      System.out.println(currentUser.getName());
   }
}

The code above creates a client reference that can connect to an e-Science Central server, authenticates a user and then prints the full name of that user to the console window. If the code is successful, the output should be a single line with the full name of the authenticated user on it.
If this step succeeds, then you have a valid e-Science Central connection that you can use to upload, download and manage data within the server.

Manipulating e-Science Central folders

All documents, data and workflows within e-Science Central are contained within folders (a logical container represented by class EscFolder), the Storage API contains a number of methods for accessing, modifying and creating folders.

Each user within the system is assigned a "Home Folder", which is created when the user registers. To access user's home folder you need a connected client object (see example above) and call:

EscFolder homeFolder = client.homeFolder();

This will return a reference to the users home folder that can be used as a base for subsequent calls.

Note that user's home folder is the base folder for all data stored by that user when they are not operating within a project.
Each project has it's own folder assigned to it which is used to store project data.
You can read more about [projects] in the users guide and about [storing data within a project] later in this section.

With the home folder reference you can then list all documents contained in this folder:

EscDocument[] docs = client.folderDocuments(homeFolder.getId());

or list all subfolders within this folder:

EscFolder[] folders = client.listChildFolders(homeFolder.getId());

Then you can use the return array to descend through the directory hierarchy using items from the folders array instead of homeFolder.

In addition to listing documents, the Storage API client provides methods for creating and manipulating folders. To create a child folder called "New Folder" within user's home folder you can use:

EscFolder child = client.createChildFolder(homeFolder.getId(), "New Folder");

Note: if the parent folder already contains a folder with the same name, a reference to that folder will be returned and no new folder will be created.

...TO BE CONTINUED ...

Uploading data to e-Science Central

The Storage API client provides a number of convenience methods to upload data to the e-Science Central storage. The simplest upload method is to upload a file denoted by a Java File object.
For example, the following code uploads file "some-text.txt" into user's home directory on the e-SC server.

import java.io.File;

...

File fileToUpload = new File("some-text.txt");
EscFolder home = client.homeFolder();
EscDocumentVersion version = client.upload(home, fileToUpload);
EscDocument uploadedDocument = client.getDocument(version.getDocumentRecordId());

Note: that if file "some-text.txt" exists before the upload, a new version of the same document will be created.

To have more control over the upload process (e.g. to upload a file with different name or upload from an InputStream rather than file) it is possible to carry out the following tasks:

import java.io.FileInputStream;

...

EscFolder home = client.homeFolder();
EscDocument document = client.createDocumentInFolder(home.getId(), "other-text.txt");
File fileToUpload = new File("some-text.txt");
long streamLength = fileToUpload.length();
try (FileInputStream stream = new FileInputStream(fileToUpload)) {
    EscDocumentVersion version = client.upload(document, stream, streamLength);
}

... TO BE CONTINUED ...

Downloading data from e-Science Central

The Storage API client offers a number of convenience methods to download data from the e-Science Central storage. The simplest form of download is to download the latest version of a document to a file stored on the local filesystem:

EscDocument documentToDownload = client.getDocument("1234");
File localFile = new File(documentToDownload.getName());
client.download(documentToDownload, localFile);

... TO BE CONTINUED ...

Uploading document metadata

Workflow API client

The java code to connect to an e-Science Central server is shown very similar to one presented for StorageClient. In this code you should replace arguments passed to the WorkflowClient constructor with the values specific to your environment. Look at the javadoc attached to the constructor to have more information about the required arguments.

import com.connexience.api.*;
import com.connexience.api.model.*;

public class PrintWorkflows
{
   public static void main(String[] args) throws Exception
   {
      // Create a new workflow client which:
      //   connects to localhost on port 8080 over insecure HTTP connection
      //   using credentials myusername/mypassword.
      WorkflowClient client = new WorkflowClient("localhost", 8080, false, "myusername", "mypassword");

      // Get the current user object.
      EscWorkflow[] userWorkflows = client.listWorkflows();

      // Print out the list of user's workflows.
      for (EscWorkflow w : userWorkflows) {
        System.out.println(w.getName() + " -- " + w.getId());
      }
   }
}

The code above creates a client reference that can connect to an e-Science Central server, authenticates a user, acquires a list of user's workflows and then prints their names and identifiers to the console window. Note that it may happen that the list is empty if the user does not have any workflows.
However, if this step succeeds and does not report any errors, then you have a valid e-Science Central connection that you can use to start, terminate and monitor workflows.

Running workflows

The most common patterns to execute workflows are:

1. Running a workflow pre-configured by a user.
2. Running a workflow using a specified file as the input.
3. Running a workflow using a set of block property overrides.

Running a workflow as it was configured by a user is the simplest method and involves just a single method call on the workflow client:

EscWorkflowInvocation invocation = client.executeWorkflow("1234");

Value 1234 in this example represents the id of a workflow, as displayed in the list produced by the previous example.

Note: you can also check the id of a workflow via the web UI -- simply right click on the workflow document and select Info from the context menu.

Note: the execute workflow methods always return an EscWorkflowInvocation object. This object can later be used to monitor progress or terminate the workflow invocation.

To execute a workflow using a specified input file, the workflow must first be configured to take a document reference as an input parameter. The process of doing this is outlined in the users guide. Once the workflow has been configured correctly,
the following code will execute workflow with id 1234 using document with id 324 as an input:

EscWorkflowInvocation invocation = client.executeWorkflowOnDocument("1234", "324");

To execute a workflow with a set of block property overrides, the process is similar, however, instead of a document id a suitable parameter list must be constructed and passed as an argument.

For example, the following code performs the same execution as the code above, but uses an parameter list to specify the input file and a block to pass it to:

EscWorkflowParameterList params = new EscWorkflowParameterList();
params.addParameter("myInputBlock", "Source", "324");
EscWorkflowInvocation invocation = client.executeWorkflowWithParameters("1234", params);

For this code to run successfully, workflow with id 1234 must include block ImportFile with name myInputBlock. The ImportFile block has property Source that denotes id of a file to import from the data store to the workflow invocation space.

Using the last method to run workflows, you may set (almost) any property of any block in a workflow. The values provided in the parameter list always override property settings defined in the workflow by the workflow designer. Thus this methods is a very flexible way to configure and execute workflows.

Monitoring workflow progress

Terminating workflows

Datasets API client

... TO BE CONTINUED ...

Configuring clients to access the server over HTTPS

During testing and development it may happen that you want to access an e-Science Central server over HTTPS but the server identifies itself with a self-signed certificate.
In such cases when running your client you may see exception:

sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

This means that the Java runtime is unable to trust the self-signed certificate and must be forced to do so. There are various methods to configure Java to accept a self-signed certificate from the server but one of the simplest is to set the trust store that includes the server's certificate when starting the client.

First the certificate needs to be imported to a java keystore:

keytool -import -alias esc-server -file server-certificate.cer -keystore mytruststore.jks -storepass mypassword -noprompt

And then properties javax.net.ssl.trustStore and javax.net.ssl.trustStorePassword must be set like:

java -Djavax.net.ssl.trustStore=mytruststore.jks -Djavax.net.ssl.trustStorePassword=mypassword -cp api-model.jar MyClient

Details of this procedure are described in the examples section of the Java Secure Socket Extension Reference Guide.