SignServer / Discussion / Help: My contribution to SignServer archivers

Hello to all,

During my tests I've experienced some performance problems when archiving SignServer responses on a database. In the forum I've discovered that this is a known problem, so I'd like to share this archiver I've made in order to accomplish something similar to the OldDatabaseArchiver but that performs at very fast speeds. The only difference is that the stored data is the Base64 encoding of the data (as byte) and not the serialization of the object.

I've tested successfully with a Time Stamp Signer: on my testing environment (SignServer 3.2.1 on JBoss 5.1.0, remote Oracle 11g) I've passed from 3 seconds to archive the time stamp response to 0.06 seconds! ;-) I've also tested stressing SignServer with 100 threads requesting 10 time stamps in parallel, the test completed in 55 seconds. Instead 10 threads requesting 10 time stamps completed in 7 seconds.

If the project developers want to include it in the project (maybe adjusting it), I'll be very glad. Unfortunately I'm not a Java developer so for me it's very difficult to do unit test classes and source control on sourceforge.

In order to use it you have to put the following properties in the worker configuration:

ARCHIVERS=org.signserver.server.archive.DirectDatabaseArchiver
DIRECTARCHIVER.ISDISABLED=false
DIRECTARCHIVER.CONNECTIONNAME=SignServerDS

DIRECTARCHIVER.ISDISABLED is used to dinamically activate or deactivate the new archiver without touching the ARCHIVERS variable. Instead DIRECTARCHIVER.CONNECTIONNAME contain the Java DataSource name as registered on the application server (you can take it inside signserver-ds.xml).

Here is the code:

/*************************************************************************
 *                                                                       *
 *  SignServer: The OpenSource Automated Signing Server                  *
 *                                                                       *
 *  This software is free software; you can redistribute it and/or       *
 *  modify it under the terms of the GNU Lesser General Public           *
 *  License as published by the Free Software Foundation; either         *
 *  version 2.1 of the License, or any later version.                    *
 *                                                                       *
 *  See terms of license at gnu.org.                                     *
 *                                                                       *
 *************************************************************************/
package org.signserver.server.archive;
import java.security.cert.X509Certificate;
import org.signserver.common.ArchiveDataVO;
import org.signserver.common.RequestContext;
import org.signserver.common.WorkerConfig;
import org.signserver.server.SignServerContext;
import org.signserver.server.archive.Archivable;
import org.signserver.server.archive.ArchiveException;
import org.signserver.server.archive.Archiver;
import org.ejbca.util.Base64;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.Date;
import javax.persistence.EntityManager;
import org.apache.log4j.Logger;
import org.ejbca.util.CertTools;
import org.signserver.common.ArchiveData;
import org.signserver.server.archive.olddbarchiver.ArchiveDataArchivable;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.sql.DataSource;
import org.apache.log4j.Logger;
import org.ejbca.core.ejb.ServiceLocator;
import org.ejbca.core.ejb.ServiceLocatorException;
/**
 * Archiver only accepting responses and currently only supports Archivables of
 * class ArchiveDataArchivable. 
 * 
 * Developers:
 * This class could be improved to support any Archivable if the
 * DirectDatabaseArchiver should be able to be used with workers not returning
 * ArchiveData object any more.
 *
 * @author Diego de Felice
 */
public class DirectDatabaseArchiver implements Archiver {
    private static final Logger LOG = Logger.getLogger(DirectDatabaseArchiver.class);
    private boolean m_IsDisabled;
    private DataSource m_datasource = null;
    @Override
    public void init(int listIndex, WorkerConfig config, SignServerContext context) {
      try
      {
          LOG.info("Configuring Direct Archiver " + listIndex);
          m_IsDisabled = Boolean.parseBoolean(config.getProperty("DIRECTARCHIVER.ISDISABLED", "false"));
          String _ConnectionName = config.getProperty("DIRECTARCHIVER.CONNECTIONNAME");
          if (_ConnectionName == null) {
              LOG.info("DIRECTARCHIVER.CONNECTIONNAME property not configured");
              m_IsDisabled = true;
          } else {
              LOG.info("Using connection name: " + _ConnectionName);
          }
        
          if( ! m_IsDisabled )
          {
              String DS_Context = "java:" + _ConnectionName;
              Context initialContext = new InitialContext();
              if ( initialContext == null)
              {
                  LOG.info("JNDI problem. Cannot get InitialContext.");
                  throw new Exception( "JNDI problem. Cannot get InitialContext." );
              }
              m_datasource = (DataSource)initialContext.lookup(DS_Context);
              LOG.info("Configured Direct Archiver " + listIndex);
          }
      }
      catch(Exception ex)
      {
          LOG.info("Cannot archive data: " + ex);
          m_IsDisabled = true;
      }
    }
    @Override
    public boolean archive(Archivable archivable, RequestContext requestContext)
            throws ArchiveException {
        
        boolean archived = false;
        
        if ( ! m_IsDisabled && Archivable.TYPE_RESPONSE.equals(archivable.getType()) && archivable instanceof ArchiveDataArchivable) {
                
            final ArchiveDataArchivable ada = (ArchiveDataArchivable) archivable;
            final Integer workerId = (Integer) requestContext.get(RequestContext.WORKER_ID);
            final X509Certificate certificate = (X509Certificate) requestContext.get(RequestContext.CLIENT_CERTIFICATE);
            final String remoteIp = (String) requestContext.get(RequestContext.REMOTE_IP);
            String uniqueId = ArchiveDataVO.TYPE_RESPONSE + ";" + workerId + ";" + ada.getArchiveId();
            LOG.info("Creating archive data, uniqueId=" + uniqueId);
            
            PreparedStatement _statement = null;
            Connection _connection = null;
            try 
            {
                if (m_datasource != null)
                {
                    _connection = m_datasource.getConnection();
                }
                else
                {
                    LOG.info("Failed: datasource was null");
                    throw new Exception( "Failed: datasource was null" );
                }
            
                String _requestIssuerDn = null;
                String _requestSn = null;
                if (certificate != null) {
                  _requestIssuerDn = CertTools.getIssuerDN(certificate);
                  _requestSn = certificate.getSerialNumber().toString(16);
                }
                
                String _queryString = "INSERT INTO ARCHIVEDATA( UNIQUEID, ARCHIVEDATA, ARCHIVEID, REQUESTCERTSERIALNUMBER, REQUESTIP, REQUESTISSUERDN, SIGNERID, TIME, TYPE ) VALUES( ?, ?, ?, ?, ?, ?, ?, ?, ? )";
                _connection.setAutoCommit(false);
                
                _statement = _connection.prepareStatement( _queryString );
                _statement.setString( 1, uniqueId );
                _statement.setString( 2, new String( Base64.encode( ada.getContentEncoded() ) ) );
                _statement.setString( 3, ada.getArchiveId() );
                _statement.setString( 4, _requestSn );
                _statement.setString( 5, remoteIp );
                _statement.setString( 6, _requestIssuerDn );
                _statement.setInt( 7, workerId );
                _statement.setLong( 8, new Date().getTime() );
                _statement.setInt( 9, (int) ArchiveDataVO.TYPE_RESPONSE );
                
                int _nrows = _statement.executeUpdate();
                _connection.commit();
                
                if (_nrows == 0) {
                    LOG.info("Cannot archive");
                }
                
                LOG.info("Archived data, uniqueId=" + uniqueId);
                archived = true;
            }
            catch(Exception ex)
            {
                LOG.info("Cannot archive data: " + ex);
            }
            finally
            {
              try
              {
                  if (_statement != null) {
                      _statement.close();
                  }
                  
                  if( _connection != null ) {
                      _connection.setAutoCommit(true);
                      _connection.close();
                  }
              }
              catch(Exception ex)
              {
                  LOG.info("Cannot finalize: " + ex);
              }
            }
        }
        
        return archived;
    }
}

My contribution to SignServer archivers

Forums

Help

My contribution to SignServer archivers document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

My contribution to SignServer archivers