UNICORE Files

README file for the automatic testing tool SIMON and the associated
SIMON-Configurator
			SIMON 1.2.0
*************************************************************************
For questions and comments, please contact
a.giesler@fz-juelich.de (Andre Giesler)

(c) Forschungszentrum Juelich 2007,2008

**************************************************************************


 ####  INTRODUCTION

SIMON is an automatic testing tool for UNICORE, based on the roctopus
library developed within the project DEISA, workpackage JRA7. It
allows to specify tests and run them automatically in regular
intervalls against specified sites. In case of failure it is able to
send a test report in text format to the specified email-address.
This tool is highly configurable and can be easily extended to the sites
particular requirements.
SIMON requires a comprehensive configuration of its individual
configuration files to come up to user needs. Therefore the SIMON-Configurator,
a JAVA Swing based GUI, was developed to enable a simplified and fast
configuration of SIMON. The SIMON-Configurator is distributed as a part of the
SIMON package, but is designed as an independent application.


 ##### SUMMARY OF CHANGES and FIXED BUGS since SIMON 1.0.0

- Availability test and functionality tests can now performed independently
	from each other.
- Enabling automatic VSite-name detection in Configurator.
- Enabling a password prompt at SIMON start so storing Keystore password as
	clear text in the local file system can be avoided. The password will
	be held in memory as long as SIMON runs.
- Better management of keystores and CA certificates offered. Only one P12
	keystore can be
	specified in the Configurator. CA certs can be easily imported to the
	keystore directory.The validity of the CAs can be checked.
- Enabling cron-job-like, scheduled execution of SIMON (predefined time or
	date)
- Configurator closes after SIMON start if desired.
- No need to specify a path to UNICORE home directory anymore.
- Recently opened simonconf files will be inserted into file menu (max 5 file
	names).
- Local gateways.xml files can be used under Windows.
- Report directory under Windows is created at installation now.
- Time-out functionality for tests.
- Enabled distinction between e-mail receivers who will informed only in case of
	problems and others who'll get informed in any case.
- Added start-scripts for server use of SIMON.
- logging bug fixed

Since 1.1.1
- Fixed problem with not-terminated test jobs which can cause abnormal high data
	because of ongoing status requests.
- Avoid overflowing job outcome directory. Ouput files are deleted automatically
	if they are oldern than 30 days (will be modifiable in next version).

Since 1.2.0
- Report.txt file is written not before all tests are finished.
- Every test job is removed from NJS after status SUCCESSFUL,
	TIME_OUT_EXECUTING, NEVER_RUN, or PENDING.
- Fixed duplicate entries of unsuccessful tests in SIMON report.
- Time-out results generate AJO-logging messages in SIMON report (optional).
- Unsuccessful results generate a short problem description if AJO9 couldn't
	be submitted to UNICORE server (Authorization, Connection, etc).
- Lifetime in days of test-job output files (normally in ./data) can be
	specified. All files (*.out, *.err) will be deleted by SIMON after
	lifetime has expired.
-


 ##### TODOs, WORK-AROUNDS and possible EXTENSIONS
- Specification of user desired target homes for import test (e.g. special
	DEISA homes).
	Work-around: Create a AJO job by the UNICORE Client which loads the
	DEISA homes by a script and performs then the file import.
- Annoying UUDB errors in shell although tests were successfully performed.
	1. Work-around (UNIX): Edit simon_conf.sh and simon.sh. Comment out:
	#$JAVA -classpath $CP de.fzj.sitemonitorer.configtool.views.
	SimonConfigtool
	And comment in:
	$JAVA -classpath $CP de.fzj.sitemonitorer.configtool.views.
	SimonConfigtool $1 $2   2>/dev/null
	2. Work-around (UNIX and Windows): Add all a priori known non-active
	VSites to blacklist. They cause the error messages.

- Aborting SIMON process under Windows requires to kill Java process in task
	manager.
- Avoid overflowed data directory with job output.
- Enabling trusted delegation to avoid user certificates
- Better identification of the cause of unsuccessful tests.
- Integrating timeouts for faster handling of non-available sites.
- Control and kill tests/ jobs in the Configurator dynamically.
- Integrating latest Roctopus library

- Allowing more than one credential in the P12 keystore (monitoring sites
	with different certificates)
- Link SIMON with higher level monitoring tools (RMIS2, INCA)


 ##### MIGRATING

 - Unfortunately it's not possible to use your old simonconf file from the
	SIMON 1.0.0 version. In following SIMON versions it will be possible to
	keep your simonconf file.
 - For upgrading SIMON version from 1.1.x to 1.2.0:
 	1. Delete SIMON_HOME/lib/simon_1.1.x.jar
 	2. Download the latest SIMON archive from www.unicore.eu
 	3. Unzip archive to an arbitrary SIMON_TMP folder
 	4. Copy the following files from SIMON_TMP to your current SIMON_HOME:
 		a) SIMON_TMP/lib/simon.jar to SIMON_HOME/lib/
 		b) SIMON_TMP/lib/roctopus.jar to SIMON_HOME/lib/
 		c) SIMON_TMP/testresult.vm to SIMON_HOME/
 		d) SIMON_TMP/README_SIMON.txt to SIMON_HOME/
 	5. Restart SIMON


 ##### PREREQUISITES

-Java 1.5.x
-Unix-style or Windows operating system
-certificate/keystore in PKCS12 format and corresponding Gateway CA
	certificates (NOTE: the CA certificates in PEM format are expected
	in the SAME directory as the p12 file!)


 ##### INSTALLATION

   extract the files:

        tar xzf simon_VERSION.tar
        cd simon_VERSION


 ##### CONFIGURATION

There are two configuration options. Edit SIMON configuration files manually
or use the SIMON-Configurator tool (recommended).


- Edit SIMON configuration files by using the SIMON-Configurator (recommended)


1) Start Configurator by calling simon_conf.sh (UNIX) or simon_conf.bat
	(Windows)

2) Use the tabs in the GUI to edit SIMON settings

	a) Common tab

		- If you want to configure the SIMON installation which is
		installed in the same directory as the Configurator, you don't
		need to edit anything in the SIMON home panel. But if you want
		to configure a different SIMON installation on your host, you
		have to set the external SIMON home. When you have configured
		SIMON at a different installation and want to start it from
		the 'Start SIMON' button in the Configurator SIMON will be
		started as a new Java process.

		- Specify a run frequency for executing the SIMON tests. Choose
		between:
		a) Only once: SIMON will be executed only once. This may be
		appropriate when you want to test your configuration before
		starting a continuous monitoring.
		b) Every X hours: You define a period in hours (Integer value)
		when SIMON  will be executed periodically.
		c) Schedule: Use this option when you want a high flexibility
		in configuring the run frequency. The Schedule option is like
		a UNIX cron job. You can define a fixed time at	every day, day
		of week, day of month, or month. Just as well you can configure
		a start	time with an incremental execution every arbitrary hour
		then. Examples:
		minute:15  hour:4/2  day of month:No  month:All
			day of week:MON-FRI
		This adjustment fires a SIMON start at 04:15am and then every 2
		hours till 23:59:59 p.m. and only on weekdays.

	b) Tests tab

		- If 'VSite availability test' is activated a test will be
		performed detecting generally which UNICORE VSites will be
		available on which USites. Additionally the number of running
		AJO jobs on these VSites is determined. This test will be
		performed in every SIMON execution because it's a coproduct of
		the funtionality tests. If you deactivate the test, it only
		won't be added to the SIMON-report. Of course you can run
		SIMON with the 'VSite availability test' alone.
		- If 'AJO test' is activated you have to specify a directory
		on your host where the AJOs will be taken for your site
		functionality tests. If the specified directory already
		contains AJOs, they'll be listed in the AJO testing table and
		will be marked as selected. Unselected AJOs means that
		they are stored in the internal SIMON-tmp directory. When they
		are selected they will be copied to the AJO directory and vice
		versa for later reusing. It should be noted that every
		selected AJO will be ececuted against the destination VSites
		which will be defined later. - Add external AJOs to the table
		if you want. They'll be automatically copied to your specified
		AJO directory. Some AJOs are provided with the SIMON release.
		The simpliest way to create your own AJOs is the UNICORE
		client (http://www.unicore.org/).
		- No longer needed AJOs might be removed again. They'll be
		deleted from the AJO directory or from the tmp directory
		(if unselected).
		- Activate the 'Import test' if desired and get a file to
		import.
		- Specify a time period in seconds when tests should be timed
		out by SIMON. 60 - 120 seconds are recommended.
		- Activate checkbox if you want a detailed description of jobs
		which were aborted after time-out period exceeded.

	c) Sites tab

		- Specify the gateways.xml file which contains the Usites and
		gateway URLs of the UNICORE sites. You have two options to
		appoint a gateways.xml file.
		a) Use the browser button on the right of the text field to
		specify a XML file on your local file system. The local file
		path will be saved later internally with a file praefix
		(file:///... for windows or file:// for linux). You don't need
		to add this praefix.
		b) Insert an URI to a http file, e.g.
		http://www.fz-juelich.de/unicore/unicoreSites_DEISA.xml and
		press Enter to validate your input.

		- Add the needed P12 keystore with your certificate and private
		key by the add button on the right of the keystore text field.
		Input directly by the text field won't be accepted. The keystore
		has to comprise only one cert and key. After clicking on the add
		button the keystore dialog pops up. Use the browser button to
		choose the desired keystore. Dependent on the
		keystore-password-setting the password field for unlocking the
		keystore is activated or not. It's enabled if the modus is
		'Stored in file system'. Then you have to input also the
		password. Otherwise you will be prompted later for the
		keystore password when starting SIMON. In the first case the
		password is immediately checked. If something went wrong you'll
		be informed by a popup dialog.

		-If you specified a valid keystore you should have got the
		USites from the gateways.xml file which were listed in the
		'USite	entries list'. There you have the option to add USites
		to the	blacklist, i.e. no availability test will be performed
		on that	sites. This is useful if you know a priori that a USite
		is not	available or you don't want to check it generally. The
		advantage is to save time because SIMON doesn't try to
		contact a dead site for instance.

		- Add Vsites to the Vsite name list which you want to test. You
		have two options to add VSite names:
		a) Use the text field above the VSite name table and
		specify a VSite (e.g. FZJ JUMP). When you click the
		add Button the entry will be added to the table.
		b) The second and more convenient option is to get the VSites
		automatically. Use the 'Get' button on the right of the table to
		do that. Every USite (also blacklisted sites) will be asked for
		available VSites names if you enabled also the 'All sites'
		checkbox below. Otherwise only the first USite entry tag in the
		gateways.xml file will be used (the tooltip of the checkbox
		shows which). If there is no answer from Usites gateway after 20
		seconds a timeout is performed and the next gateway is used.

		- After defining the gateways file, the keystore and the Vsite
		names you're able to add rows to the testing table and to
		configure the sites you want to test. Do that by clicking on the
		add button on the right of the 'Configuring VSites table'. Click
		on the cells to pop up the combo boxes if more than one Usite or
		Vsite (which is the normal case) has been defined. Select a row
		by the 'Select' column to constitute that this Vsite should be
		tested over the defined Usite/gateway by the tests specified in
		the test tab. Deselected sites won't be tested. The 'USite' and
		'Gateway' columns are associated together.

		- Change the lifetime of test-job output in the specified outcome
		directory. Default value are 10 days.

	d) Reports tab

		- Edit the settings for email reporting. By default emails will
		only be sent in case of unsuccessful UNICORE site tests (AJO
		and Import test). Disable the checkbox to change that behaviour.
		If you disable that checkbox you can nevertheless determine some
		receivers in the appropriate list (below the cc list) who will
		beyond informed only in case of problems.
		- Edit the outcome directory for the site availability test
		report.

	e) Logging tab

		- Edit the logging properties for SIMON if you want. Use the
		handler levels to get a finer logging. The console handler will
		only work in case that SIMON and Configurator are using the same
		installation directory. - Edit the logging properties for the
		Configurator if you want. Ordinarily this is not necessary. You
		could use the Apply button for dynamic changes in the logging.


3) 	Settings menu
	- You are able to configure some global settings:
	a) Activate the expert modus in Settings -> Expert modus to be able to
	edit SIMON settings, which usually aren't needed to be changed
	(recommended settings like internal paths etc.).
	b) Activate 'Load as default' to load your current simonconf file by
	default when the configurator is started. But you have to perform
	'Configure SIMON' before the 'Load as default' setting can be saved.
	c) Activate 'Close at SIMON start' to close the Configurator frame when
	starting SIMON.
	d) The keystore password modus is set to 'prompt' by default which means
	that the user is prompted for the password when starting SIMON. The
	Prompt is a small dialog in Configurator when starting SIMON from there
	or a shell prompt when starting it directly from the console. For
	testing or using SIMON on a private machine it may be convenient to
	store the password in the local file system in clear text which the
	appropriate second option enables.

4) 	Saving configuration Save your settings by using the save or saveAs
	commands in the file menu or the appropriate buttons in the toolbar. The
	settings will be saved to a 'simonconf' file on your host. You can later
	reuse the settings in that file by loading them from there.

5) 	Configuring SIMON
	Use 'Configure SIMON' from the file menu or the appropiate button in the
	toolbar to write your settings from the saved or loaded 'simonconf' file
	to the config files in your SIMON installation. Keep in mind that the
	current settings in the Configurator will be saved to the current loaded
	simonconf file before the settings are written from the simonconf file
	to the SIMON config files. SIMON is now ready to use.

6) 	Starting SIMON
	a) Start SIMON either by the 'Start SIMON' button in the toolbar or use the
	simon.sh (UNIX) and the simon.bat (Windows) scripts respectively. If you
	choose the button you will be asked about the keystore password if you
	have set the prompt password modus.
	b) If you want to deploy SIMON on a UNIX server machine in the background you
	can	use the provided start-scripts silent_prompt.sh or silent_stored.sh. The
	first is a script which prompts you for the keystore password and starts then
	SIMON in the background. Use the latter if you have already stored the
	password in the SIMON configuration files.
7) 	Restarting Configurator
	If you restart the Configurator (after you have saved a configuration
	before) the name of the simonconf file appears in the file menu. Click on
	the name to load the settings conveniently.



- Edit SIMON configuration files manually (not recommended)


1) cd ./etc

2) check and edit init.config if necessary

3) edit testsuite.config if necessary

4) edit email.config

5) edit logging.properties if necessary

6) cd test_conf/roctopus_conf

7) edit roctopus.config in order to adjust the path to the roctopus scv-file
   and its properties file

8) cd ../ajo_conf

9) edit ajos.config in order to specify a directory containing serialized ajos
   or JobContainer-objects to be submitted
   ATTENTION: if you intend to submit ajos generated by the GUI Unicore Client,
   you need to edit the cli_userdefaults.txt-file!

10) For unix-style operating system call simon.sh, for Windows simon.bat



**************************************************************************