[Unicore-cvs-commits] SF.net SVN: unicore:[6665] gateway/trunk

[<bsc...@us...>]

Revision: 6665
          http://unicore.svn.sourceforge.net/unicore/?rev=6665&view=rev
Author:   bschuller
Date:     2010-03-22 08:36:47 +0000 (Mon, 22 Mar 2010)

Log Message:
-----------
switch documentation and changelog to asciidoc

Modified Paths:
--------------
    gateway/trunk/Changes.txt

Added Paths:
-----------
    gateway/trunk/docs/gateway-doc.txt

Removed Paths:
-------------
    gateway/trunk/docs/GW.doc
    gateway/trunk/docs/screenshot.png

Property Changed:
----------------
    gateway/trunk/docs/

Modified: gateway/trunk/Changes.txt
===================================================================
--- gateway/trunk/Changes.txt	2010-03-22 08:27:19 UTC (rev 6664)
+++ gateway/trunk/Changes.txt	2010-03-22 08:36:47 UTC (rev 6665)
@@ -1,8 +1,15 @@
-#
-# Change log for the UNICORE 6 Gateway
-#
+Change log for the UNICORE 6 Gateway
+====================================
 
+6.3.1
+-----
+
+ - fix "ping" to https VSites, avoiding the (harmless) "close notify" exceptions on the VSite
+ - convert documentation (and change log) to asciidoc format
+
 6.3.0
+-----
+
   - a new implementation of POST method processing, faster and less error prone, fixed bug 2547272
   - SOAP faults are now standard complaint.
   - Chunked HTTP dispatch is really configurable now (config option was ignored up to now, and 
@@ -23,10 +30,14 @@
  - bugfix: vsites can be removed from connections.properties now without requiring a gateway restart
  
 6.2.2
+-----
+
   - allow to configure HTTP protocol details ("connection: close" and
     "Expect: continue")
 
 6.2.1
+-----
+
   - forward VSite HTTP error code to the client when doing HTTP PUT 
   - log all connection attempts (client IPs) at debug level 
   - allow to disable site details on the web page ("monkey page")
@@ -34,6 +45,8 @@
   - minor code cleanup 
 
 6.2.0
+-----
+
  - improved logging. All loggers have prefix "unicore.gateway", and the logging file 
    config is periodically checked for changes. Exceptions are logged in a nicer fashion.
    The connecting client is logged at DEBUG level.   
@@ -45,17 +58,23 @@
  - update to Jetty 6.1.15
 
 6.1.3
+-----
+
  - experimental support for "gateway plugins", for tunneling other protocols through the 
    established SSL connection to the gateway 
  - update to httpclient 3.1 final
  - initial support for CRL checking (through new version of securityLibrary)
 
 6.1.2 p1
+--------
+
  - fix IllegalStateException
  - log everything to gateway.log (through log4j)
  - simplify Vsite resolution (now only checks the site name)
 
 6.1.2
+-----
+
 - package renaming to eu.unicore.gateway
 - AJP support
 - bind to specified interface only
@@ -63,6 +82,7 @@
 - send 404 if GET request cannot be resolved, or is not for the GW web page ("/")
 
 6.1.1
+-----
 
 - pluggable proxy validation
 - update Jetty to 6.1.8
@@ -70,39 +90,43 @@
 - add optional NIO connector (activate using "jetty.useNIO=true" in gateway.properties)
 
 6.1
+---
 
-- move to Jetty 6
-- remove code that is not strictly necessary
-- clean up dependencies (e.g. activesoap, commons-io, etc)
-- standardise directory structure and build procedures
-- registration of Vsites
-- connections.properties modifications take effect at runtime
-- use log4j logging framework
+ - move to Jetty 6
+ - remove code that is not strictly necessary
+ - clean up dependencies (e.g. activesoap, commons-io, etc)
+ - standardise directory structure and build procedures
+ - registration of Vsites
+ - connections.properties modifications take effect at runtime
+ - use log4j logging framework
 
 6.0.0 rc1:
+----------
 
--fix bugs in parsing security headers
--accept more credentials in security header
--SAML Consignor token is inserted as the 1st child of SOAP Header. For unauthenticated
-connections such assertion DOESN'T include Subject.
--old-style Unigrids security tokens (U/E/C) are now not mandatory and warning is
-issued when they are found at log level info.
--stop.sh and start.sh can be invoked from any directory
--there is possibility to configure if the consignor assertion should be signed 
-(and it's time thereshold). 
--add Maven build
+ - fix bugs in parsing security headers
+ - accept more credentials in security header
+ - SAML Consignor token is inserted as the 1st child of SOAP Header. For unauthenticated
+  connections such assertion DOESN'T include Subject.
+ - old-style Unigrids security tokens (U/E/C) are now not mandatory and warning is
+  issued when they are found at log level info.
+ - stop.sh and start.sh can be invoked from any directory
+ - there is possibility to configure if the consignor assertion should be signed 
+  (and it's time thereshold). 
+ - add Maven build
 
 
 6.0.0 beta1:
+------------
 
--use latest codebase
--fix concurrency problems
--add dynamic registration
--fix missing "xmls" dir in distribution which led to "500 internal server error"
+ - use latest codebase
+ - fix concurrency problems
+ - add dynamic registration
+ - fix missing "xmls" dir in distribution which led to "500 internal server error"
 
 6.0.0 alpha7:
+-------------
 
--change default config dir from etc/ to conf/
--replace hardcoded "/" by File.separator
--make logging properties configurable
+ - change default config dir from etc/ to conf/
+ - replace hardcoded "/" by File.separator
+ - make logging properties configurable
 


Property changes on: gateway/trunk/docs
___________________________________________________________________
Added: svn:ignore
   + gateway-doc.html


Deleted: gateway/trunk/docs/GW.doc
===================================================================
(Binary files differ)

Added: gateway/trunk/docs/gateway-doc.txt
===================================================================
--- gateway/trunk/docs/gateway-doc.txt	                        (rev 0)
+++ gateway/trunk/docs/gateway-doc.txt	2010-03-22 08:36:47 UTC (rev 6665)
@@ -0,0 +1,452 @@
+UNICORE Gateway
+===============
+
+This is a brief introduction to running and using the UNICORE 6 gateway.
+Please note also the following places for getting more information:
+
+UNICORE Website: http://www.unicore.eu
+
+Support list: uni...@li...
+
+Developer's list: uni...@li...
+
+Introduction
+------------
+
+The Gateway is the entry point into a UNICORE site. It is installed in
+front of any networking firewall. It authenticates incoming messages
+and forwards them to their intended destination. The following figure
+shows how a client request to the UNICORE virtual site (Vsite) “Alpha”
+is resolved and forwarded. The gateway receives the reply and sends it
+back to the client.
+
+In effect, traffic to the “virtual” URL, e.g.
+“https://mygateway:8088/Alpha” is forwarded to the real URL, e.g.
+“https://host1:7777”. In this way, only a single open port in a
+site's firewall has to be configured. (Technical note: this process
+only works for “most” HTTP requests (i.e. POST, GET and PUT), and is
+not a complete HTTP reverse proxy implementation).
+
+The mappings of virtual URL to real URL for the available sites are
+listed in a configuration file “connections.properties”.
+Additionally, the gateway supports dynamic registration of sites.
+
+The second functionality of the gateway is authentication of incoming
+requests. Connections to the gateway are made using
+client-authenticated SSL, so the gateway will check whether the caller
+presents a certificate issued by a trusted authority.
+
+Configuration
+-------------
+
+The gateway is configured using a set of configuration files, which
+usually reside in the /conf subdirectory.
+
+connections.properties
+~~~~~~~~~~~~~~~~~~~~~~
+This is a simple list connecting the names of sites and their physical addresses. 
+An example is:
+
+----
+DEMO-SITE = https://localhost:7777
+REGISTRY = https://localhost:7778
+----
+
+If this file is modified, the gateway will re-read it at runtime, so there is no need to 
+restart the gateway in order to add or remove sites.
+
+Additionally sites may register dynamically at runtime.
+
+gateway.properties
+~~~~~~~~~~~~~~~~~~
+
+Use the “hostname” property to configure the network interface and 
+port the gateway will listen on, and to select between https (recommended!) and http.
+
+----
+hostname = https://localhost:8080
+----
+
+NOTE: If you set the host to “0.0.0.0”, the gateway will listen on all network interfaces 
+of the host machine, else it will listen only on the specified one.
+
+If the scheme of the hostname URL is set to https, the Gateway uses the configuration 
+data from security.properties to configure the HTTPS listener. 
+
+security.properties
+~~~~~~~~~~~~~~~~~~~
+
+----
+keystore=/keystore/path
+keystorepassword=xxx
+truststore=/truststore/path
+truststorepassword=xxx
+----
+
+The gateway supports the usual JKS (.jks) and PKCS12 (.p12) 
+keystore / truststore types. These are determined automatically using 
+the file extension.
+
+As truststore types, you may also use “file” or “directory” by explicitely setting 
+the “truststoretype” parameter. In case of “file”, the “truststore” parameter is 
+assumed to denote a single file containing trusted certificates in PEM format. In case 
+of “directory”, the truststore setting is assumed to denote a directory. This directory 
+will be scanned for files in PEM format (having the file extension “.pem” or “.cert”), 
+which will be loaded as trusted certs. For example, to load all the pem files 
+in /etc/security/trusted, you can set
+
+----
+truststore=/etc/security/trusted
+truststoretype=directory
+----
+
+*CRL checking*
+
+The gateway supports CRL checking if the CRLs are available as local files or on the network.
+To enable CRL checking, you need to edit the gateway start.sh script and set a system property:
+
+----
+OPTS=$OPTS” -Dcrlmanager.properties.file=<path to properties file>”
+----
+
+Windows note: you can set system properties in the wrapper.conf file.
+
+The properties file contains the following settings:
+
+----
+#crl checking mode: STRICT, LAX or NONE
+crlcheck.mode = LAX
+
+#update interval in seconds
+crlcheck.interval = 86400
+
+# list of URLs to CRLs
+1.crl.url = http://ca.grid-support.ac.uk/pub/crl/ca-crl.crl
+2.crl.url = file:///path/to/local/CRLFile
+----
+
+In LAX mode, CRLs are checked but it is not an error if for a given certificate no CRL has 
+been loaded. In STRICT mode, a CRL must be checked for each certificate. Finally, NONE means 
+no CRL checking. The update interval is given in seconds. The CRL location is given as a URL. 
+Local files, HTTP servers, anonymous FTP etc can be used for CRL distribution. 
+
+NOTE: currently HTTPS connections will use the usual Java security, so it might be necessary 
+      to add the CRL server certificate to the Java truststore (i.e. the Gateway truststore 
+      is not used for this purpose).
+
+
+Dynamic registration of Vsites
+++++++++++++++++++++++++++++++
+
+Dynamic registration is controlled by three properties in gateway.properties
+
+----
+registration.enable=true
+----
+
+If set to true, the gateway will accept dynamic registrations which are made by 
+sending a HTTP POST request to the URL /VSITE_REGISTRATION_REQUEST
+
+Filters can be set to forbid access of certain hosts, or to require certain strings 
+in the Vsite addresses. For example
+
+----
+registration.deny=foo.org example.org
+----
+
+will deny registration if the remote hostname contains “foo.org” or “example.org”. 
+Conversely,
+
+----
+registration.allow=mydomain.org
+----
+
+will only accept registrations if the remote address contains “mydomain.org”.
+These two (deny and allow) can be combined.
+
+
+Disabling the detailed Vsite listing (monkey page)
+++++++++++++++++++++++++++++++++++++++++++++++++++
+
+To disable the Vsite details page, set 
+
+----
+#disable details display on the web page
+webpage.disable=true
+----
+
+Server parameters
++++++++++++++++++
+
+To fine-tune the operational parameters of the embedded Jetty server, you can set some properties. These are the defaults:
+
+----
+#minimum number of threads
+jetty.minThreads=1
+#maximum number of threads
+jetty,maxThreads=255
+#time after which an idle connection will be terminated
+jetty.maxIdleTime=3000
+#same, but under low resource conditions
+jetty.lowResourceMaxIdleTime=100
+#low resource threshold
+jetty.lowThreads=50
+----
+
+You can use the non-blocking IO connector offered by Jetty:
+
+NOTE: due to a known issue (see http://jira.codehaus.org/browse/JETTY-387) the 
+NIO connector has been reported to not work properly for large messages (e.g. 
+when transfering files using BFT). Therefore we currently discourage the use 
+of the NIO connector until this issue is resolved.
+
+To enable the use of the NIO connector, set
+
+----
+#use NIO connector
+jetty.useNIO=true
+----
+
+This will scale up to higher numbers of concurrent connections than the default connector.
+
+Controlling the number of concurrent connections
+++++++++++++++++++++++++++++++++++++++++++++++++
+
+The gateway acts as a https client for the VSites behind it. 
+The number of concurrent calls is limited, and controlled by two parameters
+
+----
+# maximum total number of concurrent calls to Vsites
+http.connection.maxTotal=100
+# total number of concurrent calls per site
+http.connection.maxPerService=20
+----
+
+Proxy certificate support
++++++++++++++++++++++++++
+
+UNICORE optionally supports proxy certificates as used by other Grid middleware systems. 
+In general, we think proxies are a bad idea, but for interoperability purposes, proxies 
+are supported as an extension. To enable the proxy certificate support,
+
+----
+#enable proxy authentication
+proxyAuthentication=true
+#optional validator class name (default: eu.unicore.proxy.RFC3820ProxyValidator)
+proxyValidator=<class name>
+-----
+
+You need to install the necessary extension libraries into the gateway lib directory. 
+The proxy extension is available separately from Sourceforge, if in doubt, ask on the 
+UNICORE support or development mailing lists.
+
+AJP Connector for using Apache httpd as a frontend
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you wish to use the Apache webserver (httpd) as a frontent for the gateway (e.g. for security or 
+fault-tolerance reasons), you can enable the AJP connector instead of the usual one.
+
+*Requirements*
+
+ - Apache httpd
+ - mod_jk for Apache httpd
+
+*Enabling AJP13 Gateway with Jetty and mod_jk*
+
+Enable "mod_ssl" module in httpd configuration files, for instance "ssl.conf":
+
+----
+ LoadModule ssl_module modules/mod_ssl.so
+ Listen 443
+----
+
+Enable "mod_jk" module in httpd configuration files, for instance "jk.conf":
+
+----
+ LoadModule jk_module modules/mod_jk.so 
+ JkWorkersFile "conf.d/worker.properties"
+----
+
+Define a "mod_jk" worker, to dialog with Gateway's AJP connector, in 
+"worker.properties" configuration file as referred in the above "jk.conf":
+
+----
+ worker.list=jetty
+ worker.jetty.port={{gateway_port}}
+ worker.jetty.host={{gateway_ip}}
+ worker.jetty.type=ajp13
+ worker.jetty.lbfactor=1
+----
+
+Configure a httpd virtual SSL host, dedicated to act as Gateway's frontend,
+in the httpd configuration files, for instance "unicore.conf". This virtual host
+lets the "mod_jk" worker defined above in "worker.properties", manage all the
+requests received and forwards the full client's certificate chain:
+
+----
+ # Apache VirtualHost configuration to serve as frontend
+ # for an "AJP connector enabled" UNICORE6 Gateway
+ <VirtualHost {{frontend_ip}}:{{frontend_port}}>
+     # Pass every request on this VirtualHost to the jetty worker
+     # defined in mod_jk's worker properties file.
+     JkMount /* jetty
+     # Pass SSL_CLIENT_CERT environment variable to the AJP connector
+     JkOptions +ForwardSSLCertChain
+ 
+     # Log
+     ErrorLog "logs/unicore_error_log"
+     TransferLog "logs/unicore_access_log"
+     JkLogFile "logs/unicore_mod_jk.log"
+ 
+     # Enable SSL
+     SSLEngine on
+ 
+     # Export SSL-related environment variables, especially SSL_CLIENT_CERT
+     # which contains client's PEM-encoded certificate
+     SSLOptions +StdEnvVars +ExportCertData
+ 
+     # Client have to present a valid certificate
+     SSLVerifyClient require
+ 
+     # Server certificate
+     SSLCertificateFile "{{/path/to/httpd_cert.pem}}"
+     SSLCertificateKeyFile "{{/path/to/httpd_key.pem}}"
+ 
+     # Trusted CAs
+     SSLCACertificateFile "{{/path/to/cacert.pem}}"
+ </VirtualHost>
+----
+
+On the Gateway, enable Jetty AJP connector instead of its HTTP connector in Gateway
+configuration file "gateway.properties":
+
+----
+ #enable AJP connector
+ jetty.ajp=true
+----
+
+*External references*
+
+ - Configuring AJP13 using mod_jk or mod_proxy_ajp - Jetty
+   http://docs.codehaus.org/display/JETTY/Configuring+AJP13+Using+mod_jk
+
+ - The Apache Tomcat Connector - Webserver HowTo
+   http://tomcat.apache.org/connectors-doc/webserver_howto/printer/apache.html
+
+Gateway plugins
+~~~~~~~~~~~~~~~
+The gateway supports tunneling other protocols through its socket. This is still experimental, so use at your own risk.
+To establish the tunnel, a special HTTP message is sent to the gateway:
+
+----
+HEAD / HTTP/1.1
+Upgrade: YOURPROTOCOLNAME
+----
+
+the method must be “HEAD”, and the message must contain the “Upgrade” header.
+The gateway replies:
+
+----
+HTTP/1.1 101 Switching protocols
+Upgrade: YOURPROTOCOLNAME
+----
+
+After this the gateway's socket connection is passed to a custom handler, 
+which can read data from the client and write replies. The handler can be 
+configured in gateway.properties:
+
+----
+protocolPlugin.YOURPROTOCOLNAME=your.handler.classname
+----
+The handler class must implement the eu.unicore.gateway.util.ProtocolPluginHandler 
+interface. For more details please refer to the sourcecode of the plugin interface class.
+
+logging.properties
+~~~~~~~~~~~~~~~~~~
+
+The Gateway uses Log4j for logging, configured using a properties file (conf/logging.properties).
+Edit this file if you need to change the granularity of the logging, or where the logs are written to.
+By default, the logs are written to files in the logs folder, and files are rolled over daily.
+Since the 6.2.0 release, you change the log levels at runtime by editing and saving the configuration 
+file (conf/logging.properties by default). The following logger names exist:
+
+[grid="all"]
+`-----------------------------`-------------------------------------------------------
+ unicore.gateway 	            General gateway logging
+ unicore.gateway.connections 	Log IPs of clients, and the DN after the SSL handshake
+ unicore.gateway.jetty		    HTTP processing, Jetty server
+ unicore.gateway.security	    Certificate details and other security
+----
+
+Here is an example logging.properties file
+
+----
+# Set root logger level to INFO and its only appender to A1.
+log4j.rootLogger=INFO, A1
+
+# A1 is set to be a file appender with date rollover
+log4j.appender.A1=org.apache.log4j.DailyRollingFileAppender
+log4j.appender.A1.File=logs/gateway.log
+
+# configure daily rollover: once per day the log will be copied
+# to a file named e.g. gateway.log.2008-12-24
+log4j.appender.A1.DatePattern='.'yyyy-MM-dd
+
+# A1 uses the PatternLayout
+log4j.appender.A1.layout=org.apache.log4j.PatternLayout
+log4j.appender.A1.layout.ConversionPattern=%d [%t] %-5p %c{1} %x - %m%n
+
+# log levels
+log4j.logger.unicore.gateway=INFO
+log4j.logger.unicore.gateway.connections=INFO
+log4j.logger.unicore.gateway.jetty=INFO
+log4j.logger.unicore.gateway.security=INFO
+
+# also configure JDK (java.util.logging) logging
+handlers=java.util.logging.ConsoleHandler
+# Default global logging level.
+# Loggers and Handlers may override this level
+.level=SEVERE
+----
+
+
+Web interface ("monkey page")
+-----------------------------
+
+For testing and simple monitoring purposes, the gateway displays a website 
+showing detailed site information (the details view can be disabled).
+Once the Gateway is running, open up a browser and navigate to 
+https://localhost:8080 (or whichever URL the gateway is running on). 
+As the gateway usually runs using SSL, you will need to import a suitable 
+client certificate into your web browser.
+
+
+A HTML form for testing the dynamic registration is available as well, 
+by clicking the link in the footer of the main gateway page.
+
+
+Building the Gateway from source
+--------------------------------
+To checkout the latest version of the Gateway source code, do
+
+----
+svn co http://unicore.svn.sourceforge.net/svnroot/unicore/gateway/trunk gateway
+----
+
+You will need to install Maven from http://maven.apache.org
+Compile using
+
+----
+mvn install
+----
+
+Compiles the code and runs the tests.
+
+----
+mvn assembly:assembly
+----
+
+builds distribution archives in zip and tar.gz format in the target/ directory
+
+
+


Property changes on: gateway/trunk/docs/gateway-doc.txt
___________________________________________________________________
Added: svn:mime-type
   + text/plain

Deleted: gateway/trunk/docs/screenshot.png
===================================================================
(Binary files differ)


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.