[Ubermq-commits] jms/doc ubermq.pdf,NONE,1.1 ubermq.sgm,NONE,1.1 ubermq-1.0.sgm,1.2,NONE
Brought to you by:
jimmyp
From: <ji...@us...> - 2002-12-09 21:26:21
|
Update of /cvsroot/ubermq/jms/doc In directory sc8-pr-cvs1:/tmp/cvs-serv10910 Added Files: ubermq.pdf ubermq.sgm Removed Files: ubermq-1.0.sgm Log Message: updated documentation --- NEW FILE: ubermq.pdf --- %PDF-1.3 1 0 obj << /S /GoTo /D [2 0 R /Fit ] >> endobj 4 0 obj << /Length 172 /Filter /FlateDecode >> stream xÚO=Â@ÝïWd³.^rß« ÁA¸Nâ¢â JKÿ¿×VpÐA2ä ÷ÁÊCà ½Î#²³p¾×ÌTÞ ã²åRBò ³_=JVI,7¬ úé2)Ss(êSÛíö¥d BAÝ·Ý¢tªèç»nM[ÓvÊ Lv -[Þ`î+¯êJVÅcxV±NÁ¢ ÒYB£þÿðu#=Õendstream endobj 2 0 obj << /Type /Page /Contents 4 0 R /Resources 3 0 R /MediaBox [0 0 609.714 789.041] [...1658 lines suppressed...] 0000045094 00000 n 0000045155 00000 n 0000045216 00000 n 0000045278 00000 n 0000045446 00000 n 0000048215 00000 n 0000048292 00000 n 0000051145 00000 n 0000051184 00000 n 0000051222 00000 n 0000051458 00000 n trailer << /Size 319 /Root 317 0 R /Info 318 0 R >> startxref 51658 %%EOF --- NEW FILE: ubermq.sgm --- <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V3.1//EN"> <book> <bookinfo> <title>UberMQ 1.2 User's Guide</title> <corpauthor>UberMQ Group</corpauthor> <date>December 9, 2002</date> <copyright><year>2002</year> <holder>UberMQ Group, part of Rhombus Technologies, LLC</holder> </copyright> </bookinfo> <preface><title>Foreword</title> <para> UberMQ is a clean-room implementation of the JMS pub/sub API. It has been designed for speed, scalability, and customization. This guide is intended to help you use and discover the features of our messaging software. </para> <para> This book is divided into two sections: the user's guide, which will guide you through the setup and configuration of UberMQ as a JMS server, and the developer's guide, which will elucidate the various interfaces that you may use to customize the operation of the infrastructure. </para> <para> UberMQ was written and is supported by tireless volunteers and messaging professionals. We are committed to your success with UberMQ, and provide a comprehensive suite of consulting services to help you with the product. Whether it's a one-day engagement for setup, configuration, and to get you familiar with UberMQ's features, or a long-term engagement to provide new features or extensions for your messaging needs, we are here to help. Please visit our website at <ulink url="http://www.ubermq.com"> <citetitle>www.ubermq.com</citetitle></ulink>, or email us at <email>ser...@ub...</email>. </para> <para> Thanks for evaluating UberMQ and for your support of open-source software! We'd love to hear about the projects you are attempting with our platform. Good luck! </para> </preface> <part label="I"> <title>User's Guide</title> <partintro> <para> The User's Guide is intended to help you learn about the features of UberMQ, set up the software, and manage configuration settings. </para> </partintro> <chapter><title>UberMQ Features</title> <para> UberMQ is a fully functional, feature-complete implementation of the <ulink url="http://java.sun.com/jms/">JMS 1.0.2b specification from Sun</ulink>. Some of the highlights of the UberMQ feature set are: <itemizedlist mark=opencircle> <listitem> <para> <emphasis>Java NIO</emphasis>. UberMQ is entirely implemented using JDK 1.4's New I/O (NIO) primitives, for unprecedented speed and scalability. A single thread can now service thousands of simultaneously connected messaging nodes. </para> </listitem> <listitem> <para> <emphasis>Clustering</emphasis>. Connect multiple UberMQ servers in a cluster for added fault-tolerance and reliability. UberMQ automatically removes duplicates and re-orders messages into a single coherent message stream. </para> </listitem> <listitem> <para> <emphasis>Customizable Overflow Management</emphasis>. Your infrastructure will no longer be compromised by a single unresponsive subscriber, or a publisher who is sending information too quickly. UberMQ's pluggable flow control mechanism guarantees liveness for your infrastructure and automatically disconnects unresponsive nodes. If you don't like the default bevahior, you can also create your own overflow logic and plug it in by changing the server configuration file. </para> </listitem> <listitem> <para> <emphasis>Durable Subscriptions</emphasis>. A fast, robust implementation that does not require a database. Get the benefits of durable subscriptions without paying the performance penalty associated with most JMS providers. </para> </listitem> <listitem> <para> <emphasis>Hierarchical Topic Space</emphasis>. Topics are laid out in a tree in UberMQ. You can subscribe to single leaves, or entire subtrees. Alternatively, you can specify a topic's subscription profile using a regular expression. </para> </listitem> <listitem> <para> <emphasis>Independent Transport Layer</emphasis>. UberMQ ships with TCP/IP, LRMP (a reliable UDP variant) and intra-process pipe connectivity. However, UberMQ is designed so you can write your own transport layer code and plug it into the infrastructure. You can also customize the wire format of datagrams used by UberMQ so you can achieve full interoperability with legacy or third-party systems. </para> </listitem> </itemizedlist> </para> <para> All of these features are in addition to the basic JMS pub/sub functionality that you are accustomed to. The result is a fast, robust, scalable messaging solution that can be used in a variety of distributed enterprise scenarios, or even inside a standalone application. </para> </chapter> <chapter><title>Setup</title> <section> <title>Directory Structure</title> <para> UberMQ comes pre-configured and ready for immediate use. Once you have downloaded the software and unpacked the archive, you will see the following directory structure: <programlisting> bin/ Shell scripts lib/ Third-party Java libraries dist/ The UberMQ JAR doc/ Documentation javadoc/ Javadoc API Documentation README A readme to get you started LICENSE The UberMQ license (LGPL) </programlisting> </para> </section> <section> <title>Running the Software</title> <para> You do not need a full development environment to run the UberMQ server. Simply invoke <userinput>bin/server.sh [properties-file]</userinput> or <userinput>bin/server.bat [properties-file]</userinput> to get started. You should see the line "UberMQ started." </para> <para> The properties file given in the first argument is used to customize how UberMQ runs. This includes clustering information, port information, and some performance tuning settings that you may wish to modify for your environment. If you leave this value unspecified, the default is the <userinput>server.properties</userinput> file in the current working directory. A default properties file has been provided with some reasonable values. </para> <para> At this point, you may be wondering, what next? Now it's time to throw your JMS clients at the server. Simply direct your JMS code to instantiate a <userinput>com.ubermq.jms.client.UnicastTopicConnectionFactory</userinput> and provide the hostname and port of the computer that is running the UberMQ software. Your clients should immediately begin to function normally. </para> <para> If you are just getting started with JMS, a good place to learn more about the specification and to see example code is Sun's <ulink url="http://java.sun.com/jms/"><citetitle>Java Message Service</citetitle></ulink> website. </para> </section> <section> <title>Administrative Software</title> <para> One of the most compelling features of UberMQ are the two administrative interfaces that allow you to examine your running message servers and inspect message traffic that is going through your infrastructure. </para> <para> The administrative interface can be invoked using the <userinput>bin/admin.sh</userinput> or <userinput>bin/admin.bat</userinput> script. This application displays active connections, server status, connection tables, and allows you to enable, disable and close any connected user. </para> <para> The server statistics require a brief comment: the server displays its uptime, as well as the number of messages received, sent and dropped by the server. The message in, out and drop statistics will not match up in most applications, because each incoming message can produce zero or more outgoing messages. It also displays average latency per message and an implied throughput and utilization metric based on the latency. The implied throughput is the reciprocal of the latency (which is measured as the time that a message is being processed by the server), and a utilization, which is the fraction of total server uptime that was consumed by message delivery. A utilization of 100% implies that your server is sending the maximum rate of messages, equivalent to the implied throughput. </para> <para> The message viewer application can be invoked using <userinput>bin/viewer.sh</userinput> or <userinput>bin/viewer.bat</userinput>. This application automatically receives every message that passes through the infrastructure, and displays it in a tree according to the hierarchical topic that the message was published on. </section> <section> <title>General Configuration</title> <para> To configure your UberMQ server, you should start with the <userinput>server.properties</userinput> file. It is the main place to change settings. Some of the more useful settings will be described here, and the appendix will contain a full listing of all settings recognized by UberMQ. </para> <para> The standard UberMQ server is a typical TCP socket-based server, which listens on a port for incoming connections. The default port for UberMQ is 3999. You can change this default port to any value by changing the <userinput>server.port</userinput> value in the properties file. </para> <para> The server also has its own log file that is used to store information between server invocations. This is used to store information about durable subscriptions, among other potential uses. You should make sure that the server is using the same log file when it starts up, and that no other server processes are trying to access the same file simultaneously. The name of this file is given by the <userinput>server.logfile</userinput> property, and can be set to any location. If the file does not exist, it will be created when necessary. </para> <para> Another group of settings that can be very important is the durable subscription management settings. Every durable subscription registered by clients is recorded in the server's log file. In addition, each subscription is allocated a logfile, which records messages and acknowledgements as they are sent to and from the subscriber. If you are using durable subscriptions, you will want to set the following settings to appropriate values. </para> <variablelist><title>Durable Subscription Settings</title> <varlistentry><term><userinput>server.durable.logpath</userinput></term> <listitem> <para> The directory in which the server stores durable subscription logfiles. The default value is <filename>/var/durable</filename>. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.durable.logprefix</userinput></term> <listitem> <para> Each durable subscription has a unique name associated with it, provided by the JMS client application. This property allows you to specify a prefix for the file that is created in the log directory, to differentiate between other message servers that may also be using the same directory for their durable subscribers. For instance, if this property is <filename>serverA-</filename>, your subscribers will create logfiles named <filename>serverA-subscriber1</filename> or <filename>serverA-subscriber2</filename>. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.durable.logsize</userinput></term> <listitem> <para> The size of each durable logfile, in bytes. UberMQ durable subscribers are currently implemented such that they automatically create a file of this exact size. They do not attempt to grow the file, to prevent fragmentation. This number serves as the maximum amount of data that is saved for a durable when it is offline. Messages that need to be delivered to a subscriber with no room left in its logfile will be handled by the overflow manager. </para> </listitem> </varlistentry> </variablelist> </section> <section> <title>Clustering</title> <para> UberMQ features the ability to arrange your message servers in a cluster. This provides your messaging infrastructure with another level of fault-tolerance and reliability. When an UberMQ server is in a cluster, it will send multiple copies of messages around to other members of the cluster, for ultimate delivery to any subscriber anywhere in the cluster. </para> <para> Clustering provides a convenient administrative metaphor to provide distributed points of failure for your messaging nodes. Your client code does not need to change; it can subscribe to messages normally and receive messages as if the publisher and subscriber were connected to a single server instance. Furthermore, UberMQ's sequencing manager will automatically reconstitute messages from various parts of the cluster into a single ordered stream, and automatically remove duplicates if they appear. </para> <para> Setting up clustering is easy. Simply create a graph of forwarding paths through the cluster, and specify those connections in the server properties file. Remember that the connections you specify are directed connections, and bidirectional connections must be specified by both endpoints. Also take care not to create cycles in your clustering graph. For example, consider a three-node cluster, consisting of nodes A, B and C. Connect the nodes in the following way: </para> <programlisting> A <-> B <-> C </programlisting> <para> The server properties relating to clustering are described below. </para> <variablelist><title>Clustering Settings</title> <varlistentry><term><userinput>clustering.forward</userinput></term> <listitem> <para> The URLs of the UberMQ message servers to forward messages to. This is a comma-delimited value. In the above example, the forwarding list for node B would be "A,C". </para> </listitem> </varlistentry> <varlistentry><term><userinput>clustering.enable</userinput></term> <listitem> <para> Set this to true if clustering is enabled. If this is false, all other clustering properties will be ignored. </para> </listitem> </varlistentry> </variablelist> </section> </chapter> <chapter><title>Usage Notes</title> <itemizedlist mark=opencircle> <listitem> <para> <emphasis>Durable Subscription Names</emphasis>. Names of durable subscriptions must be unique for a given server instance. The behavior if two subscribers attempt to connect to the same server with the same durable name is undefined. </para> </listitem> <listitem> <para> <emphasis>Message Selectors</emphasis>. Message selectors are executed on the server-side. That means that bandwidth will not be utilized for messages that are not relevant for a particular subscriber. However, only a subset of message selector functionality is included in UberMQ 1.0. Only clauses of the form "where [property] [>,<,!=,<>,=] [value]" are supported. The more complex logical expressions specified in JMS 1.0.2b are coming in a future version. </para> <para> It is possible to override the default selector implementation by implementing the com.ubermq.jms.server.routing.Selector interface, if you require more complete functionality. </para> </listitem> <listitem> <para> <emphasis>Non-durable messaging and recover()</emphasis>. Recovering unacknowledged messages from a non-durable session is not implemented in this version of UberMQ. </para> </listitem> <listitem> <para> <emphasis>Transacted Sessions</emphasis>. Transacted sessions are not implemented in this version of UberMQ. Any rollback attempts will fail, however, commit operations will silently succeed, in order to preserve existing JMS code. </para> </listitem> </itemizedlist> </chapter> </part> <part label="II"> <title>Developer's Guide</title> <partintro> <para> The Developer's Guide contains information about the design and implementation of UberMQ from a technical perspective, and describes how you can use your own custom components to change the behavior of the infrastructure. </para> </partintro> <chapter><title>UberMQ Design</title> <para> UberMQ is a hub-and-spoke implementation of a JMS message server. Currently, only pub/sub messaging is supported, because it is more generally useful than queue-based messaging. Furthermore, queue-based messaging is a special case of a correctly implemented durable subscription. </para> <para> UberMQ features an open design, and actively encourages re-implementation of key interfaces to provide additional or modified functionality. This comes into play in the areas of overflow management, transport layers, and datagram wire formats. </para> <para> The server is not a heavy-weight server in the traditional sense. It is a single hub through which many clients connect and interoperate through, but the design is flexible enough that this hub can serve a single JVM via Pipe channels, or many distributed clients over TCP/IP. The hub can be embedded in the same process as its client applications for higher speed and easier maintenance. The server starts very quickly, and only requires two threads for normal operation. </para> <para> At the end of the day, a message server has to be good at performing I/O operations, and has to be able to scale. Thanks to Java NIO, UberMQ shines in this area. Message delivery latency is very low, especially when messaging in done in the same process. Similarly, throughput is outstanding. Garbage creation is kept to a minimum so that collection does not consume valuable CPU resources. </para> <para> You will find that UberMQ will scale to meet your heaviest I/O demands, and provide you with a lightweight package so you can deploy a server anywhere. </para> </chapter> <chapter><title>Overflow Management</title> <para> One of the critical aspects of a high-quality enterprise messaging system is how it responds to overflow conditions. These conditions occur when a publisher is publishing more data than a subscriber can reasonably process. The underlying cause of this is typically a catatonic subscriber that is no longer normally processing messages. <caution><title>Don't over-publish!</title> <para> An application should not be designed in such a way that a publisher can normally produce more information than a subscriber can realistically consume. </para> </caution> <para> UberMQ has buffering and queuing mechanisms built in to avoid transient overflow conditions, but needs to step in more drastically when mismatched publish and consume rates continue for too long. JMS provides some conceptual assistance here, in the form of persistent versus non-persistent message delivery modes, and message time-to-live (TTL). </para> <para> Message delivery modes allow a publisher to flag its messages as not being critical. Non-persistent messages favor speed over guaranteed delivery, and as such, when UberMQ runs into a buffer overflow, non-persistent messages are automatically dropped so it can return to normal operation. Persistent messages, however, must be delivered by definition, and such a course of action is not permissible. </para> <para> In these cases, UberMQ employs an exponential backoff algorithm, as is used by the Ethernet, to temporarily reduce the rate of publication and to allow subscribers to catch up. The exponential backoff typically works very well to resume normal operation. This algorithm is combined with a message TTL, and adds the optimization that messages may be dropped after a certain period of time has elapsed. </para> <para> In no case will UberMQ compromise the integrity of the infrastructure if a single client has become unresponsive. UberMQ has a few more drastic options that it can use when exponential backoff repeatedly fails to return the infrastructure to normal operation, namely disconnection. The default UberMQ behavior is to disconnect the subscriber that is overflowing and remove it from all future message broadcasts. This decision reflects the attitude that the infrastructure is more important than the whole. </para> <para> You can also easily customize the overflow management used by the server by creating a class that implements <userinput>com.ubermq.kernel.IOverflowHandler</userinput> or <userinput>com.ubermq.kernel.IConnectionOverflowHandler</userinput>. The JavaDoc for these interfaces describes the contract, and you can look at the existing implementations for ideas and examples. Direct the server to your implementation class by setting the <userinput>server.dgram.overflowhandler.class</userinput> property. The server will instantiate your object using the default no-arg constructor. </para> </chapter> <chapter><title>Datagrams</title> <para> The fundamental quantum of information in UberMQ is the datagram. A base datagram has only a few fundamental operations and mostly provides no functionality, except as a base to define your own datagram groups. UberMQ's JMS server defines three types of datagrams: message, control, and acknowledgement. </para> <para> The message datagram is the core of UberMQ. It encapsulates a JMS message, complete with headers, properties and body. The IMessageDatagram interface specifies a standard way for the JMS client code to access these raw properties from a datagram. </para> <para> The control datagram, specified by IControlDatagram, encapsulates command sequences that are sent to a server. These are commands like subscribe, unsubscribe, start, stop, etc. The actual command interfaces are derived from the ICommandSubGram interface. </para> <para> Finally, the acknowledgement datagram represents an acknowledgement for a message or command, either positive or negative. It is very compact, and has very limited functionality, by design. </para> <para> Datagram instances are created by using factories. By replacing the factory implementations that UberMQ uses, you can create your own datagram implementations with customized wire formats, or other unique behavior. The main IDatagramFactory is used to transform byte streams into datagrams, and vice versa. It is possible to replace only this class to give your UberMQ messages a customized header, or serialize them into an alternate transport layer. </para> <para> There are three other datagram factories, which are used when client code needs to construct datagrams from scratch. This is used frequently when publishing messages, or when clients create control messages to send to the server. The IMessageDatagramFactory creates message datagrams, IAckDatagramFactory creates acknowledgements, and the IControlDatagramFactory creates commands. These can be independently replaced as necessary. </para> <caution><title>Be consistent!</title> <para> It is important to remember that both producer and consumer must be using the same datagram factories, otherwise they will not be able to communicate, and the results will be undefined. The default UberMQ server uses the standard implementations of each factory, so if your clients use custom factories, you will need to create a custom server instance that also uses your factories. </para> </caution> </chapter> <chapter><title>Connection Factories</title> <para> Connection factories are the main entry point for JMS clients to your infrastructure. UberMQ ships with several default connection factories, and each one specifies various aspects of the connection that can be customized. </para> <para> Conceptually, a connection is an entry point into a JMS infrastructure. It need not be a physical connection, but in the case of TCP or pipes, it maps nicely. In the case of multicast, a connection is really a metaphor for a broadcast socket bound to a multicast address. </para> <para> By creating your own connection factories, you not only use custom code to connect or bind to a transport layer, but you can also take control of the way datagrams are created. This gives you the power to change UberMQ's wire protocol as you see fit, for maximum interoperability. </para> <para> Creating a connection factory is a fairly straightforward process. You need to create three classes: one implementing TopicConnectionFactory, your connection I/O class, which implements com.ubermq.kernel.IConnectionInfo, and a class implementing IClientSession, which serves to instantiate the above. Most of the time, your connection class will need to deal with handling actual I/O. To ease the burden of implementing the interface, you can extend com.ubermq.kernel.AbstractConnectionInfo, which provides you with separate read and write buffers, mutex locking, and code to interface with a datagram factory and message processor. </para> <para> If you choose not to extend AbstractConnectionInfo, you will need to make sure that your connection correctly frames and creates datagrams, and forwards those datagrams to an IMessageProcessor instance. </para> <para> Your factory instance will typically create an instance of com.ubermq.jms.client.TopicConnection, and specify all of the relevant factories and session implementations for it to be able to do the work of providing JMS functionality to clients. One of the objects you must specify is an instance of IClientSession. In order to create an underlying I/O connection, the JMS TopicConnection will call the connect method of your session object. From this method, you return your IConnectionInfo implementation, and register your I/O connection with the relevant thread(s) to perform the actual processing. </para> </chapter> <chapter><title>Embedded UberMQ servers</title> <para> Since UberMQ is so light-weight, it is frequently useful to embed a running server inside your application with other messaging components for speed and versatility. This is easy to accomplish. Simply instantiate a MessageServer object, and call the start() method. </para> <para> In order to fully take advantage of in-process servers, your JMS clients can use the PipeConnectionFactory to connect directly to the server object itself. Simply instantiate a factory and pass in the server object as the only argument. The rest of your JMS code is unchanged! </para> </chapter> </part> <appendix><title>UberMQ Server Properties</title> <variablelist><title>All Settings</title> <varlistentry><term><userinput>general.connection.buffer</userinput></term> <listitem> <para> The size of the read and write buffers that each connection are allocated. Note that the read and write buffers are independent. This defaults to 1MB each. This constrains the maximum datagram size to be this value as well. You may see a drastic improvement in memory usage if you decrease this value. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.datagramfactory</userinput></term> <listitem> <para> The fully qualified class name of a class implementing IDatagramFactory that will be used by the server. The factory must have a public no-arg constructor, or a static getInstance() method for instantiation. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.rwthreads</userinput></term> <listitem> <para> The number of reader threads to employ to handle channel I/O. This should typically be equivalent to the number of processors on the machine. In no cases should this exceed the number of processors - this will drastically impede performance. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.admin.enable</userinput></term> <listitem> <para> Whether the RMI-based administration interface is enabled. If the administrative interface is enabled, the server will display the RMI URI that can be used to connect to the server at startup time. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.admin.port</userinput></term> <listitem> <para> The TCP port that is used to listen for incoming RMI requests. The default port is 3998. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.dgram.backoff, server.dgram.initialtimeout, server.dgram.maximumtimeout</userinput></term> <listitem> <para> The default UberMQ overflow handler is an exponential backoff algorithm. The backoff starts at the initialtimeout value (in milliseconds), and is multiplied by the factor after each output retry. The exponential backoff fails if the maximum is reached and the output continues to overflow. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.dgram.maximumprops</userinput></term> <listitem> <para> The maximum size, in bytes, that a message datagram's properties can be. This is only used when a subscriber has a selector running. </para> </listitem> </varlistentry> <varlistentry><term><userinput>server.dgram.sendacks</userinput></term> <listitem> <para> Whether the message server should send acknowledgements back to publishers that their messages have been accepted by the server. This is highly recommended, otherwise publishers can easily publish information so quickly that a subscriber could never reasonably process it. </para> <para> Note that the acknowledgement does not indicate that the message has been received or processed by a subscriber, just that the message server has output the message to the appropriate endpoints. </para> </listitem> </varlistentry> </variablelist> </appendix> </book> --- ubermq-1.0.sgm DELETED --- |