client-connection

Home

Client-Side Connection Manager

Clients contact the web server via HTTP to register themselves. The server will give names and IP addresses for nodes this one should connect to (sometimes no connections should be made). Each client listens for new connections, nodes listen for their supernode, and supernodes listen for connections from nodes in their orchestra, and for other supernodes.

In effect, when the connection manager is started, it connects to what it should connect to in the federation that is already there, and waits for connections from things joining the federation in the future.

The supernode keeps lists of all nodes and supernodes it is connected to. This information is available through the API.
TODO: Does anyone need a message sent when things connect (e.g. chat)?

The connection manager for clients and supernodes encapsulates the setup and connection phase, and offers message delivery service in the form of the API below.

We leave the designation of who is the conductor up to the conducting team. It should be possible to do whatever they need to do through publish subscribe.

Design

Our design focuses on the network topology between the nodes and the supernodes and leaves protocol decisions to other modules, such as the Publication/Subscription module. As is such, our modules create connections between the nodes, supernodes, and the HTTP server, and allow other modules to communicate to specific parties via Strings.

HttpConnection: Handles connection to the HTTP server

- Constructor sets the server path
- init creates all needed federations, orchestras and nodes
- shutdown unregisters our user
- getSupernodeAddr gets the IP address of the supernode
- getOtherSupernodesAddrs gets the IP addresses of the other supernodes

NodeConnection: Encapsulates a leaf node

- init connects to the HTTP server and then to the supernode
- shutdown cleans up sockets and the HttpConnection object
- sendMessage sends a message to the supernode

SupernodeConnection: Encapsulates a supernode

- init connects to the HTTP server, then to all other supernodes. Listens for other supernode connections and child node connections
- shutdown cleans up all child connections
- sendNodeMessage sends a message to a node
- sendSupernodeMessage sends a message to a supernode
- connectedNodes returns the set of nodes connected to this supernode
- connectedSupernodes returns the set of supernodes connected to this supernode

These sets are not guaranteed to be up to date though, as sending a message can cause a connection to die. (If the other node was restarted, the connection will die when you write to it.)

Implementation Details

The connection manager is initialized during program start up, at which point it registers a timer in the GUI scheduler, to poll for network IO frequently. When polled, the connection manager uses select to determine which connections have anything to do.

When writing, we serialize the message, and send the size over beforehand. Then for reading, we read the size, and after reading that many bytes from the socket, then we deserialize it. This behavior is encapsulated in the NonBlockingIO object.

We connect in nonblocking mode, so we just start connecting, and then finish it when select tells us to. At that point we send a string which is our address (e.g. /ochh/node). This is so the accepting side will know the address of the newly connected node. Things get a little tricky as we have also enabled low level debugging messages that are strings.

There are many places where the code can throw exceptions. Hopefully, the code will give a short output detailing what happened, and continue in an appropriate fashion. Sometimes, we will close the connection which had a problem, and continue. Other times, the exception is considered fatal to the connection manager, and it will restart. Right now repeated problems in the connection manager will make it restart continuously. This part is a bit tricky, so if anyone runs into problems with exceptions from the connection manager, let us know.

Home