Recent changes to server-performance

WikiPage server-performance modified by Paul

Paul — Thu, 08 Mar 2012 22:52:52 -0000

--- v9 
+++ v10 
@@ -3,63 +3,60 @@
 Performance Monitor
 ===================
 
-The performance monitor runs on the server. It accepts HTTP requests that carry status information and stores the information. Another URL can be used to generate a summary of the performance information.
-
-Here are some things to monitor:
-- For each orchestra, how many messages and bytes came into and went out of the super-node from each other super-node.
-- How many messages and bytes were sent to nodes.
-- How many messages and bytes were received from nodes.
-- What is the latency between LSU and each super-node.
-- How many nodes are connected in each orchestra.
-- What else would be good to know?
-
-Design
-------
-- Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and when the system clock is updated. A backup of performance information is logged locally and uploaded periodically via an HTTP GET request to the performance server. 
-- On the performance server (located at music.cs.cmu.edu), the HTTP requests update a corresponding node entry in a log file and whose potentially concurrent writes are protected by a local file lock. This is not the fastest method, but requires minimal setup and bandwidth should not be an issue even with <# of nodes> simultaneous requests. 
-- 
-
-
-:::java 
-# 
+The performance monitor manages status information from every node and super node. It consists of a server which collects status information, and client code which runs on every node and super node, periodically sending status information.
+
+Information collected is:
+- bytes sent and received over time for every node and super node
+- latencies between each node and its super node
+- latencies between each pair of super nodes.
+
+Client Design
+-------------
+Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and when the system clock is updated. Performance information is logged locally and uploaded periodically via an HTTP GET request to the performance server. 
+
+#
     //initialization:
     PerfMonitorNode perfMon = new PerfMonitorNode();
     //upon message sent:
     perfMon.sentMessage(int msgLength);   // length of message in bytes
     //Upon message received:
     perfMon.recvMessage(int msgLength); // length of message in bytes
     //Upon clock synchronization response:
     perfMon.rtt(int rtt);  // round trip time in milliseconds
 
-
-
-       
-
 PerfMonitorNode|  

 ------------- | -------------
 **Constructor Summary**        | 

 PerfMonitorNode() | Initializes a new performance monitor for your node, which should create a new thread that interrupts every N seconds to upload new data, creates a new log file unless one exists already, and create and clear some internal variables (or update them from the log file).
   |  
 **Public Method Summary** | 

 public void sentMessage(int msgLength)  | Should be called on every message sent to log outgoing bytes. Waits for lock to be free, acquires it, updates totalBytesSent and totalMsgsSent, then releases the lock.
 public void recvMessage(int msgLength) | Should be called on every message received to log incoming bytes. Waits for lock to be free, acquires it, updates totalBytesRecv and totalMsgsRecv, then releases the lock.
 public void rtt(int rtt) | Update latest round trip time (in milliseconds). Should be called inside clock synchronization function. 
   |
 **Private Method Summary** |
 private void sendUpdate() | Sends update of current log information to main server. Should be called every N seconds from a thread started at the constructor.
   |
 **Private Variables Summary** |
 private int totalBytesSent |
 private int totalBytesRecv |
 private int totalMsgsSent |
 private int totalMsgsRecv |
 private smoothedRtt | Smoothed round-trip time
 private Lock l | Private lock that manages writes to variables above. See [Lock](http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Lock.html)
 
-
+Server Design
+-------------
+The server system will run on music.cs.cmu.edu and consist of 4 files:
+- a Python script which receives GET requests from the nodes containing status information
+- a plain-text file containing aggregate status information
+- a lock file for the text file
+- a Python script that reads the text file and produces an HTML view of the data
 
 Subtask Assignments
 -------------------
-- Document who will do what when.
+- Torstein has written a first version of the node monitoring code.
+- Nolan is writing tests.
+- Paul is writing the server scripts.
 
 [Home](/p/floctrl/wiki/)

WikiPage server-performance modified by Nolan Hergert

Nolan Hergert — Thu, 08 Mar 2012 22:33:43 -0000

--- v8 
+++ v9 
@@ -13,54 +13,50 @@
 - How many nodes are connected in each orchestra.
 - What else would be good to know?
 
-Node Monitor Design
--------------------
-- Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and when the system clock is updated. Performance information is logged locally and uploaded periodically via a GET request to the monitoring server.
-
-
+Design
+------
+- Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and when the system clock is updated. A backup of performance information is logged locally and uploaded periodically via an HTTP GET request to the performance server. 
+- On the performance server (located at music.cs.cmu.edu), the HTTP requests update a corresponding node entry in a log file and whose potentially concurrent writes are protected by a local file lock. This is not the fastest method, but requires minimal setup and bandwidth should not be an issue even with <# of nodes> simultaneous requests. 
+- 
+
+
 :::java 
 # 
     //initialization:
     PerfMonitorNode perfMon = new PerfMonitorNode();
     //upon message sent:
     perfMon.sentMessage(int msgLength);   // length of message in bytes
     //Upon message received:
     perfMon.recvMessage(int msgLength); // length of message in bytes
     //Upon clock synchronization response:
     perfMon.rtt(int rtt);  // round trip time in milliseconds
 
 
 
        
 
 PerfMonitorNode|  

 ------------- | -------------
 **Constructor Summary**        | 

 PerfMonitorNode() | Initializes a new performance monitor for your node, which should create a new thread that interrupts every N seconds to upload new data, creates a new log file unless one exists already, and create and clear some internal variables (or update them from the log file).
   |  
 **Public Method Summary** | 

 public void sentMessage(int msgLength)  | Should be called on every message sent to log outgoing bytes. Waits for lock to be free, acquires it, updates totalBytesSent and totalMsgsSent, then releases the lock.
 public void recvMessage(int msgLength) | Should be called on every message received to log incoming bytes. Waits for lock to be free, acquires it, updates totalBytesRecv and totalMsgsRecv, then releases the lock.
 public void rtt(int rtt) | Update latest round trip time (in milliseconds). Should be called inside clock synchronization function. 
   |
 **Private Method Summary** |
 private void sendUpdate() | Sends update of current log information to main server. Should be called every N seconds from a thread started at the constructor.
   |
 **Private Variables Summary** |
 private int totalBytesSent |
 private int totalBytesRecv |
 private int totalMsgsSent |
 private int totalMsgsRecv |
 private smoothedRtt | Smoothed round-trip time
 private Lock l | Private lock that manages writes to variables above. See [Lock](http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Lock.html)
 
-Server Design
--------------
-The server system will consist of 4 files on www.music.cs.cmu.edu:
-- a python script which receives GET requests from nodes containing status updates
-- a plain-text file containing aggregate status information
-- a lock file for the text file
-- a python script which reads the text file and produces an HTML view of the data
+
 
 Subtask Assignments
 -------------------

WikiPage server-performance modified by Paul

Paul — Thu, 08 Mar 2012 22:33:10 -0000

--- v7 
+++ v8 
@@ -13,49 +13,54 @@
 - How many nodes are connected in each orchestra.
 - What else would be good to know?
 
-Design
-------
-- Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and when the system clock is updated. A backup of performance information is logged locally and uploaded periodically via an HTTP request to the performance server. 
-- On the performance server, HTTP requests update variables inside a large hash table whose keys are node names. There shouldn't be a concurrency issue, as each node entry is only edited every N seconds by only one node.
-
-
+Node Monitor Design
+-------------------
+- Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and when the system clock is updated. Performance information is logged locally and uploaded periodically via a GET request to the monitoring server.
+
+
 :::java 
 # 
     //initialization:
     PerfMonitorNode perfMon = new PerfMonitorNode();
     //upon message sent:
     perfMon.sentMessage(int msgLength);   // length of message in bytes
     //Upon message received:
     perfMon.recvMessage(int msgLength); // length of message in bytes
     //Upon clock synchronization response:
     perfMon.rtt(int rtt);  // round trip time in milliseconds
 
 
 
        
 
 PerfMonitorNode|  

 ------------- | -------------
 **Constructor Summary**        | 

 PerfMonitorNode() | Initializes a new performance monitor for your node, which should create a new thread that interrupts every N seconds to upload new data, creates a new log file unless one exists already, and create and clear some internal variables (or update them from the log file).
   |  
 **Public Method Summary** | 

 public void sentMessage(int msgLength)  | Should be called on every message sent to log outgoing bytes. Waits for lock to be free, acquires it, updates totalBytesSent and totalMsgsSent, then releases the lock.
 public void recvMessage(int msgLength) | Should be called on every message received to log incoming bytes. Waits for lock to be free, acquires it, updates totalBytesRecv and totalMsgsRecv, then releases the lock.
 public void rtt(int rtt) | Update latest round trip time (in milliseconds). Should be called inside clock synchronization function. 
   |
 **Private Method Summary** |
 private void sendUpdate() | Sends update of current log information to main server. Should be called every N seconds from a thread started at the constructor.
   |
 **Private Variables Summary** |
 private int totalBytesSent |
 private int totalBytesRecv |
 private int totalMsgsSent |
 private int totalMsgsRecv |
 private smoothedRtt | Smoothed round-trip time
 private Lock l | Private lock that manages writes to variables above. See [Lock](http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Lock.html)
 
-
+Server Design
+-------------
+The server system will consist of 4 files on www.music.cs.cmu.edu:
+- a python script which receives GET requests from nodes containing status updates
+- a plain-text file containing aggregate status information
+- a lock file for the text file
+- a python script which reads the text file and produces an HTML view of the data
 
 Subtask Assignments
 -------------------

WikiPage server-performance modified by Nolan Hergert

Nolan Hergert — Wed, 07 Mar 2012 23:44:36 -0000

--- v6 
+++ v7 
@@ -15,7 +15,8 @@
 
 Design
 ------
-- Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and updating the system clock. A backup of performance information is logged locally and uploaded periodically via an HTTP request to the performance server. 
+- Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and when the system clock is updated. A backup of performance information is logged locally and uploaded periodically via an HTTP request to the performance server. 
+- On the performance server, HTTP requests update variables inside a large hash table whose keys are node names. There shouldn't be a concurrency issue, as each node entry is only edited every N seconds by only one node.
 
 
 :::java

WikiPage server-performance modified by Nolan Hergert

Nolan Hergert — Wed, 07 Mar 2012 23:41:35 -0000

--- v5 
+++ v6 
@@ -18,17 +18,8 @@
 - Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and updating the system clock. A backup of performance information is logged locally and uploaded periodically via an HTTP request to the performance server. 
 
 
-
-
-:::java
-# test
-//On node/supernode:
-
-
-
-
 :::java 
-#     
+# 
     //initialization:
     PerfMonitorNode perfMon = new PerfMonitorNode();
     //upon message sent:

WikiPage server-performance modified by Nolan Hergert

Nolan Hergert — Wed, 07 Mar 2012 23:39:31 -0000

--- v4 
+++ v5 
@@ -23,24 +23,23 @@
 :::java
 # test
 //On node/supernode:
+
+
+
+
+:::java 
+#     
     //initialization:
-        PerfMonitorNode perfMon = new PerfMonitorNode();
-        perfMon.startBucketThread();
-
-
-
-:::java 
-# est 2       
+    PerfMonitorNode perfMon = new PerfMonitorNode();
     //upon message sent:
     perfMon.sentMessage(int msgLength);   // length of message in bytes
     //Upon message received:
     perfMon.recvMessage(int msgLength); // length of message in bytes
-
-
-Upon clock synchronization response:
-~~~~~~~~
-perfMon.rtt(int rtt);  // round trip time in milliseconds
-~~~~~~~~  
+    //Upon clock synchronization response:
+    perfMon.rtt(int rtt);  // round trip time in milliseconds
+
+
+
        
 
 PerfMonitorNode|

WikiPage server-performance modified by Nolan Hergert

Nolan Hergert — Wed, 07 Mar 2012 23:37:53 -0000

--- v3 
+++ v4 
@@ -62,7 +62,7 @@
 private int totalMsgsSent |
 private int totalMsgsRecv |
 private smoothedRtt | Smoothed round-trip time
-private Lock l | Private lock that manages writes to variables above. See java implementation of [Lock](http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Lock.html)
+private Lock l | Private lock that manages writes to variables above. See [Lock](http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Lock.html)

WikiPage server-performance modified by Nolan Hergert

Nolan Hergert — Wed, 07 Mar 2012 23:37:25 -0000

--- v2 
+++ v3 
@@ -45,13 +45,24 @@
 
 PerfMonitorNode|  

 ------------- | -------------
-Constructor Summary        | 

+**Constructor Summary**        | 

 PerfMonitorNode() | Initializes a new performance monitor for your node, which should create a new thread that interrupts every N seconds to upload new data, creates a new log file unless one exists already, and create and clear some internal variables (or update them from the log file).
   |  
-Method Summary | 
-public void sentMessage(int msgLength)  | Should be called on every message sent to log outgoing bytes
-public void recvMessage(int msgLength) | Should be called on every message received to log incoming bytes
-void resubscribe(String topic, FloReceiver fr) | Changes the FloReceiver object that handles messages for a given topic.
+**Public Method Summary** | 

+public void sentMessage(int msgLength)  | Should be called on every message sent to log outgoing bytes. Waits for lock to be free, acquires it, updates totalBytesSent and totalMsgsSent, then releases the lock.
+public void recvMessage(int msgLength) | Should be called on every message received to log incoming bytes. Waits for lock to be free, acquires it, updates totalBytesRecv and totalMsgsRecv, then releases the lock.
+public void rtt(int rtt) | Update latest round trip time (in milliseconds). Should be called inside clock synchronization function. 
+  |
+**Private Method Summary** |
+private void sendUpdate() | Sends update of current log information to main server. Should be called every N seconds from a thread started at the constructor.
+  |
+**Private Variables Summary** |
+private int totalBytesSent |
+private int totalBytesRecv |
+private int totalMsgsSent |
+private int totalMsgsRecv |
+private smoothedRtt | Smoothed round-trip time
+private Lock l | Private lock that manages writes to variables above. See java implementation of [Lock](http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Lock.html)

WikiPage server-performance modified by Nolan Hergert

Nolan Hergert — Wed, 07 Mar 2012 23:19:07 -0000

--- v1 
+++ v2 
@@ -15,8 +15,45 @@
 
 Design
 ------
-- Put a design here
-- Design should include both server code and client code. It's OK to just create an API where you offer a method to be called when a message is sent. It will be the job of the message sender to call your method with the requested information.
+- Performance monitoring occurs on every node in the network, including the super-nodes. All of these nodes are expected to call our functions upon receiving and sending bytes and updating the system clock. A backup of performance information is logged locally and uploaded periodically via an HTTP request to the performance server. 
+
+
+
+
+:::java
+# test
+//On node/supernode:
+    //initialization:
+        PerfMonitorNode perfMon = new PerfMonitorNode();
+        perfMon.startBucketThread();
+
+
+
+:::java 
+# est 2       
+    //upon message sent:
+    perfMon.sentMessage(int msgLength);   // length of message in bytes
+    //Upon message received:
+    perfMon.recvMessage(int msgLength); // length of message in bytes
+
+
+Upon clock synchronization response:
+~~~~~~~~
+perfMon.rtt(int rtt);  // round trip time in milliseconds
+~~~~~~~~  
+       
+
+PerfMonitorNode|  

+------------- | -------------
+Constructor Summary        | 

+PerfMonitorNode() | Initializes a new performance monitor for your node, which should create a new thread that interrupts every N seconds to upload new data, creates a new log file unless one exists already, and create and clear some internal variables (or update them from the log file).
+  |  
+Method Summary | 
+public void sentMessage(int msgLength)  | Should be called on every message sent to log outgoing bytes
+public void recvMessage(int msgLength) | Should be called on every message received to log incoming bytes
+void resubscribe(String topic, FloReceiver fr) | Changes the FloReceiver object that handles messages for a given topic.
+
+
 
 Subtask Assignments
 -------------------

WikiPage server-performance modified by Roger B. Dannenberg

Roger B. Dannenberg — Sat, 25 Feb 2012 04:08:40 -0000

[Home](/p/floctrl/wiki/) Performance Monitor =================== The performance monitor runs on the server. It accepts HTTP requests that carry status information and stores the information. Another URL can be used to generate a summary of the performance information. Here are some things to monitor: - For each orchestra, how many messages and bytes came into and went out of the super-node from each other super-node. - How many messages and bytes were sent to nodes. - How many messages and bytes were received from nodes. - What is the latency between LSU and each super-node. - How many nodes are connected in each orchestra. - What else would be good to know? Design ------ - Put a design here - Design should include both server code and client code. It's OK to just create an API where you offer a method to be called when a message is sent. It will be the job of the message sender to call your method with the requested information. Subtask Assignments ------------------- - Document who will do what when. [Home](/p/floctrl/wiki/)