Provide admission controls on Teams specific to resources to reject Jobs that are unlikely to be completed within a threshold. This is achieved by providing running average of Job completion times for the Team and multiple by number of Jobs queued for the Team to determine which Jobs are unlikely to be processed within the threshold.
Each Team would then report periodically (and immediately on identifying overload conditions) to a central co-ordinating manager. The manager will then be able to indicate to each WoOF server where to redirect requests for the overloaded Jobs to better service them within the Threshold time (i.e. away from any WoOF server that is experiencing overload conditions).
The manager will also be able to indicate when a resource is saturated. If all WoOF servers are indicating their respective Team is overloaded, then the manager can indicate to send "too busy" page (along with sending alerts for manual intervention).
Elastic scale can then be considered by adding further WoOF servers when admission control happens at the request level. Monitoring average request servicing times can determine if the WoOF server is over a threshold. The manager may then either inform the WoOF server to redirect load to another existing WoOF server or create a new WoOF server to handle load (elastic scale). The Team loads can then be monitored to determine if adding the additional server has overloaded a particular resource. Note a minimum threshold can also be set that allows candidate WoOF servers to be shutdown as no longer necessary. For an IaaS, look at billing cycle for when best time to spin down the instance (e.g. if billing by hour, look to shut down 5 minutes before top of hour as starting the instance means paying for it for the hour so make good use of what is paid for).
While WoOF is stateless (with HttpSession likely kept relevant by cookie and peristed between requests), it puts the scaling constraints on the underlying resources (managing persistent state). For many uses, a relational database would handle most company situations with a NoSQL sharded store providing potentially better scalability. Once load is pushing beyond this, it will be a point for relooking at the application architecture (and a problem the company will happily have as they are likely to now be generating enough money to provide custom enhancements such as separating information storage to overcome this).
Log in to post a comment.