Menu
▾
▴
#1401 smf: upgrade time improvement
Decrease the execution time for campaigns by merging parallel procedures.
Parallel procedures are procedures that have saSmfExecLevel set to the same value. When procedures are merged they will be executed in one thread. Today procedures are executed in separate threads with possible cluster/node reboots in each procedure. When procedures are merged they would install multiple software packages on a node with one reboot at the end.
There are three use cases
- Upgrade one node at the time in the cluster
- Upgrade all nodes at once using cluster reboot
- Upgrade several nodes at once
There will be an IMM attribute that decides if procedures are to be merged to maintain backward compatibility.
Related
Diff:
Diff:
Please close this ticket.
We dont allow "private" tickets.
Diff:
Diff:
Diff:
From bertils powerpoint
Proposed solution
Use existing rolling procedures but add new IMM configuration object that tells SMF how to execute the upgrade (one node at the time, all at once, several nodes at the same time). The information describing how to upgrade is given in IMM objects. This means we use the existing campaign schema.
Add possibility to configure a reboot upgrade (all nodes at the same time) as first priority. Then add rolling upgrade (one node at the time) as second priority.
As third priority add the possibility to configure upgrade of several nodes at the same time. This step requires more work to define exactly how you describe how many and what nodes should be upgraded at the same time.
This solution implies that applications can reuse their existing campaign files and does not need to adapt to this new feature (however testing is needed).
Issues to be considered during implementation
1) How to handle callbacks?
Callbacks is a mechanism in SMF where a campaign can specify that a certain callback should be sent out at a certain time during the upgrade. Any SW can subscribe to these callbacks and do something when they arrive. The callbacks is sometimes related to the steps in a procedure (first, every, halfway or last step).
2) How will the alternatives work with parallel node lock support in AMF. (e.g. lock/unlock on Node group)
If the procedures specifies nodes as activation unit SMF will lock the nodes one by one i.e. not in parallel (AMF can today only do one lock at the time). So SMF has to create temporary node group and do lock etc on that object. The new procedure type can specify node groups directly but someone still need to create them.
3) How to handle single step procedures?
If a campaign also includes single step procedures, what to do? Could be installation procedure for new CBA component.
This ticket represents the first of three development steps to minimize the upgrade time.
This first step will allow the user to configure SMF to merge all procedures (rolling and single step) in a campaign into a single step procedure.
A campaign is typically a result of merge of separate campaigns delivered with different independent components. The drawback of this is that the campaign may e.g. roll over the same node group several times. This will give major upgrade time issues for operators having applications with problem to handle rolling upgrades i.e. database applications may have to synch for each unlock in a rolling upgrade. They sometime prefer system outage for a shorter time than rolling upgrades going on for hours.
This patch will make it possible to write a campaign containing rolling procedures (which most of the operators prefer) and in special cases order SMF to run all the actions in a single step.
It will also help vendors which do not need to write specific campaigns for these systems.
Campaign prerequisite: Procedures execution order must be independent.
Coming steps are:
2) Merge campaign procedures into a rolling over nod procedure, which do all actions from all procedures for the specific node within the same lock/unlock/reboot sequence. This will prevent the campaign to run several rolling procedures over the same nodes.
3) Not exactly decided, but will update sections of the cluster in a rolling fashion.
E.g. SMF can be configured to update a 40 node cluster in chunks of 10 nodes i.e. 10 nodes are upgraded single step. Four steps will be executed.
Attached aatch for building on 32-bit machines.
changeset: 6962:38d62e4723f7
tag: tip
user: Ingvar Bergstrom ingvar.bergstrom@ericsson.com
date: Wed Sep 30 15:31:32 2015 +0200
summary: smfd: merge camp procedures into single step [#1401]
Related
Tickets:
#1401