ReleaseMethod

Release method

Note: Page status :

  • 1st version about 99% done,
  • please all, tell me what you think (at the end, you'll have to sign on and commit yourself to respect it :-)
  • please, native English speakers, review and fix sentences, words, spelling, ... for clearer contents !
  • please, experienced software developers, review, comment, complete !

Here we define and describe the steps we use for building a new release,

  • from the ground up, right after we've just published the previous release to the world,
  • until the day when we say : now, it's OK, we can publish the new release, and shout it to the world.

These steps, their execution sequence and the associated work rules are the general framework that we use for working on SD, and have been using already for months and years (generally).

So this should not change a lot of your everyday life, SD developers !

Definition for votes : The majority of the dev.team means "half the number of active members + 1" or "number of agreements > number of disagreements + number of not interested" (only active members counted here).

We write it here in order to get a reference point, that is some kind of "rules table", for when questions arise, things get unclear, or even when we face conflicting opinions ... during an ongoing discussion.

The development task graph

Definition: a task is a job that 1 developer can achieve on his own in a limited amount of time.

The bunch of tasks that the dev. team have to achieve in order to deliver the final release can be sorted into a dependency graph where most tasks :

  • need 1 or more preceding tasks to be completed before getting started (except "root" tasks),
  • must be completed before 1 or more following tasks can get started (except "leaf" tasks).

This graph can be fairly complicated, but 1 thing is important if we want to avoid wasting time : in the ideal case, each task should be done only once (in other words, no task must be done twice).

This seems obvious for most tasks (fix / enhance code or artwork), but must all the more apply to testing !

In order to achieve this "no-return" strategy, each developer must keep every day in mind the "task graph", or at least the sub-graph where he's working and the frontier (interface) of this sub-graph with the other devs' ones.

Building a release is "simply" achieving tasks of the graph in the right order, from the beginning of the graph to its end : as the developments get closer to the final release publication day, each unfinished task gets closer to the "leave" tasks, and thus the amount of remaining work keeps decreasing (provided there's no return).

Tracking the tasks

Each and every individual task has to be tracked through a ticket in our ticket system (currently Trac, see the menu on the upper right of this page, and its "View Tickets" and "New Ticket" items). For bugs (filed by a user), the developer who picks up the bug is responsible to enter the missing ticket fields appropriately.

A ticket for a task must contain at least :

  • description : short description of a concrete, independent requirement of what has to be achieved (from the end-user's point of view when relevant)
  • summary : a brief, one-sentence summary of the description above (as clear and accurate as possible, avoid useless words ; have to get understood or recognised at first glance),
  • type : defect (for bugs) or enhancement / task (whatever, we plan to not degrade the game ;-)
  • version : the version from which we are starting the work,
  • milestone : the milestone when the task is expected to be achieved,
  • priority / component / keywords : ...
  • comments :
    • the first comment should analyse / elaborate the problem from a technical standpoint : as a discussion in order to find the right solution for complicated tasks, or as a transparent explanation of the chosen solution, when things are more obvious or straightforward.
    • then, the dependencies on other tasks have to be listed,
    • at the end of the analysis, when needed, the task has to be broken down into simple steps, which helps a lot in setting up a better schedule,
    • and an approximate target date has to be proposed for the final achievement.

For clearness and readability, complicated tasks should be analysed in a dedicated wiki page, the ticket only shortly summing up things and explicitly linking to this page.

When the analysis and scheduling is over, the ticket virtually moves to a "follow-up" state : it will now be used by the developers (of the task) and testers as the mandatory place for :

  • giving news on the on-going work, when important steps are achieved,
  • at least when the final work is committed to the SVN repo., and thus open for testing and comments,
  • the ticket should not be closed until all testers have given their agreement that it has been tested thoroughly (agreement given via comments in the ticket).
  • When you report a SVN commit or simply refer to a given state of the project, you should use the Trac feature to link to that given commit directly: "rXXXX" where XXXX is the number of the commit. When you do a SVN commit, you should use the ticket number in the commit message, in the form of "Re: #YYY" or "Fixes: #YYY" where YYY is the reference number of the ticket. Example: "Re: #9999 Use light dumper settings for Moon explorers"

Here's the way ticket status change in time, as the work on is progressing :

  • tickets are created (by default) with a new status, assigned to somebody (which means to "no one" actually, that is "not assigned"),
  • when someone picks it up to tell the team he's going to work on sooner or later, one (re)assigns it to himself (using his SF.net id, in the text field next to the "(re)assign" radio-button at the bottom of the ticket page),
  • when one actually starts working on it, he accepts it (through the "accept" radio-button),
  • when the work is finished, including testing, he closes it,
  • due to the limitations of our Trac instance at SF.net, we can't setup an "in test" status (it would make things clearer),
  • the "assignee" of a ticket may also change, as devs. change their mind, or share or takeover the work (but be careful about the new assignee agreeing the change if you are not him or her !).

Overview of our steps towards a new release

Here you find the list of steps of our release process, the way each step comes after the previous one, and a short sum-up of what we do during that step.

Step name Definition
Development plan building From the initial brain storming sessions, and the heavy thinking / sorting of the features to include / update in the future release,
to the point when we have built up a clear and sorted development task dependency graph, along with a well defined schedule,
through possibly multiple iterations for analysing task dependencies and consequences on other tasks (creating new tasks or modifying already identified ones)
Some few actual developments are allowed, but only those which are simple and straightforward to achieve, needing nothing of the above analysis and discussions (probably limited to postponed bug fixes and simple tasks)
Development From the day when we have a clear and sorted development task dependency graph, along with a well defined schedule,
to the end of actual development of this plan by each developer, but with no or weak coverage testing of the overall integrated software
Alpha Release From the beginning of serious in-team integration testing, and relevant bug fixing,
to a 'completed and stable enough' state which permits the publication of an Alpha release to a limited number of advanced users, for early end-user feedback
The software is probably not very stable, might often crash or even corrupt user data
Multiple Alpha release might occur, and thus as many loops in this Alpha Release step
Beta Release Feature freeze ; no new feature, whatever its limited 'size', can be included ; only bug fixes
The Alpha loops finish when stability and robustness makes it possible to release the software to a wider range of end-users, and when the development of all planned features is completed, only leaving bug fixes in the target => this release is the first Beta one
Release Candidate The software is almost ready for final release
No feature development or enhancement of the software is undertaken
Tightly scoped, precision bug fixes are the only code you're allowed to write in this phase, and even then only for the most heinous and debilitating of bugs. Same goes for any artist content.
Risky bug fixes must be postponed to after the final release, or the dev. team together can vote to put the project back to a new Beta Release loop
The RC loops finish when there's no remaining final release blocker bug and when all needed testing is over
Final Release No more change to the code or data files
Only final re-packaging and package testing
Back to a new Release Candidate loop if a blocker bug is discovered
At the end, the final packages are published and the new version is announced publicly

In short, now you understand it a bit more :

  • Development plan building = The energy of the whole team is mostly concentrated on proposing, discussing, analysing feasibility and dependencies between tasks, sorting and scheduling tasks, eliminating tasks which can't be achieved for technical or schedule reasons ; only few developments going on (postponed bug fixes, simple and no-discussion tasks, ...),
  • Development = ... under heavy development, no officially published release (except towards devs who don't build from SVN),
  • Alpha = Mostly feature-complete, published to power-testers, for early end-users feedback,
  • Beta = Feature-complete, but buggy, published to all users for more end-users feedback,
  • RC = Last tests, will be re-branded to 'Final' if no critical bugs, assumed to be 'killer-bug'-free,
  • Final = Final release, same code as last RC.

Description of our steps towards a new release

Here you find more details about each step of our release process, including concrete examples of what can be done and what can't.

Building of the dev plan

Right after a release is out, we start thinking about the next one. But we need to think it a bit before jumping inside the code and art-work !

First we define what we'd like to see in the future release (+ the may be next one(s)), as bug fixes and/or new features.

For this, we create 2 wiki pages :

  • 1 page to list what each developer would like to work on him/herself,
  • 1 page to list the remainder of things that anyone would like to see in the release, but no developer said he/she was willing to do it yet.

These pages only list simply described items : no deep technical explanations yet (later).

Each team member (and also willing contributors) writes his/her items, then makes corrections to them (it's an iterative process) :

  • from the second page, a developer can pick up an idea and add them to his personal plan,
  • obviously in this case the item is removed from the second page and added to the 1st one,
  • a developer can also change his/her mind and remove items from his/her plan : move an item to the second page, or not - if he/she finally doesn't find it useful for the release.
  • some discussions will likely happen about things that appear as needed by someone, and not needed, or difficult to achieve for others, ... etc.

Once things get stable (no new idea or planned items), the team has to agree on the whole set of features and bug fixes chosen by the developers / artists (of course, items from the second page will not get achieved in the release - they are 'iced').

Then comes the time of technically analysing the chosen list of features and bug fixes : naturally, the dev. who picked the item leads / animates / writes down the analysis, but with mandatory help of the director and expert(s) of the relevant technical domains of the game (see Roles and responsibilities). No work should be started without the chosen implementation being approved by this/these expert(s). Each bug fix, new feature, chore should (at least informally) be examined / analysed the classical way when developing any real-life project :

  • requirements : what do we precisely want, as far as the end-user is concerned ? What does the "simply described item" mean precisely ?
  • specifications : what does this mean exactly, with regards to the game components and features ?
  • design :
    • how can this be implemented (possibly multiple solutions, and - very important - consequences on other game components), by the means of the code and artwork ?
    • how much working time will each possible solution cost?

While the above process can often stay informal for simple bug fixes or new features (only described inside the ticket text), more complicated ones need something more formal ; a simple wiki page is often enough and will keep track of the required thinking before taking the hands inside. Very small but valuable effort, which gives you a simple reminder reference when questions arise some months later ... and you've forgotten most of the details, or even rethought them differently ;-)

In the end, given that each item :

  • is clearly specified (what),
  • has been chosen a technical solution for the implementation (how),
  • has an estimated work time cost (when),

the release schedule can be built.

Of course, such an analysis & scheduling process may likely :

  • add new items in the plan,
  • change some existing items,
  • make us realize that some chosen items are too complicated or would take too much time for the target release date,
  • ...

So, even at this point, some iterations back to the 2 wiki "will / would like" wiki pages might be needed.

Development

The most fun and the longest step : hand on !

Theoretically speaking :

  • No new = unplanned feature can be introduced here (must be postponed after the release),
  • Nothing can be removed from the dev. plan,
  • The work and release schedule must be respected.

BUT ... life is life : the above rules are actually too strict for our leisure-time activity !

Here are such 'out-of-law' cases, and the solution to cope with them :

  1. Some new unplanned task might be discovered, as development goes on and devs dig deeper into the technical details during the implementation of the planned features !
    1. The dev. team must discuss and decide what to do through a vote :
    2. after all the needed directors and experts of the involved technical domains have analysed the problem, and proposed solutions,
    3. at least the majority of the dev. team must agree with the chosen solution.
    4. The work and release schedule has to be reviewed and updated, because of the extra work needed (no choice).
  2. In case a new developer / artist joins the dev team and chooses to collaborate the current release ...
    1. He/she should be assigned features / bugs fixes which were possibly not in the plan, or he/she can take over some planned tasks from another developer ...
    2. In either case, the dev. plan has to be (partially) updated as deep as needed : new features / bug fixes must be analysed the same way as they should have been if proposed at the beginning of the cycle, in order to track every possible dependency among the new tasks and the already planned ones.
    3. And then the work schedule has to be updated accordingly.
    4. But taking care of not changing the release' schedule ... unless the majority of the dev. team agrees
  3. Same as (2) in case of a simple re-assignments of planned tasks to different developers.
  4. Same as (2) in case the team (or only a developer) really wants to add something to the plan : things that can be realized on the go, should be done, but only if they do not extend beyond the scope of the task they arose with.
  5. If some dev. discovers or realizes that some planned task will likely not be achieved on time for any reason (technical ones, real-life ones, ...),
  6. He/she must warn the team as soon as possible.
  7. Then the dev. plan must be re-analysed and updated to take into account all the consequences of this task(s) removal :
    1. technical (new) work to be achieved in order to revert or hide the changes already done to the code and art-work : this may be something really difficult, especially for complicated changes, so the devs must set up some technical tricks when they know in advance, or heavily suspect, or ... that the revert may be needed some day)
    2. re-scheduling work and release stages, of course (no choice).

In any case, for each of these "problematic" cases,

  • the dev. team must take and write a clear decision, based on a vote if needed, in order to prevent further discussions as much as possible,
  • as a general rule of thumb, the work schedule should not be inflated through the back-door by extending scheduled tasks beyond their planned scope and impact.

Alpha Release

The code and art work is mostly completed (say 95 %) with respect to the features planned for the final release. But the software may be unstable, may often crash or even corrupt user data.

This stage exists mostly for :

  • urging the last unfinished - but planned - developments,
  • running serious in-team tests after most / all of the planned developments have been integrated together,
  • fixing any discovered bug, or make needed improvements or optimizations.

When some advanced end-users are there and willing to help the dev. team towards its final target, some alpha releases may occur especially for their needs. Some dev. team members may also require early Alpha releases, when they can't build the game from sources for any reason, but need the most current code changes for working on art-work, for example.

New features can't be entirely started during alpha release stage : only unfinished ones may be continued, provided the estimated amount of time and work are compatible with the schedule, i.e. provided that the remaining implied tasks are close to the leaves of the task tree. When a planned feature is not yet started, or not enough advanced with respect to the schedule, the dev. team must discuss and decide through a vote to include or not the feature in the final release.

Examples of forbidden actions :

  • add a new car in a HQ set (would need physics and robot setups, huge and shared work),
  • add a new track in a HQ set (would need physics and robot setups, huge and shared work),
  • heavily rework the way starting grids are computed,
  • add a brand new race mode (a new XML file in src/raceman : too much testing),
  • do big changes to the 3D model of a car (might imply long discussions, introduce visual bugs with wheels or lights, ...),
  • completely change the engine sound of a car (long discussions)

Examples of authorized actions :

  • introduce a new building in a track, provided it's not a pit building (big graphics change, but not physics-related in any way),
  • do small changes to the 3D model of a car (warning : affects all liveries ; but don't imply long discussions, or introduce visual bugs with wheels or lights, ...),
  • fix bugs in the new "Resume race from results" feature, after first in-depth tests showed multiple issues,
  • add a new livery to a car (but implies re-shooting the car in the showroom for the in-game preview),
  • change the highlighting colour scheme of the menu buttons (mouse hover / click),
  • complete the last bits in the lists of tracks, cars and opponents for a newly introduced race mode (but not rework it much),
  • fix issues of the engine sound of a car which don't imply long discussions,
  • rework the layout of a menu screen

Beta Release

The code and art work is completed with respect to the features planned for the final release, and the software is stable enough for publication to power-users, aka "beta-testers" ; but some bad bugs may still be there ...

This stage exists mostly for :

  • intensive and all-domain testing, by the dev. team members, and by some end-users when a beta release has been published,
  • fixing any discovered bug, or make necessary minor improvements or optimizations, provided that :
    • the changes to the code or art-work are small and limited in scope and consequences,
    • they don't imply re-doing extensive tests,
    • they are easy to achieve for a sub-team of 1 or 2 devs at most, and don't imply possibly lengthy discussions in the team to decide what to do,
    • testing them is something straightforward and doesn't take a long time.

Thus, the authorized changes in the beta stage should be limited as much as possible to the "leaves" of the development graph, or at most up to the closest nodes, in order not to imply redoing tests that were already done.

This means that a good and clean fix should often be postponed to after the final release if a simple workaround exists and makes the fix more tightly scoped (an example of dynamically keeping the task graph from growing without control right before the release).

When for some reason a bug fix is close to or will likely cross these limits, the dev. team must discuss and decide what to do through a vote :

  • after all the needed experts of the involved technical domains have analysed the problem, and proposed solutions,
  • at least the majority of the dev. team must agree with the chosen solution (if any acceptable one),
  • at least 1 developer who knows about the involved technical domains must commit himself to intensively test the solution and other relevant features (checking for non-regression) : if the implementer is really the only one who is able and if things are really so complicated that it would be wasted time to teach someone else how to achieve it, then the implementer is the one ; but otherwise (most cases), the tester must be someone else,
  • if at least one of the above conditions are not met, even after thinking it hard and examining even the dirtiest possible solutions ... then we are back to an alpha release loop (but it's something quite unlikely at this stage).

The developers who take part to the changes are responsible of carefully observing if their changes cross these limits, and notify the team if so. But each member of the dev. team should also do this naturally : two eyes do better than one.

Examples of forbidden actions :

  • any action forbidden in the Alpha Release stage
  • modify the 3D model of a car (might affect many liveries, imply long discussions, introduce visual bugs with wheels or lights, ...),
  • introduce a new building in a track (big graphics change, might introduce graphics bugs),
  • modify the way starting grids are computed,
  • add a new race mode (a new XML file in src/raceman),
  • change the highlighting colour scheme of the menu buttons (mouse hover / click),
  • rework the layout of a menu screen (long discussions)

Examples of authorized actions :

  • fix the broken "Resume race from results" feature, after analysing and realizing that some simple data was missing in the results file,
  • fix a car livery bug (but implies re-shooting the car in the showroom for the in-game preview),
  • update the preview image for some liveries of some cars,
  • improve text of titles, labels and tips in menu (but beware about not starting long taste-only-related discussions),
  • fix typos in menus,

Release Candidate

Most of the killer bugs have been already fixed ... but some few ones might appear again, as more end-users try the game, and as the dev team is going on testing again and again.

But only very tightly scoped and no-risk "killer" bugs must be fixed, provided that the consequences of the fix don't imply any "more than 1 minute" discussion. If a killer bug can't get fixed through a tightly scoped and no-risk change to the code, or implies long discussions, even after thinking it hard and examining even the dirtiest possible quick fixes ... then we are back to a beta release loop.

Non-killer bugs are always postponed after the final release.

Example of forbidden actions :

  • any action forbidden in the Beta Release stage
  • add a new livery to a car (implies a new preview, and possibly a review by some other artist, too late)

Examples of authorized actions :

  • fix a typo in a menu label or tip,
  • update end-user documentation (install instructions, user manual, ...).

Final Release

Everything is done now, except for the publication, as the final release is simply the last Release Candidate, after it has been accepted as "The Good One" by the dev. team and the end-user testers.

At this step, a good publication package has to be finalised :

  • repackage the last RC into the final one (see How to build release packages),
  • write a clear, short and smashing end-user release notes (a sum-up of the full release notes already written for each alpha, beta, RC release),
  • write a shiny announcement text and take the best possible screen shots showing off what's new and awesome in the release
  • publish the final packages at SF.net (our primary release channel) and everywhere needed,
  • publish the announcement text and screen shots everywhere (social networks, forums, news, boards, community, ...).

Future changes of the Release Method

Of course, the above described rules, as well as the other "dev. rules" :

... might change in the future, as we get more experienced and as the team gets new members ... so there's need for another rule ... for changing the rules (this one included).

It is a simple one : if for some reason something has to be changed in the Release Method, a two-thirds majority of the dev. team has to agree.

Tips and tricks

How to make it simple to revert changes to the code or art-work

When, for some reason, some already well-advanced changes to the code or art-work have to be hidden or reverted in order not to threat the release schedule, things may get complicated ... The developers must set up some technical tricks when they know in advance, or heavily suspect, or ... that the revert may be needed some day.

These tips include :

  • conditional compilation (code) : use of "#ifdef SomeSymbol" constructs, along with a CMake variable for enabling / disabling the symbol definition in a few seconds (and a rebuild),
    • easy to set up,
    • but might bloat the code when there are multiple code changes in multiple source files,
    • this drawback can get far less annoying when combining this tips with the "code separation" one,
  • code separation (code) : code the changes in new source files, and connect the new code to the old one through as few as possible calls to the new services, under come computed conditions
    • a simple variant is used in the track module, for dealing with track files of different formats (track .cpp, track2.cpp, ... + a dispatching switch construct somewhere),
    • a more sophisticated one is used for the physics engine modules : Simu V2.0, V2.1, and V3.0 implement the same interface, but in different ways,
    • ...
  • VCS branching (code, artwork) :
    • a dedicated SVN branch might be created for developing a set of "suspect" features ...
    • but the branch must be merged one day into the main release branch (the trunk) ... and this may be a difficult task ... ????? (kilo: khm... git... khm...)

References

Archives

Related

Wiki: ArchivesReleaseMethod
Wiki: DevelopersCharter
Wiki: Home
Wiki: Index
Wiki: ReleaseMethod
Wiki: RolesResponsibilities
Wiki: ThePeopleBehind

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.