[Slashcode-development] Request for Comments: Automated Schema Updates for Slash

[Cliff W. <cl...@lr...>]

For those interested, I'd like to propose the following and open
up the floor for comments. If anyone is interested in this
subject, please speak up!


Automated Schema Updates for Slash
----------------------------------

While the current system for updating the Slash schema isn't all that
complex, it is a bit tedious, and as the number of Slash sites to
maintain increases, this tedium can then become an updating 
nightmare. Currently, all updates must be done by hand, for each 
site: this includes core updates, core plugin updates, core theme 
updates, third party plugin updates and third party theme updates. 
Rigth now, most of the core changes are tracked in 
slash/sql/mysql/updates (since MySQL is the only backend really 
being supported, at the moment). This is good, but what happens when 
site administrators have to worry about updates for third party 
add-ons? And for multiple Slash sites?

What is proposed is a community effort to design and implement an 
automated tool that can determine, parse, and apply any changes that 
need to be made to a Slash backend to bring it current with the 
updates in database schema and data storage methods employed by 
Slash, to reduce the hassle and the problems that occur with Slash 
software updates.

To simplify this problem, I have broken the issue up into 4 phases, 
including a "prep" phase:

	0. Prep

	I. Determining Current Schema Characteristics

	II. Parsing Schema Changes

	III. Applying Schema Changes

	IV. Updating Data

	V. Schema Verification

Now, to discuss the issues surrounding each phase:

0. Prep

This is basically the "housekeeping" step that needs to be done to 
the codebase. There are two major concerns: converting the current 
upgrades file into something that is both human and machine readable. 
The second is providing a system where third-party themes and 
plugins can specify their own updates, as a working system will need 
to consider those changes as well. This tool might also assist 
developers in keeping their changes aligned with changes in Slash 
(by providing warnings and errors when their changes fall out of 
date and conflict with the core), especially if the developer is 
following Slash development from CVS. (For the sake of argument, 
"third-party" is used to define anything not included with core 
Slash, and the determination of what is, and is not "core" is left 
up to the Slash Development Team). 

Much of the work has been done, since the upgrades file is heavily 
commented (including notations of the updates for T_ and R_ release 
flags, and plugins specific updates). If we can standardize on such 
notations, we can parse them out.


I. Determining Current Schema Characteristics

Before updating a schema, it must be analyzed for validity. The 
analysis doesn't necessarily have to be expensive, but checks 
bypassed here, may be necessary (and more expensive to 
implement) if put off to a later phase (most likely to the 
verification phase).

Analysis here could be as simple as a variable (one for the core 
code, and a variable each for every third party plugin AND theme). 
Or it could be as complex, involving a thorough (and processor 
intensive) analysis of the current schema for validity.

Regardless of the implementation, we must be able to provide some 
kind of starting point for the update tool to work from.


II. Parsing Schema Changes

From said starting point, a list of changes needs to be constructed 
from the given update data, from the core and from third party 
addons.


III. Applying Schema Changes

Changes are then applied to the site, with core changes being applied 
first and third-party changes applied last.


IV. Updating Data

In the event that schema changes from one version to the next, 
involve updating data in some manner, than these changes need to be 
done in this stage.


V. Schema Verification

The entire schema may then need to be verified. It might be wise to 
take the existing schema (both core and third party) and compare it 
to what's currently live and note the differences.


Note, that in the event of a multi-version update, the tool will run
the phases in the following order (say from version x, thru y, to z):
                     
	  Start     Vx -> Vy 	 Vy -> Vz   Fin
	I -> II -> III -> IV -> III -> IV -> V

Such a tool, once developed to handle a single site, automatically. 
Can then be applied to update Slash clusters, via "glue"-methods 
like shell scripts or Perl, cutting down on the laber necessary to 
prepare a Slash site for a software update, and minimizing the 
chance of human error in the update 
process.


Now I don't have a complete answer to this entire problem, so this
is where the community gets involved. If you are interested in 
working on this problem, I'd be very interested in hearing your 
thoughts. If enough like minded individuals are interested in 
putting their heads together, maybe we can come up with a workable 
solution that everyone can use.

I'm looking forward to reading the thoughts from this list.

- Cliff