gusdev-gusdev

[Gusdev-gusdev] schema features v. patches

[Steve F. <st...@pc...>]

  folks-

as the mail thread attached demonstrates, we are trying to figure out 
how to handle changes to the schema.  (the case attached arose because 
we upgraded our dbEST parser but never got a chance to test it on the 
schema before the schema was released, and now we have an urgent need to 
run the dbEST parser).

as we know, jonathan crabtree is our point man on this issue.

the main goal i see is to have a light-weight process in place for 
handling this.

here are my thoughts:

1. we need to decide which version of the schema the public schema 
browser points to.  it seems to me that it must always point to the most 
recent official version of the schema.  at CBIL, this might mean we need 
a db instance which is neither production nor development that houses 
the official schema.

2. i am going to venture the following tentative definition of a bug:
     - a correction to something we are already modelling
     - does not introduce any new tables
     - can it introduce new attributes?
     - *does not require migrating data*

3. if we do go with such a bug notion, then which tracker do we use? 
 here is a proposal:
    - all schema changes --bugs and features-- go to the schema feature 
tracker (might have to rename it)
    - after we release a version of the schema (including a patched 
version, eg 3.0.1) we add a "group" to the tracker called, eg, "3.0.1 
(bug)"  and any bugs against that version are tagged with that group
    - after we release a version of the schema, we add a "group" for the 
next version of the schema, eg "3.1".   feature requests are tagged with 
this group.  ultimately, we decide as a team which requested features 
are going to make it into the 3.1 release.  any that don't are either 
re-tagged with the next release "3.2" or maybe relegated to a "not 
scheduled yet" group.   Jonathan is responsible for making these changes 
(expressing the consensus).

4. we develop a light-weight mechanism for "patching" an instance, ie, 
the scripts that will surgically make the patch.  these always work on 
the most recent official version of the schema.  eg, patch 3.0.7 is only 
officially applicable to the 3.0.6 version of the schema

steve

-------- Original Message --------
Subject: Re: dots.est.polyA_siganl
Date: Mon, 17 Feb 2003 10:14:42 -0500 (EST)
From: "Deborah F. Pinney" <pi...@sn...>
To: Angel Pizarro <an...@sn...>
CC: Jonathan Crabtree <cra...@sn...>, Steve Fischer 
<st...@pc...>, <cb...@sn...>



Please note, changes are needed for otehr tables too including Clone,
Library,  and NASequenceImp.

NaSequenceImpOn Mon, 17 Feb 2003, Angel Pizarro wrote:

> On Mon, 17 Feb 2003, Jonathan Crabtree wrote:
>
> >
> > On Fri, 14 Feb 2003, Angel Pizarro wrote:
> > >
> > > right, sorry about that, I changed the assignment from Bugs to GUS Schema
> > > Features and assigned category = DoTS and group = 3.0
> >
> > We could actually handle this (and changes like this) in one of two ways:
> >
> >  1. Treat this as a feature request/schema change for 3.1 (since 3.0 is
> >     now "out the door" in some sense).  In that case, the appropriate
> >     group would be 3.1, not 3.0 (and we would create schema change entries
> >     in group 3.1 *before* the release of that version, versus bug fixes
> >     for version 3.1, which would be assigned to that group only *after*
> >     3.1 has been released.)
> >  2. Treat this as a bug/omission in 3.0.  In this case the chosen category
> >     (3.0) is correct, but we then have to worry about the issue of
> >     providing patches (so that people who have already downloaded what
> >     they thought was the 3.0 release can get updated to the version with
> >     the "bug" fix).
> >
> > I'd be inclined to go with #1, except in cases where the change is quite
> > clearly a bug.  For example, if we know (and have documented that!) the
> > est.polyA_signal column is supposed to be an exact replica of a column in
> > dbEST, and we fail to give it the same datatype/constraints, then that
> > is clearly a bug.  The problems we're running into with EST table at the
> > moment will hopefully not happen in future because we plan to actually
> > test and "certify" all the plugins against the proposed schema before
> > doing a release.
>
> I am for #2 since it is as you say a known bug. Which was why I had
> originally submitted it as a bug report, until Steve said that was not the
> place for these things. As debbie noted in a very recent email, there are
> other bugs that need to be addressed. We can prvide one large patch for
> the EST table to encompass all of the needed changes, or give them
> piece-meal. Or both.
>
> Angel
>
> PS. Sorry, I forgot about the version table change to the polya_signal
> column. Shoddy workmanship. won't happen again.
>

Re: [Gusdev-gusdev] schema features v. patches

[Jonathan C. <cra...@sn...>]

On Mon, 17 Feb 2003, Steve Fischer wrote:
> 1. we need to decide which version of the schema the public schema
> browser points to.  it seems to me that it must always point to the most
> recent official version of the schema.  at CBIL, this might mean we need
> a db instance which is neither production nor development that houses
> the official schema.

Yes, this is definitely true.  Since the current schema browser requires
an installed GUS instance, we'll either have to solve the multiple GUS
instances per Oracle instance problem, or perhaps set up a separate small
Oracle instance.

> 2. i am going to venture the following tentative definition of a bug:
>      - a correction to something we are already modelling
>      - does not introduce any new tables
>      - can it introduce new attributes?
>      - *does not require migrating data*

I don't think this is the right approach.  If the system does not perform
according to its documented specifications (or, more informally, if it
doesn't do what it's expected or supposed to), then there is a bug.  The
question of how much (and what kind) of work would be required to fix the
bug is an entirely separate question (and one that I think should not have
any bearing on whether the problem is in fact a bug.)  A bug fix could
potentially introduce new tables or attributes, or even require migrating
data.  It's very unlikely this would happen, but not impossible.  And,
once we institute the policy of testing plugins before doing a schema
release, the problem of having to modify the schema because it doesn't
support everything the plugins want to do should disappear.

> 3. if we do go with such a bug notion, then which tracker do we use?
>  here is a proposal:
>     - all schema changes --bugs and features-- go to the schema feature
> tracker (might have to rename it)
>     - after we release a version of the schema (including a patched
> version, eg 3.0.1) we add a "group" to the tracker called, eg, "3.0.1
> (bug)"  and any bugs against that version are tagged with that group
>     - after we release a version of the schema, we add a "group" for the
> next version of the schema, eg "3.1".   feature requests are tagged with
> this group.  ultimately, we decide as a team which requested features
> are going to make it into the 3.1 release.  any that don't are either
> re-tagged with the next release "3.2" or maybe relegated to a "not
> scheduled yet" group.   Jonathan is responsible for making these changes
> (expressing the consensus).

Sounds good, except I would prefer to associate bugs with the version of
the schema that *contains* the bug, not the version of the schema that--if
released--will (hopefully) fix it.

> 4. we develop a light-weight mechanism for "patching" an instance, ie,
> the scripts that will surgically make the patch.  these always work on
> the most recent official version of the schema.  eg, patch 3.0.7 is only
> officially applicable to the 3.0.6 version of the schema

Since we're incorporating the patch number into the version number system
we've already developed, we don't need a separate system for the patch files;
it will be the same as the system for the migration files.  Patches will
hopefully be less work than migrations, but that may not always be the case.
If you download release 3.0.2 and want to get to 3.0.7, you have to apply the
following patches/migration scripts:

3.0.2 -> 3.0.3
3.0.3 -> 3.0.4
3.0.4 -> 3.0.5
3.0.5 -> 3.0.6
3.0.6 -> 3.0.7

Basically I'm not sure what you mean by a "light-weight" patching
mechanism; the mechanism we've talked about for migrating from one release
to the next is already light-weight, in the sense that it does the minimum
amount of work needed to get from one to the other.

Jonathan

Re: [Gusdev-gusdev] schema features v. patches

[Steve F. <st...@pc...>]

about the "group" for bugs:  yes, it the version of the system that 
contained the bug.  (that is what i meant to say)

so, is there any operational difference between a patch and a release? 
 or is the only thing different which digit of the release number is 
affected?

steve

Jonathan Crabtree wrote:

>On Mon, 17 Feb 2003, Steve Fischer wrote:
>  
>
>>1. we need to decide which version of the schema the public schema
>>browser points to.  it seems to me that it must always point to the most
>>recent official version of the schema.  at CBIL, this might mean we need
>>a db instance which is neither production nor development that houses
>>the official schema.
>>    
>>
>
>Yes, this is definitely true.  Since the current schema browser requires
>an installed GUS instance, we'll either have to solve the multiple GUS
>instances per Oracle instance problem, or perhaps set up a separate small
>Oracle instance.
>
>  
>
>>2. i am going to venture the following tentative definition of a bug:
>>     - a correction to something we are already modelling
>>     - does not introduce any new tables
>>     - can it introduce new attributes?
>>     - *does not require migrating data*
>>    
>>
>
>I don't think this is the right approach.  If the system does not perform
>according to its documented specifications (or, more informally, if it
>doesn't do what it's expected or supposed to), then there is a bug.  The
>question of how much (and what kind) of work would be required to fix the
>bug is an entirely separate question (and one that I think should not have
>any bearing on whether the problem is in fact a bug.)  A bug fix could
>potentially introduce new tables or attributes, or even require migrating
>data.  It's very unlikely this would happen, but not impossible.  And,
>once we institute the policy of testing plugins before doing a schema
>release, the problem of having to modify the schema because it doesn't
>support everything the plugins want to do should disappear.
>
>  
>
>>3. if we do go with such a bug notion, then which tracker do we use?
>> here is a proposal:
>>    - all schema changes --bugs and features-- go to the schema feature
>>tracker (might have to rename it)
>>    - after we release a version of the schema (including a patched
>>version, eg 3.0.1) we add a "group" to the tracker called, eg, "3.0.1
>>(bug)"  and any bugs against that version are tagged with that group
>>    - after we release a version of the schema, we add a "group" for the
>>next version of the schema, eg "3.1".   feature requests are tagged with
>>this group.  ultimately, we decide as a team which requested features
>>are going to make it into the 3.1 release.  any that don't are either
>>re-tagged with the next release "3.2" or maybe relegated to a "not
>>scheduled yet" group.   Jonathan is responsible for making these changes
>>(expressing the consensus).
>>    
>>
>
>Sounds good, except I would prefer to associate bugs with the version of
>the schema that *contains* the bug, not the version of the schema that--if
>released--will (hopefully) fix it.
>
>  
>
>>4. we develop a light-weight mechanism for "patching" an instance, ie,
>>the scripts that will surgically make the patch.  these always work on
>>the most recent official version of the schema.  eg, patch 3.0.7 is only
>>officially applicable to the 3.0.6 version of the schema
>>    
>>
>
>Since we're incorporating the patch number into the version number system
>we've already developed, we don't need a separate system for the patch files;
>it will be the same as the system for the migration files.  Patches will
>hopefully be less work than migrations, but that may not always be the case.
>If you download release 3.0.2 and want to get to 3.0.7, you have to apply the
>following patches/migration scripts:
>
>3.0.2 -> 3.0.3
>3.0.3 -> 3.0.4
>3.0.4 -> 3.0.5
>3.0.5 -> 3.0.6
>3.0.6 -> 3.0.7
>
>Basically I'm not sure what you mean by a "light-weight" patching
>mechanism; the mechanism we've talked about for migrating from one release
>to the next is already light-weight, in the sense that it does the minimum
>amount of work needed to get from one to the other.
>
>Jonathan
>
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Gusdev-gusdev mailing list
>Gus...@li...
>https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>
>
>  
>

Re: [Gusdev-gusdev] schema features v. patches

[Jonathan C. <cra...@sn...>]

On Mon, 17 Feb 2003, Steve Fischer wrote:
> so, is there any operational difference between a patch and a release?
>  or is the only thing different which digit of the release number is
> affected?

In terms of the CVS structure and migration scripts, no, I don't think
there is any difference.  However, for users who aren't using CVS and who
are downloading tar'ed releases, a patch will typically consist of a
smaller set of files that you can install "on top of" or alongside an
existing full release.  We'll have to think about exactly how to implement
this, but here's one possibility; a tar file that contains

 1. Another tar file, which contains all the files under $PROJECT_HOME
    that have changed (or just the diffs, if we want to get fancy.)
 2. The schema migration scripts, which will make the database reflect the
    new files installed from 1.

After installing both of these parts we'd instruct people to re-install to
$GUS_HOME.

Jonathan