Re: [Tsheetx-developers] Database Config Tables

[Scott M. <sco...@gm...>]

Mark, can you give me a brief outline of how the system figures out if it
needs to upgrade itself?  I think I'm done with the sql stuff, but need to
test it, and want to know what calls what.

I'd start figuring it out on my own, but I need to head out for the weekend
myself, hopefully you'll have responded by the time I get in Morning :-)

-Scott

On Fri, Feb 18, 2011 at 8:16 PM, Scott Miller <sco...@gm...>wrote:

> Oh, I'd forgotten another issue I wanted to discuss.  Do you really think
> that it's necessary to have the configuration table use a long text field?
> Honestly, I'm thinking a tiny text field (256 characters) might be good
> enough, especially since we're planning to not have HTML info in this new
> configuration table.  However, as soon as we agree to cover ourselves for
> the need for longer fields with text or medium text, we may as well go with
> long text. The same amount of storage is actually used (the actual size of
> the data) with any of those text types.
>
> So, is 256 characters enough for a limit in our new configuration table?
> The longest field needed in the new configuration table so far takes up less
> than 30 characters.
>
> -Scott
>
>
> On Fri, Feb 18, 2011 at 8:04 PM, Scott Miller <sco...@gm...>wrote:
>
>> Yes, using svn for version numbers would be a good idea, I remember doing
>> some research on this, but I've never actually done it.
>>
>> I like the ideas behind how to update/upgrade in the future.  But for now,
>> what I think I'm going to try to do is take the old way of having an
>> sql.in file for our next version (1.5.3), and add additional logic behind
>> that to convert the existing config table and turn it into the new
>> configuration table, then when the "real" sql file is run against the
>> database, that work is done correctly.  So, from that, I think the added
>> logic needed for the version 1.5.3 upgrade stuff will actually get placed in
>> the install script itself...
>>
>> -Scott
>>
>>
>> On Fri, Feb 18, 2011 at 4:08 PM, Mark Wrightson <ma...@rw...
>> > wrote:
>>
>>>  Hi Scott,
>>>
>>> The way you would 'auto increment' the version would be to use the svn
>>> feature that can insert version numbers into a specific place in a file upon
>>> commit.  But i think auto increment is a bad idea.
>>>
>>> I am going to be away all weekend so won't be able to do any work.  I
>>> have been thinking about how to do upgrades a bit and what I was thinking
>>> was a bit like having 1.5.3.sql also have a 1.5.3.php file with a string of
>>> instructions that can be run.
>>>
>>> How about having a folder: 'install/version' ?
>>> For now maybe place your cvt-cfg.php in install/version as '1.5.4.php' or
>>> whatever the next version number is? and then modify the text in the
>>> upgrade.php file to indicate that this file needs to be run?
>>>
>>> For future work: to form a structure for the upgrade.php files I think we
>>> should have something like:
>>>
>>> class upgrade_1_5_4 extends Upgrade{
>>>         .... some stuff in here
>>> }
>>>
>>> abstract class Upgrade{
>>>
>>>     abstract public function __construct();
>>>     abstract public function functionName();
>>> }
>>>
>>> If the upgrade php files have a common structure and known function names
>>> then it would mean the upgrade script could be written quite robustly.
>>>
>>>
>>> Regards
>>> Mark
>>>
>>> _____________________________________________
>>>
>>> Mob:  07725 695178
>>> Email: ma...@rw...
>>>
>>>
>>> On 18/02/2011 15:41, Scott Miller wrote:
>>>
>>> Mark,
>>>
>>> Well, what I have is a php script that takes the LDAP and other misc
>>> stuff from the existing config table and inserts those entries into the new
>>> configuration table.  So, it is also ignoring all the html and acl entries.
>>> It changes several LDAP entries and creates the new LDAPurl item, and
>>> modifies the version to be 1.5.3.  I've thought about trying to figure out
>>> how to "auto increment" the version, but I'm thinking we actually don't want
>>> that to happen.
>>>
>>> So, this is in a script currently called cvt-cfg.php in the install
>>> directory.  I'm just now starting to think about how and where to run this
>>> script...  should it be imbedded as part of the upgrade script(s)?  Or
>>> should I just check this new file in for now?
>>>
>>> -Scott
>>>
>>> On Thu, Feb 17, 2011 at 10:25 PM, Mark Wrightson <
>>> ma...@rw...> wrote:
>>>
>>>>  Hi Scott,
>>>>
>>>> Yeah remove the id column.  The option name should really be the unique
>>>> key anyway.
>>>>
>>>> If you:
>>>>
>>>> 1.create an sql file with the change in install/sql/{version_number}.sql
>>>> 2. increment the version in the config table, and put that in the sql
>>>> file from above.
>>>> 3. increment the version value in config.factory.class.php (such that
>>>> this value will match the value entered in the sql above)
>>>> it will force the upgrade script to appear and inform developers that
>>>> there is a change to the database :)
>>>>
>>>> I think for the time being we should sideline the authorisation changes
>>>> as they do work, and concentrate on finishing the css changes that are still
>>>> on going.  Some user testing wouldn't go a miss either.
>>>>
>>>> Regards
>>>>  Mark Wrightson
>>>>
>>>> _____________________________________________
>>>>
>>>> Mob:  07725 695178
>>>> Email: ma...@rw...
>>>>
>>>>
>>>>   On 17/02/2011 21:52, Scott Miller wrote:
>>>>
>>>> Hey Mark,
>>>>
>>>> I've actually had time to do some stuff with it today, yesterday ended
>>>> up being filled with work.
>>>>
>>>> Can I propose that we remove the ID column from the new configuration
>>>> table?  I can't see that being useful in the future...
>>>>
>>>> -Scott
>>>>
>>>> On Thu, Feb 17, 2011 at 9:06 AM, Mark Wrightson <
>>>> ma...@rw...> wrote:
>>>>
>>>>>  Ah right ok.  It may take a while to rewrite the install script.  I
>>>>> need to get a structure on paper first i think.
>>>>>
>>>>> I'll have a think about the excel stuff.  One of the feature requests
>>>>> was to output to csv also.
>>>>>
>>>>> _____________________________________________
>>>>>
>>>>> Mob:  07725 695178
>>>>> Email: ma...@rw...
>>>>>
>>>>>
>>>>>   On 16/02/2011 16:41, Scott Miller wrote:
>>>>>
>>>>> Yes, the installer stuff is what I was talking about.  If you can clean
>>>>> it up, great, I don't have a good vision for how that would look, so the
>>>>> best I can do is take what we've got and continue to add in our new
>>>>> changes.  I should be able to fix bugs in the current system if you can find
>>>>> them, but again, if you can re-write it so that it's not nearly as
>>>>> complicated and works better, I'd say that would be a good thing.
>>>>>
>>>>> As for the excel exporting, there was a really old mechanism that I
>>>>> could never get to work, and someone else had created and submitted a
>>>>> different report with a different mechanism that worked for me, so I just
>>>>> took what worked and applied it to the rest of the reports.  The way it
>>>>> works, we would have had a ton of "replicated code" if we attempted to have
>>>>> exporting call a different script than the one that created the report for
>>>>> the screen, and that would have been even worse to maintain.  Again, if you
>>>>> have a better way to do it, great.
>>>>>
>>>>> -Scott
>>>>>
>>>>> On Wed, Feb 16, 2011 at 4:28 PM, Mark Wrightson <
>>>>> ma...@rw...> wrote:
>>>>>
>>>>>>  Hi Scott,
>>>>>>
>>>>>> I don't understand what you mean?
>>>>>> ---
>>>>>>
>>>>>> "Yes, it took me quite a while to wrap my brain around what all was
>>>>>> happening.  The same script is called repeatedly for each step of the
>>>>>> process, so figuring out how and where to add things was interesting."
>>>>>>  ---
>>>>>>
>>>>>> Ohhh - do you mean how the original install script worked was
>>>>>> interesting?  If that is what you mean then yes i do agree.  I'm looking to
>>>>>> take inspiration from the old install script and write a new object
>>>>>> orientated version.  The main grief I had with the old installer was that if
>>>>>> you asked it to create the database for you, it sometimes messed things up
>>>>>> good n proper and got itself all confused.  In the end I went through
>>>>>> phpmyadmin, reversed what it had done and setup the database manually
>>>>>> through that.
>>>>>>
>>>>>> Yes completing the transition from the old config to the new config
>>>>>> would definitely be good :)
>>>>>> Other things that need looking at are going through and just seeing if
>>>>>> the timesheet app works because very little has been tested since it was
>>>>>> rewritten.
>>>>>> Oh also the excel export features on a couple of pages really have
>>>>>> been hacked into tsng.  the view and export options should be totally
>>>>>> separate pages all together to tidy up the code instead of being munged all
>>>>>> into one file.
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Mark
>>>>>>
>>>>>> _____________________________________________
>>>>>>
>>>>>> Mob:  07725 695178
>>>>>> Email: ma...@rw...
>>>>>>
>>>>>>
>>>>>>   On 16/02/2011 16:06, Scott Miller wrote:
>>>>>>
>>>>>> Yes, it took me quite a while to wrap my brain around what all was
>>>>>> happening.  The same script is called repeatedly for each step of the
>>>>>> process, so figuring out how and where to add things was interesting.
>>>>>>
>>>>>> I'll help get that fixed up if you want, but today, I'll attempt to
>>>>>> work on taking data from the existing config database and populating our new
>>>>>> configuration table with that stuff.  And if I get that done, I'll then
>>>>>> start working on getting that config stuff loaded with the config classes...
>>>>>>
>>>>>> Maybe tomorrow I'll do something about the install/upgrade stuff...
>>>>>>
>>>>>> -Scott
>>>>>>
>>>>>> On Wed, Feb 16, 2011 at 12:49 AM, Mark Wrightson <
>>>>>> ma...@rw...> wrote:
>>>>>>
>>>>>>>  Hi Scott, Ok I will commit everything that I have done.
>>>>>>> This will then put you in a good position to complete the transfer
>>>>>>> from the old config table to the new one.
>>>>>>>
>>>>>>> I am just adding in some warnings to the install script to say that
>>>>>>> it isn't complete and how to do a manual update.
>>>>>>>
>>>>>>> It is now 00:47 here. :)
>>>>>>>
>>>>>>>
>>>>>>> Mark
>>>>>>>
>>>>>>> _____________________________________________
>>>>>>>
>>>>>>> Mob:  07725 695178
>>>>>>> Email: ma...@rw...
>>>>>>>
>>>>>>>
>>>>>>>   On 15/02/2011 23:47, Scott Miller wrote:
>>>>>>>
>>>>>>> well, considering it's in the demo branch, I'd say commit it now.
>>>>>>> Might quickly document that the install script isn't working yet...
>>>>>>>
>>>>>>> I'd like to help you, but I'm a bit wary about what part to change as
>>>>>>> much as you changed within the past day or so :-)
>>>>>>>
>>>>>>> -Scott
>>>>>>>
>>>>>>> On Tue, Feb 15, 2011 at 11:44 PM, Mark Wrightson <
>>>>>>> ma...@rw...> wrote:
>>>>>>>
>>>>>>>>  I think we should skip 1.6 and go straight to 2.0 - a major update
>>>>>>>> requires a major release number. :)
>>>>>>>>
>>>>>>>> Yes, I was the one suggesting removing everything except the string
>>>>>>>> field in the config table.
>>>>>>>>
>>>>>>>> This install script stuff is quite complex, i'm now at the point
>>>>>>>> where the site will automatically detect that the database version is no
>>>>>>>> longer
>>>>>>>> in sync with the codebase version and will start an upgrade
>>>>>>>> process.  I've also got the hooks in place to detect a new installation.
>>>>>>>> The new install script won't need to be deleted and it uses the
>>>>>>>> templating system so can use a theme to make it look really smart :)
>>>>>>>>
>>>>>>>> When do you think I should start committing it back up? - it is
>>>>>>>> currently at a stage where it tells the user what needs to happen, just
>>>>>>>> doesn't actually perform the install /upgrade.  I think the actual install
>>>>>>>> script bit could take a while to write.  I could commit it now, it would
>>>>>>>> just mean that only a manual install would be possible.
>>>>>>>>
>>>>>>>>
>>>>>>>> Mark
>>>>>>>>
>>>>>>>> _____________________________________________
>>>>>>>>
>>>>>>>> Mob:  07725 695178
>>>>>>>> Email: ma...@rw...
>>>>>>>>
>>>>>>>>
>>>>>>>>   On 15/02/2011 22:59, Scott Miller wrote:
>>>>>>>>
>>>>>>>> I think with all the changes we're making with this version, it will
>>>>>>>> be version 1.6 :-)
>>>>>>>> I don't remember if you were the one to suggest removing all but the
>>>>>>>> string variable in the new config table, I'd thought about that myself, so,
>>>>>>>> that's probably a good direction to head.
>>>>>>>>
>>>>>>>> -Scott
>>>>>>>>
>>>>>>>> On Tue, Feb 15, 2011 at 5:16 PM, Mark Wrightson <
>>>>>>>> ma...@rw...> wrote:
>>>>>>>>
>>>>>>>>> Hi Scott,
>>>>>>>>>
>>>>>>>>> I thought I would drop you an email to ask what position you were
>>>>>>>>> in regards to updating the config database table?
>>>>>>>>> last night in my work to update the installer I had to write a
>>>>>>>>> small amount of code along the lines of what you are working on.
>>>>>>>>>
>>>>>>>>> I made the following additions to config.class.php
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> One new function:
>>>>>>>>>
>>>>>>>>>    public static function getDbConfig(){
>>>>>>>>>
>>>>>>>>>        $q = "SELECT * FROM ".tbl::getNewConfigTable();
>>>>>>>>>
>>>>>>>>>        $data = Site::getDatabase()->sql($q,true,
>>>>>>>>> MySQLDB::TYPE_OBJECT);
>>>>>>>>>        if($data == MySQLDB::SQL_EMPTY || $data ==
>>>>>>>>> MySQLDB::SQL_ERROR)return;
>>>>>>>>>
>>>>>>>>>        foreach($data as $obj){
>>>>>>>>>
>>>>>>>>>            if($obj->name == 'version'){
>>>>>>>>>                parent::$databaseVersion = $obj->value;
>>>>>>>>>                self::runVersionCheck();
>>>>>>>>>            }
>>>>>>>>>
>>>>>>>>>            //more config variables to be stored into the
>>>>>>>>>            //config class or config.factory.class
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>        }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>    }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> and in /index.php
>>>>>>>>>
>>>>>>>>> Config::getDbConfig(); on the line after the database has been
>>>>>>>>> instantiated. (self::$database = new MySQLDB();)
>>>>>>>>>
>>>>>>>>> so:
>>>>>>>>> self::$database = new MySQLDB();
>>>>>>>>> Config::getDbConfig();
>>>>>>>>>
>>>>>>>>> I realised that the database stored can't be done when config is
>>>>>>>>> initialised as the database hasn't been connected at this point.
>>>>>>>>> There is then a reverse dependency that prevents the database from
>>>>>>>>> connecting until config has been initialised.
>>>>>>>>>
>>>>>>>>> Therefore the program flow had to be:
>>>>>>>>> 1. init Config
>>>>>>>>> 2. init Database
>>>>>>>>> 3. get database stored config
>>>>>>>>>
>>>>>>>>> What I would like to do is align this with the updates you are
>>>>>>>>> working on.  I have populated my new temporary config table with just
>>>>>>>>> ('version','1.5.3')
>>>>>>>>>
>>>>>>>>> Finally, what country are you based in scott? I'm in the UK.
>>>>>>>>>
>>>>>>>>> Cheers
>>>>>>>>> Mark
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> _____________________________________________
>>>>>>>>>
>>>>>>>>> Mob:  07725 695178
>>>>>>>>> Email: ma...@rw...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>