Thread: [SQLObject] bug in sqlobject-admin
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: Imri G. <lor...@gm...> - 2010-04-17 11:28:26
|
Heya I've worked a bit with sqlobject-admin, and there's a bug in the implementation of the status command. Steps to reproduce: 1. Create a simple table with a decimal column, for example: class MyThing(sqlobject.SQLObject): bla1 = sqlobject.DecimalCol(size = 10, precision = 2) 2. Use sqlobject-admin create --egg=... -c sqlite://.... 3. Use sqlobject-admin status --egg=... -c sqlite://... You should get the following exception: [snip] File "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ sqlobject\manager\command.py", line 765, in command col = col.withClass(soClass) File "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ sqlobject\col.py", line 408, in withClass **self._kw) File "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ sqlobject\col.py", line 1329, in __init__ "You must give a size argument" AssertionError: You must give a size argument After researching this issue a bit, I still haven't figured out whether this is the result of a bug in the command implementation, or the way that keyword arguments are handled in the column class hierarchy. Cheers, Imri -- Imri Goldberg -------------------------------------- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -------------------------------------- -- insert signature here ---- |
From: Oleg B. <ph...@ph...> - 2010-04-19 09:58:48
|
On Sat, Apr 17, 2010 at 01:25:59PM +0300, Imri Goldberg wrote: > I've worked a bit with sqlobject-admin, and there's a bug in the > implementation of the status command. Thank you for reporting. I have never used sqlobject-admin so I have to learn how to use it to reproduce the bug. Oleg. -- Oleg Broytman http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Imri G. <lor...@gm...> - 2010-04-19 13:11:10
|
Heya I'll be happy to help you work it, and also to solve the bug. Re using sqlobject-admin: I'm curious - what is your preferred way of managing db migrations (i.e. versioning schema changes)? Cheers, Imri On Mon, Apr 19, 2010 at 12:58 PM, Oleg Broytman <ph...@ph...> wrote: > On Sat, Apr 17, 2010 at 01:25:59PM +0300, Imri Goldberg wrote: > > I've worked a bit with sqlobject-admin, and there's a bug in the > > implementation of the status command. > > Thank you for reporting. I have never used sqlobject-admin so I have to > learn how to use it to reproduce the bug. > > Oleg. > -- > Oleg Broytman http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > sqlobject-discuss mailing list > sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss > -- Imri Goldberg -------------------------------------- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -------------------------------------- -- insert signature here ---- |
From: Oleg B. <ph...@ph...> - 2010-04-20 11:45:42
|
On Mon, Apr 19, 2010 at 04:11:09PM +0300, Imri Goldberg wrote: > Re using sqlobject-admin: I'm curious - what is your preferred way of > managing db migrations (i.e. versioning schema changes)? I seldom do a simple ADD/DELETE COLUMN. Usually when I change the schema it's more like "collect some info, ADD COLUMN, put info into the new column". Something like this (code from a real script upgradedb5.py): log.open("upgradedb5.log") if db_conn.dbName == 'sqlite': db_conn.use_table_info = True if 'userID' not in ExportResult.sqlmeta.columns: ExportResult.sqlmeta.addColumn(ForeignKey('User', name="user", default=None), changeSchema=True) for result in ExportResult.select(ExportResult.q.plate <> None): result.user = result.plate.user commit() log.close() This is a rather simple example. Sometime there is a lot of code to collect information before schema changing and even more code that puts the collected information into the new columns. Oleg. -- Oleg Broytman http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Oleg B. <ph...@ph...> - 2010-04-20 16:23:50
|
On Sat, Apr 17, 2010 at 01:25:59PM +0300, Imri Goldberg wrote: > 1. Create a simple table with a decimal column, for example: > > class MyThing(sqlobject.SQLObject): > bla1 = sqlobject.DecimalCol(size = 10, precision = 2) > > 2. Use sqlobject-admin create --egg=... -c sqlite://.... > 3. Use sqlobject-admin status --egg=... -c sqlite://... [skip] > File > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > sqlobject\manager\command.py", line 765, in command > col = col.withClass(soClass) > File > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > sqlobject\col.py", line 408, in withClass > **self._kw) > File > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > sqlobject\col.py", line 1329, in __init__ > "You must give a size argument" > AssertionError: You must give a size argument > > After researching this issue a bit, I still haven't figured out whether this > is the result of a bug in the command implementation, or the way that > keyword arguments are handled in the column class hierarchy. Ok, found the cause. CommandStatus in sqlobject/manager/command.py list columns using columnsFromScheme(), a kind of forced fromDatabase=True. And most connection classes don't recognize size/precision in DecimalCol'umns. In this particular case it would be enough, I think, to extend columnsFromScheme to pass None's as size/precision. I will do some experiments before committing the hack. Oleg. -- Oleg Broytman http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Oleg B. <ph...@ph...> - 2010-04-20 18:00:15
|
On Tue, Apr 20, 2010 at 08:23:39PM +0400, Oleg Broytman wrote: > In this particular case it would be enough, I think, to extend > columnsFromScheme to pass None's as size/precision. I will do some > experiments before committing the hack. Fixed and committed in the revisions 4175-4177 (branches 0.11, 0.12 and the trunk). Will be in the next release. Oleg. -- Oleg Broytman http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Imri G. <lor...@gm...> - 2010-04-20 18:15:48
|
Excellent, thanks a bunch! Cheers, Imri PS, if there is some work to be done on sqlobject, I'll be happy to help :) On Tue, Apr 20, 2010 at 9:00 PM, Oleg Broytman <ph...@ph...> wrote: > On Tue, Apr 20, 2010 at 08:23:39PM +0400, Oleg Broytman wrote: > > In this particular case it would be enough, I think, to extend > > columnsFromScheme to pass None's as size/precision. I will do some > > experiments before committing the hack. > > Fixed and committed in the revisions 4175-4177 (branches 0.11, 0.12 and > the trunk). Will be in the next release. > > Oleg. > -- > Oleg Broytman http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > -- Imri Goldberg -------------------------------------- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -------------------------------------- -- insert signature here ---- |
From: Imri G. <lor...@gm...> - 2010-04-20 18:00:21
|
Heya When I checked it, it seemed to me to happen because the 'size' and 'precision' were pop()-ed from the original **kw dict passed to DecimalCol. Since the this dict is later kept as a member, when used again by CommandStatus, 'size' and 'precision' are no longer there. Hope that helps, Imri On Tue, Apr 20, 2010 at 7:23 PM, Oleg Broytman <ph...@ph...> wrote: > On Sat, Apr 17, 2010 at 01:25:59PM +0300, Imri Goldberg wrote: > > 1. Create a simple table with a decimal column, for example: > > > > class MyThing(sqlobject.SQLObject): > > bla1 = sqlobject.DecimalCol(size = 10, precision = 2) > > > > 2. Use sqlobject-admin create --egg=... -c sqlite://.... > > 3. Use sqlobject-admin status --egg=... -c sqlite://... > [skip] > > File > > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > > sqlobject\manager\command.py", line 765, in command > > col = col.withClass(soClass) > > File > > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > > sqlobject\col.py", line 408, in withClass > > **self._kw) > > File > > "D:\applic\program\python25\lib\site-packages\sqlobject-0.12.1-py2.5.egg\ > > sqlobject\col.py", line 1329, in __init__ > > "You must give a size argument" > > AssertionError: You must give a size argument > > > > After researching this issue a bit, I still haven't figured out whether > this > > is the result of a bug in the command implementation, or the way that > > keyword arguments are handled in the column class hierarchy. > > Ok, found the cause. CommandStatus in sqlobject/manager/command.py list > columns using columnsFromScheme(), a kind of forced fromDatabase=True. And > most connection classes don't recognize size/precision in DecimalCol'umns. > In this particular case it would be enough, I think, to extend > columnsFromScheme to pass None's as size/precision. I will do some > experiments before committing the hack. > > Oleg. > -- > Oleg Broytman http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > -- Imri Goldberg -------------------------------------- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -------------------------------------- -- insert signature here ---- |
From: Oleg B. <ph...@ph...> - 2010-04-20 18:12:59
|
On Tue, Apr 20, 2010 at 09:00:11PM +0300, Imri Goldberg wrote: > When I checked it, it seemed to me to happen because the 'size' and > 'precision' were pop()-ed from the original **kw dict passed to DecimalCol. The problem was that the dictionary was simply empty. SQLiteConnection.guessClass() did at line 364: elif t.find('DECIMAL') >= 0: return col.DecimalCol, {} I changed that to return col.DecimalCol, {'size': None, 'precision': None} With the change your test works for me. BTW, DecimalCol is only recognized by SQLiteConnection. Oleg. -- Oleg Broytman http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Imri G. <lor...@gm...> - 2010-04-20 18:42:21
|
On Tue, Apr 20, 2010 at 9:12 PM, Oleg Broytman <ph...@ph...> wrote: > > BTW, DecimalCol is only recognized by SQLiteConnection. > > Oleg. > -- > Oleg Broytman http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > That's strange, as I currently use mysql. I haven't given it much thought, but it seems to work well. -- Imri Goldberg -------------------------------------- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -------------------------------------- -- insert signature here ---- |
From: Oleg B. <ph...@ph...> - 2010-04-20 19:13:01
|
On Tue, Apr 20, 2010 at 09:35:12PM +0300, Imri Goldberg wrote: > On Tue, Apr 20, 2010 at 9:12 PM, Oleg Broytman <ph...@ph...> wrote: > > BTW, DecimalCol is only recognized by SQLiteConnection. > > That's strange, as I currently use mysql. I haven't given it much thought, > but it seems to work well. If columnsFromSchema() couldn't recognize column it simply returns an instance of the generic Col class. Oleg. -- Oleg Broytman http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Imri G. <lor...@gm...> - 2010-04-26 18:13:09
|
Hi Oleg! I am considering switching to your way of doing schema changes. I would very much appreciate more information regarding the process you use to do these things. For instance, given that I have a working MyThing class, to which I'd like to add a field my_field - would you write the upgrade script first, and then copy the field definitions to the MyThing class? Would you do it the other way around? Once both are updated, do you have a special order of doing things when moving it to production? Thanks ahead, Imri On Tue, Apr 20, 2010 at 2:45 PM, Oleg Broytman <ph...@ph...> wrote: > On Mon, Apr 19, 2010 at 04:11:09PM +0300, Imri Goldberg wrote: > > Re using sqlobject-admin: I'm curious - what is your preferred way of > > managing db migrations (i.e. versioning schema changes)? > > I seldom do a simple ADD/DELETE COLUMN. Usually when I change the schema > it's more like "collect some info, ADD COLUMN, put info into the new > column". > Something like this (code from a real script upgradedb5.py): > > log.open("upgradedb5.log") > > if db_conn.dbName == 'sqlite': > db_conn.use_table_info = True > > if 'userID' not in ExportResult.sqlmeta.columns: > ExportResult.sqlmeta.addColumn(ForeignKey('User', name="user", > default=None), > changeSchema=True) > > for result in ExportResult.select(ExportResult.q.plate <> None): > result.user = result.plate.user > > commit() > > log.close() > > This is a rather simple example. Sometime there is a lot of code to > collect information before schema changing and even more code that puts the > collected information into the new columns. > > Oleg. > -- > Oleg Broytman http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > sqlobject-discuss mailing list > sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss > -- Imri Goldberg -------------------------------------- http://plnnr.com/ - automatic trip planning http://www.algorithm.co.il/blogs/ -------------------------------------- -- insert signature here ---- |
From: Oleg B. <ph...@ph...> - 2010-04-26 19:00:59
|
Hi! On Mon, Apr 26, 2010 at 09:13:01PM +0300, Imri Goldberg wrote: > I am considering switching to your way of doing schema changes. > I would very much appreciate more information regarding the process you use > to do these things. > > For instance, given that I have a working MyThing class, to which I'd like > to add a field my_field - would you write the upgrade script first, and then > copy the field definitions to the MyThing class? Would you do it the other > way around? > Once both are updated, do you have a special order of doing things when > moving it to production? I use SQLObject in commercial programs that my company sells to customers. So I have to distribute new class descriptions; so new tables and columns are added to the code before upgrade scripts. Also I write upgrade scripts in such a way they can be run safely many times. When I announce new build in the internal mailing list and at the corporation's wiki I notify support department there is a new upgrade script. I don't expect support persons remember what upgrade scripts had been run for a particular customer; so when installing a new version a support person can run all upgrade scripts in order. Because of this an upgrade script always looks into the real database to see if it needs to do its job. For a new tables it's quite simple: from Database.Requests import RequestIDs if not RequestIDs.tableExists(): RequestIDs.createTable() I.e., the script imports the class declaration from the Database modules, tests if the table exists and creates the table. For a new column it's a bit harder - I have to investigate the DB deeper. Sometimes I simply create a second class declaration and connect it to the same table: class OldRequestIDs(Base): class sqlmeta: table = 'request_i_ds' # connect the table to the real table status = StringCol(length=255, default=None) OldRequestIDs.sqlmeta.delColumn("status", changeSchema=True) Sometimes I draw the table declaration from the DB: class Plate(SQLObject): class sqlmeta: fromDatabase = True if 'idPl' not in Plate.sqlmeta.columns: Plate.sqlmeta.addColumn(IntCol("IDpl", default=None, dbName="id_pl"), changeSchema=True) (Wow, three different ways of naming 'id_pl' column, damn!) Oleg. -- Oleg Broytman http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |