#241 sqlobject-admin fails to process sqlite database with enums

closed-fixed
SQLite (24)
5
2008-07-24
2008-07-03
Anonymous
No

Seen in both 10.1 and 10.2, installed via setuptools.

The problem occurs when running a command such as...

======
sqlobject-admin record --output-dir=sqlobject-history
======

... using a sqlite database that has enum types.

The symptoms are that an infinite loop is reached and executes until an out of memory exception occurs:

======
Creating version 2008-07-03
Making directory sqlobject-history/2008-07-03
Python(3304,0xa0237fa0) malloc: *** mmap(size=1140854784) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
File "/usr/local/bin/sqlobject-admin", line 5, in <module>
pkg_resources.run_script('SQLObject==0.10.2', 'sqlobject-admin')
File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/pkg_resources.py", line 442, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/pkg_resources.py", line 1160, in run_script
execfile(script_filename, namespace, namespace)
File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/EGG-INFO/scripts/sqlobject-admin", line 35, in <module>
command.the_runner.run(sys.argv)
File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/manager/command.py", line 101, in run
runner.run()
File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/manager/command.py", line 307, in run
self.command()
File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/manager/command.py", line 1015, in command
diffs = db_differences(cls, conn)
File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/manager/command.py", line 55, in db_differences
soClass)
File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/sqlite/sqliteconnection.py", line 286, in columnsFromSchema
return self._columnsFromSchemaParse(tableName, soClass)
File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/sqlite/sqliteconnection.py", line 313, in _columnsFromSchemaParse
colData = colData[:st] + colData[en+1:]
MemoryError
======

The problem is specifically in sqliteconnection.py, on or around line 313 with the following loop:

======
while colData.find('(') > -1:
st = colData.find('(')
en = colData.find(')')
colData = colData[:st] + colData[en+1:]
======

When this code processes colData with the following value:

======
CREATE TABLE test_table (
id INTEGER PRIMARY KEY,
test_enum VARCHAR (9) CHECK (test_enum in ('available', 'reserved', 'used')),
test_string VARCHAR (64) UNIQUE
)
======

After an iteration or so, colData will have the following value (still valid):

======
id INTEGER PRIMARY KEY,
test_enum VARCHAR CHECK (test_enum in ('available', 'reserved', 'used')),
test_string VARCHAR (64) UNIQUE
======

Now comes the problem. During this iteration, st will get the index of the first open paren on the enum line, and en will get the index of the first close paren on the enum line. This is incorrect. The correct behavior is for en to get the index of the second close paren on the enum line. This incorrect behavior results in the following new value for colData:

======
id INTEGER PRIMARY KEY,
test_enum VARCHAR CHECK ),
test_string VARCHAR (64) UNIQUE
======

en from this point on continuously gets the index of that close paren left behind on the enum line, while st gets the index of the open paren on the following line... blowing up the file.

At a minimum, doing en = colData.find(')', st) will prevent it from blowing up, although this isn't technically correct behavior, since it will still leave behind that close paren.

Discussion

  • Oleg Broytman

    Oleg Broytman - 2008-07-24

    Logged In: YES
    user_id=4799
    Originator: NO

    Fixed in the revisions 3525-3528 (branches 0.9, 0.10 and the trunk). Thank you!

     
  • Oleg Broytman

    Oleg Broytman - 2008-07-24
    • assigned_to: nobody --> phd
    • status: open --> closed-fixed
     

Log in to post a comment.