[SQL-CVS] [ sqlobject-Bugs-2009801 ] sqlobject-admin fails to process sqlite database with enums

[SourceForge.net <no...@so...>]

Bugs item #2009801, was opened at 2008-07-03 19:16
Message generated for change (Comment added) made by phd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=2009801&group_id=74338

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: SQLite
Group: SQLObject release (specify)
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Oleg Broytmann (phd)
Summary: sqlobject-admin fails to process sqlite database with enums

Initial Comment:
Seen in both 10.1 and 10.2, installed via setuptools.

The problem occurs when running a command such as...

======
sqlobject-admin record --output-dir=sqlobject-history
======

... using a sqlite database that has enum types.

The symptoms are that an infinite loop is reached and executes until an out of memory exception occurs:

======
Creating version 2008-07-03
Making directory sqlobject-history/2008-07-03
Python(3304,0xa0237fa0) malloc: *** mmap(size=1140854784) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
  File "/usr/local/bin/sqlobject-admin", line 5, in <module>
    pkg_resources.run_script('SQLObject==0.10.2', 'sqlobject-admin')
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/pkg_resources.py", line 442, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/pkg_resources.py", line 1160, in run_script
    execfile(script_filename, namespace, namespace)
  File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/EGG-INFO/scripts/sqlobject-admin", line 35, in <module>
    command.the_runner.run(sys.argv)
  File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/manager/command.py", line 101, in run
    runner.run()
  File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/manager/command.py", line 307, in run
    self.command()
  File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/manager/command.py", line 1015, in command
    diffs = db_differences(cls, conn)
  File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/manager/command.py", line 55, in db_differences
    soClass)
  File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/sqlite/sqliteconnection.py", line 286, in columnsFromSchema
    return self._columnsFromSchemaParse(tableName, soClass)
  File "/Library/Python/2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/sqlite/sqliteconnection.py", line 313, in _columnsFromSchemaParse
    colData = colData[:st] + colData[en+1:]
MemoryError
======

The problem is specifically in sqliteconnection.py, on or around line 313 with the following loop:

======
while colData.find('(') > -1: 
            st = colData.find('(')
            en = colData.find(')')
            colData = colData[:st] + colData[en+1:]
======

When this code processes colData with the following value:

======
CREATE TABLE test_table (
    id INTEGER PRIMARY KEY,
    test_enum VARCHAR (9) CHECK (test_enum in ('available', 'reserved', 'used')),
    test_string VARCHAR (64) UNIQUE
)
======

After an iteration or so, colData will have the following value (still valid):

======
    id INTEGER PRIMARY KEY,
    test_enum VARCHAR  CHECK (test_enum in ('available', 'reserved', 'used')),
    test_string VARCHAR (64) UNIQUE
======

Now comes the problem. During this iteration, st will get the index of the first open paren on the enum line, and en will get the index of the first close paren on the enum line. This is incorrect. The correct behavior is for en to get the index of the second close paren on the enum line. This incorrect behavior results in the following new value for colData:

======
    id INTEGER PRIMARY KEY,
    test_enum VARCHAR  CHECK ),
    test_string VARCHAR (64) UNIQUE
======

en from this point on continuously gets the index of that close paren left behind on the enum line, while st gets the index of the open paren on the following line... blowing up the file.

At a minimum, doing en = colData.find(')', st) will prevent it from blowing up, although this isn't technically correct behavior, since it will still leave behind that close paren.

----------------------------------------------------------------------

>Comment By: Oleg Broytmann (phd)
Date: 2008-07-24 20:09

Message:
Logged In: YES 
user_id=4799
Originator: NO

Fixed in the revisions 3525-3528 (branches 0.9, 0.10 and the trunk). Thank
you!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=540672&aid=2009801&group_id=74338