MySQL for Python

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).

Hrm, thought I marked this pending for 1.2.3b2...

I tried it out on 1.2br. It builds under MacPorts, which is awesome.
Better yet, it no longer leaks! Two thumbs way up :-D

Ok, the patch is applied to both 1.2br and trunk. Will mark closed once
1.2.3b2 is released.

JJ - the attached patch should fix the "holy crap it leaks like a sieve"
situation. Please test it, and I'll get Andy to put this in 1.2.3b2.

I now have a working repro! I have some leads on the leak, but nothing
definite yet. Thanks jj.

Sure:

CREATE DATABASE development
DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;

...

CREATE TABLE atoms (
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,

active TINYINT NOT NULL DEFAULT 1,

url TEXT NOT NULL,
name VARCHAR(255),
blurb VARCHAR(255),
thumbnail TEXT,

UNIQUE INDEX (url(255)),
INDEX (name)
) ENGINE = INNODB;

jj, can you give me the complete CREATE TABLE for the "atoms" table
referenced by your script? I have a dummy table in my test db and it
doesn't leak. I'm mostly interested in the CHARSET and ENGINE you're
using.

I've made SVN trunk kind-of work again... Some partially finished work on
conversions and encoding is present, so trunk isn't that stable right now.

I tried out MySQL-python-1.2.3b1 on Ubuntu 8.04 with Python 2.5. It still
leaks.

Ugh, trying to build MySQLdb on MacPorts is *hard*. I'm going to have a
hard time testing 1.2br.

> jj, are you still seeing this? I can't repro it. I've tried on trunk
and
1.2br, and my python process stays the exact same size (on Fedora 10).

Hey, Kyle!

Does trunk actually take care of encodings? The last time I tried it, it
didn't
https://sourceforge.net/tracker/index.php?func=detail&aid=2127852&group_id=22307&atid=374932.
Of course, that was a long time ago.

I haven't tried 1.2br. I'll try that next.

Thanks for the update :)

> charset='utf8', use_unicode=False

Markus, thanks for the tip. You're right that setting use_unicode=False
fixes the memory leak. However, if I set that then MySQLdb gives me UTF-8
strings instead of unicode objects. I'm not using SQLAlchemy, so I'd
really much prefer MySQLdb to give me unicode objects.

(Note, I use SQLAlchemy when I have a little bit of very complex data, and
I use naked SQL when I have a lot of very simple data. In my current
project, I have only three tables, but I have 10s of gigs of data.)

jj, are you still seeing this? I can't repro it. I've tried on trunk and
1.2br, and my python process stays the exact same size (on Fedora 10).

> # This program leaks memory rather quickly. Removing the charset
> # parameter fixes it.

Using the parameters

charset='utf8', use_unicode=False

also shows no leaking and should therefore be used as a temporary
workaround for 1.2.2 as mentioned at
http://www.sqlalchemy.org/trac/wiki/DatabaseNotes#MySQL

Any idea what the change was in trunk that fixed this?

Sorry, I'm going to reopen this; I was the one who originally closed it.
It turns out that MySQLdb 1.3 on trunk doesn't actually respect
charset='utf8'. See
http://sourceforge.net/tracker/index.php?func=detail&aid=2127852&group_id=22307&atid=374932.
Hence, it's very well possible that MySQLdb might leak memory again after
that bug is fixed.

Duh, it looks like this has already been fixed in trunk. I was running it
wrong earlier. It definitely leaks in 1.2.2, but it does not appear to
leak in trunk. Closing and marking as invalid.

File Added: mysqlleak.py