[Sqlalchemy-tickets] Issue #3552: Exception when pickle.dumps(DeclarativeObject) with sqlalchemy.ex
Brought to you by:
zzzeek
|
From: dwt <iss...@bi...> - 2015-10-12 14:55:52
|
New issue 3552: Exception when pickle.dumps(DeclarativeObject) with sqlalchemy.ext.mutable.Mutable field https://bitbucket.org/zzzeek/sqlalchemy/issues/3552/exception-when-pickledumps dwt: Hi there, today we noticed an explosion when we tried to pickle.dumps one of our model objects with an error like this: ``` pickle.PicklingError: Can't pickle <function remove at 0x107afcb90>: it's not found as weakref.remove ``` Googling we found this: https://groups.google.com/forum/#!topic/sqlalchemy/b3eoWwhKriI which lead us to reduce the issue to this sample code which seems to be pretty minimal in showing the error: ```python #!/usr/bin/env python import json from sqlalchemy import types # Basically stolen from # http://docs.sqlalchemy.org/en/rel_0_8/core/types.html#marshal-json-strings class JSONEncodedDict(types.TypeDecorator): """Represents an immutable structure as a json-encoded string. Usage:: JSONEncodedDict(255) Made mutation aware by the MutableDict below! Defaults to an empty dict if read as NULL from the DB for migration friendliness. """ impl = types.VARCHAR python_type = dict def process_bind_param(self, value, dialect): if value is not None: value = json.dumps(value) return value def process_result_value(self, value, dialect): if value is not None: value = json.loads(value, object_hook=dict) else: value = {} return value from sqlalchemy.ext.mutable import Mutable class MutableDict(Mutable, dict): @classmethod def coerce(cls, key, value): "Convert plain dictionaries to MutableDict." if not isinstance(value, MutableDict): if isinstance(value, dict): return MutableDict(value) # this call will raise ValueError return Mutable.coerce(key, value) else: return value def __setitem__(self, key, value): "Detect dictionary set events and emit change events." dict.__setitem__(self, key, value) self.changed() def __delitem__(self, key): "Detect dictionary del events and emit change events." dict.__delitem__(self, key) self.changed() MutableDict.associate_with(JSONEncodedDict) from sqlalchemy.ext.declarative import declarative_base Base = declarative_base(constructor=None) from sqlalchemy import Column, Integer class User(Base): __tablename__ = 'user' id = Column("iduser", Integer, primary_key=True) settings = Column(JSONEncodedDict(4096), default=dict) from sqlalchemy import create_engine from sqlalchemy.orm import scoped_session, sessionmaker engine = create_engine('sqlite://') DBSession = scoped_session(sessionmaker()) DBSession.configure(bind=engine) Base.metadata.bind = engine Base.metadata.create_all() import pickle user = User() DBSession.add(user) DBSession.flush() pickle.dumps(user) print('Worked') ``` This still errors in the latest version from pypi (1.0.8). What can you make from this? It looks like an error to us - but maybe we're using mutable wrong? |