sqlalchemy-tickets Mailing List for SQLAlchemy (Page 16)
Brought to you by:
zzzeek
You can subscribe to this list here.
2006 |
Jan
|
Feb
|
Mar
(174) |
Apr
(50) |
May
(71) |
Jun
(129) |
Jul
(113) |
Aug
(141) |
Sep
(82) |
Oct
(142) |
Nov
(97) |
Dec
(72) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2007 |
Jan
(159) |
Feb
(213) |
Mar
(156) |
Apr
(151) |
May
(58) |
Jun
(166) |
Jul
(296) |
Aug
(198) |
Sep
(89) |
Oct
(133) |
Nov
(150) |
Dec
(122) |
2008 |
Jan
(144) |
Feb
(65) |
Mar
(71) |
Apr
(69) |
May
(143) |
Jun
(111) |
Jul
(113) |
Aug
(159) |
Sep
(81) |
Oct
(135) |
Nov
(107) |
Dec
(200) |
2009 |
Jan
(168) |
Feb
(109) |
Mar
(141) |
Apr
(128) |
May
(119) |
Jun
(132) |
Jul
(136) |
Aug
(154) |
Sep
(151) |
Oct
(181) |
Nov
(223) |
Dec
(169) |
2010 |
Jan
(103) |
Feb
(209) |
Mar
(201) |
Apr
(183) |
May
(134) |
Jun
(113) |
Jul
(110) |
Aug
(159) |
Sep
(138) |
Oct
(96) |
Nov
(116) |
Dec
(94) |
2011 |
Jan
(97) |
Feb
(188) |
Mar
(157) |
Apr
(158) |
May
(118) |
Jun
(102) |
Jul
(137) |
Aug
(113) |
Sep
(104) |
Oct
(108) |
Nov
(91) |
Dec
(162) |
2012 |
Jan
(189) |
Feb
(136) |
Mar
(153) |
Apr
(142) |
May
(90) |
Jun
(141) |
Jul
(67) |
Aug
(77) |
Sep
(113) |
Oct
(68) |
Nov
(101) |
Dec
(122) |
2013 |
Jan
(60) |
Feb
(77) |
Mar
(77) |
Apr
(129) |
May
(189) |
Jun
(155) |
Jul
(106) |
Aug
(123) |
Sep
(53) |
Oct
(142) |
Nov
(78) |
Dec
(102) |
2014 |
Jan
(143) |
Feb
(93) |
Mar
(35) |
Apr
(26) |
May
(27) |
Jun
(41) |
Jul
(45) |
Aug
(27) |
Sep
(37) |
Oct
(24) |
Nov
(22) |
Dec
(20) |
2015 |
Jan
(17) |
Feb
(15) |
Mar
(34) |
Apr
(55) |
May
(33) |
Jun
(31) |
Jul
(27) |
Aug
(17) |
Sep
(22) |
Oct
(26) |
Nov
(27) |
Dec
(22) |
2016 |
Jan
(20) |
Feb
(24) |
Mar
(23) |
Apr
(13) |
May
(17) |
Jun
(14) |
Jul
(31) |
Aug
(23) |
Sep
(24) |
Oct
(31) |
Nov
(23) |
Dec
(16) |
2017 |
Jan
(24) |
Feb
(20) |
Mar
(27) |
Apr
(24) |
May
(28) |
Jun
(18) |
Jul
(18) |
Aug
(23) |
Sep
(30) |
Oct
(17) |
Nov
(12) |
Dec
(12) |
2018 |
Jan
(27) |
Feb
(23) |
Mar
(13) |
Apr
(19) |
May
(21) |
Jun
(29) |
Jul
(11) |
Aug
(22) |
Sep
(14) |
Oct
(9) |
Nov
(24) |
Dec
|
From: Michael B. <iss...@bi...> - 2017-04-21 14:32:29
|
New issue 3969: negation of labeled element? https://bitbucket.org/zzzeek/sqlalchemy/issues/3969/negation-of-labeled-element Michael Bayer: this is technically something that semantically doesn't make sense, but in the "what I expect" category it does, which is to negate the inner element of a label. ``` #!python from sqlalchemy import * expr = or_(column('x') == 1, column('y') == 2) bug = not_(expr.label('foo')) from sqlalchemy.dialects import sqlite print bug.compile() print bug.compile(dialect=sqlite.dialect()) ``` ``` #! NOT x = :x_1 OR y = :y_1 x = ? OR y = ? = 0 ``` so the parenthesis are lost because we don't unwrap the label(), and we in fact hit a rule in ColumnElement._negate() where it assumes we're negating a true/false, so for a non-native-boolean it then sticks on "= 0" to the whole thing to make things worse. |
From: caijinrong <iss...@bi...> - 2017-04-18 07:45:19
|
New issue 3968: Idle in transaction connections can not be returned to pool https://bitbucket.org/zzzeek/sqlalchemy/issues/3968/idle-in-transaction-connections-can-not-be caijinrong: Hi Michael, SQLA is used in my python project. And in my code, I have the green thread used. And the green thread was killed in some of my code. After the the thread was killed, and the transaction was just began. And the database connection was still in postgresql server side with status "idle in transaction". This caused my connection was exhausted in the pool. Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers timer() File "/usr/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__ cb(*args, **kw) File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main result = function(*args, **kwargs) File "/tmp/test.py", line 48, in operate_db query = session.query(Test).all() File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2588, in all return list(self) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2736, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2749, in _execute_and_instances close_with_result=True) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2740, in _connection_from_session **kw) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 905, in connection execution_options=execution_options) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 910, in _connection_for_bind engine, execution_options) File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 334, in _connection_for_bind conn = bind.contextual_connect() File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2039, in contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2074, in _wrap_pool_connect return fn() File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 376, in connect return _ConnectionFairy._checkout(self) File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 714, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 480, in checkout rec = pool._do_get() File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1054, in _do_get (self.size(), self.overflow(), self._timeout)) TimeoutError: QueuePool limit of size 2 overflow 0 reached, connection timed out, timeout 30 Questions: 1. When will the connection returned to the pool? In my understanding, when the rollback/commit was listened, the connection was returned. 2. Will the connection returned to the pool if my green thread was killed? >From the code, we can see the connection object seems still in the pool with active status. 3. Will the connection returned to the pool if the db transaction(id in transaction) was rolled back by my database(idle_in_transaction_session_timeout). If not, how can we handle the connection whose status is in "idle in transaction"? SQLA version: 1.0.11 PostgreSQL version: 9.6 See attachment for test case. Please let me know if you need more information. Thanks. Responsible: zzzeek |
From: Michael B. <iss...@bi...> - 2017-04-17 15:52:55
|
New issue 3967: subquery eager loading doesn't use join_depth correctly https://bitbucket.org/zzzeek/sqlalchemy/issues/3967/subquery-eager-loading-doesnt-use Michael Bayer: the join_depth parameter w/ subquery eager loading causes it to just load infinitely as long as there are objects ``` #!diff diff --git a/test/orm/test_subquery_relations.py b/test/orm/test_subquery_relations.py index 1396281..2b8068d 100644 --- a/test/orm/test_subquery_relations.py +++ b/test/orm/test_subquery_relations.py @@ -517,6 +517,29 @@ class EagerTest(_fixtures.FixtureTest, testing.AssertsCompiledSQL): eq_(self.static.user_address_result, sess.query(User).order_by(User.id).all()) + def test_cyclical_explicit_join_depth(self): + """A circular eager relationship breaks the cycle with a lazy loader""" + + Address, addresses, users, User = (self.classes.Address, + self.tables.addresses, + self.tables.users, + self.classes.User) + + mapper(Address, addresses) + mapper(User, users, properties=dict( + addresses=relationship(Address, lazy='subquery', join_depth=1, + backref=sa.orm.backref( + 'user', lazy='subquery', join_depth=1), + order_by=Address.id) + )) + is_(sa.orm.class_mapper(User).get_property('addresses').lazy, + 'subquery') + is_(sa.orm.class_mapper(Address).get_property('user').lazy, 'subquery') + + sess = create_session() + eq_(self.static.user_address_result, + sess.query(User).order_by(User.id).all()) + def test_double(self): """Eager loading with two relationships simultaneously, from the same table, using aliases.""" ``` will cause a crash. similarly, the test "test_lazy_fallback_doesnt_affect_eager" doesn't make sense; the subquery loads are emitted for all levels deep when the join_depth=1. it should only eagerly load one level deep. |
From: Michael B. <iss...@bi...> - 2017-04-13 13:51:37
|
New issue 3966: ancient mysql utc_timestamp rule seems unnecessary https://bitbucket.org/zzzeek/sqlalchemy/issues/3966/ancient-mysql-utc_timestamp-rule-seems Michael Bayer: from https://github.com/zzzeek/sqlalchemy/pull/355 ``` #! commit a21dc3d Author: Mike Bayer mi...@zz... Date: Wed Sep 10 21:09:04 2008 +0000 - the function func.utc_timestamp() compiles to UTC_TIMESTAMP, without the parenthesis, which seem to get in the way when using in conjunction with executemany(). ``` we have a hardcoded UTC_TIMESTAMP render that seems unnecessary and gets in the way of parameters like fsp. the ancient assertion that this "gets in the way with executemany()" must be tested. It would only be some ancient mysqldb driver issue at best. |
From: Olivier Le M. <iss...@bi...> - 2017-04-13 13:23:13
|
New issue 3965: Bug with hybrid property ? https://bitbucket.org/zzzeek/sqlalchemy/issues/3965/bug-with-hybrid-property Olivier Le Moign: I use SQLALchemy ORM with a declarative base and SQLITE. I use hybrid properties to encrypt TOTP secrets as follows: ``` #!python class FactorsTOTP(Model): app_secret = 'foobar' user_id = Column(Integer, ForeignKey('user.id'), nullable=False) _key = Column('key', String, nullable=False) @hybrid_property def key(self): cipher_key = urlsafe_b64encode(self.app_secret.ljust(32, b'0')) f = Fernet(cipher_key) return f.decrypt(self._key.encode('utf-8')) @key.setter def key(self, value): cipher_key = urlsafe_b64encode(self.app_secret.ljust(32, b'0')) f = Fernet(cipher_key) self._key = f.encrypt(value.encode('utf-8')).decode('ascii') ``` My problem is, when creating a row: factors_totp = models.FactorsTOTP(key=totp_key) I'm not sure why, but the key getter is being called after the setter and fails on self._key.encode('utf-8'). Simple Python operations work (self._key + 'suffix'), but not calling a function (bytes(self.key, encoding='utf-8) fails as well). ``` #!python Traceback (most recent call last): File "/Users/olemoign/.pyenv/versions/rta/bin/rta_add_admin", line 11, in <module> load_entry_point('rta', 'console_scripts', 'rta_add_admin')() File "/Users/olemoign/Developer/Parsys/rta/rta/scripts/add_admin.py", line 27, in main password, totp_key = _command_line_add_user(env['request'], admin) File "/Users/olemoign/Developer/Parsys/rta/rta/services/users.py", line 363, in _command_line_add_user factors_totp = models.FactorsTOTP(key=totp_key) File "<string>", line 4, in __init__ File "/Users/olemoign/.pyenv/versions/3.5.2/envs/rta/lib/python3.5/site-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance manager.dispatch.init_failure(self, args, kwargs) File "/Users/olemoign/.pyenv/versions/3.5.2/envs/rta/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ compat.reraise(exc_type, exc_value, exc_tb) File "/Users/olemoign/.pyenv/versions/3.5.2/envs/rta/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise raise value File "/Users/olemoign/.pyenv/versions/3.5.2/envs/rta/lib/python3.5/site-packages/sqlalchemy/orm/state.py", line 411, in _initialize_instance return manager.original_init(*mixed[1:], **kwargs) File "/Users/olemoign/.pyenv/versions/3.5.2/envs/rta/lib/python3.5/site-packages/sqlalchemy/ext/declarative/base.py", line 653, in _declarative_constructor (k, cls_.__name__)) TypeError: 'key' is an invalid keyword argument for FactorsTOTP ``` This isn't so bad, as I can do factors_totp = models.FactorsTOTP() factors_totp.key = totp_key but I just wanted to raise the issue. Thank you for your great library. |
From: Michael B. <iss...@bi...> - 2017-04-11 13:56:45
|
New issue 3964: sqltypes.ARRAY does not link to postgresql.ARRAY (?!?!) https://bitbucket.org/zzzeek/sqlalchemy/issues/3964/sqltypesarray-does-not-link-to Michael Bayer: the bind/ result processors of postgresql.ARRAY are not invoked with the sqltypes.ARRAY. I'm not sure how the ARRAY enhancement got committed with an omission like this. |
From: Michael B. <iss...@bi...> - 2017-04-10 18:14:32
|
New issue 3963: aliased vs. non-aliased specificity in loader options on lazy load https://bitbucket.org/zzzeek/sqlalchemy/issues/3963/aliased-vs-non-aliased-specificity-in Michael Bayer: all of these cases emit eager load for B.cs when they are specified as lazyload: ``` #!python from sqlalchemy import * from sqlalchemy.orm import * from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() class A(Base): __tablename__ = 'a' id = Column(Integer, primary_key=True) bs = relationship("B") class B(Base): __tablename__ = 'b' id = Column(Integer, primary_key=True) a_id = Column(ForeignKey('a.id')) cs = relationship("C") class C(Base): __tablename__ = 'c' id = Column(Integer, primary_key=True) b_id = Column(ForeignKey('b.id')) e = create_engine("sqlite://", echo=True) Base.metadata.create_all(e) s = Session(e) s.add(A(bs=[B(cs=[C()])])) s.add(A()) s.commit() s.close() aa = aliased(A) q = s.query(aa, A).filter(aa.id == 1).filter(A.id == 2).options(joinedload(A.bs).joinedload(B.cs)) for a, real_a in q: for b in a.bs: # was eager loaded, when should lazy load for c in b.cs: print c s.close() print "--------------------------------" aa = aliased(A) q = s.query(aa, A).filter(aa.id == 1).filter(A.id == 2).options(defaultload(A.bs).joinedload(B.cs)) for a, real_a in q: for b in a.bs: # was eager loaded, when should lazy load for c in b.cs: print c s.close() print "--------------------------------" aa = aliased(A) q = s.query(A, aa).filter(A.id == 1).filter(aa.id == 2).options(defaultload(aa.bs).joinedload(B.cs)) for a, real_a in q: for b in a.bs: # was eager loaded, when should lazy load for c in b.cs: print c ``` _chop_path is important here, this fixes: ``` #!diff diff --git a/lib/sqlalchemy/orm/strategy_options.py b/lib/sqlalchemy/orm/strategy_options.py index bae2b73..60ed2be 100644 --- a/lib/sqlalchemy/orm/strategy_options.py +++ b/lib/sqlalchemy/orm/strategy_options.py @@ -376,7 +376,8 @@ class _UnboundLoad(Load): _WILDCARD_TOKEN,) and c_token != p_prop.key: return None elif isinstance(c_token, PropComparator): - if c_token.property is not p_prop: + if c_token.property is not p_prop or \ + c_token._parententity is not p_mapper: return None else: i += 1 ``` |
From: Ramin <iss...@bi...> - 2017-04-10 10:11:53
|
New issue 3962: Unsupported SQLAlchemy of mssql instance https://bitbucket.org/zzzeek/sqlalchemy/issues/3962/unsupported-sqlalchemy-of-mssql-instance Ramin: Hi, i going to connect of SQLAlchemy to other mssql instance exm: `<ip>\SQLInstance` but i see your [docs](http://docs.sqlalchemy.org/en/latest/dialects/mssql.html#module-sqlalchemy.dialects.mssql.pymssql) SQAlchemy default connect to `.` instance, default instance code : ```python from sqlalchemy import create_engine engine = create_engine('mssql+pymssql://<user>:<pass>@<ip>:1433/database_name') from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() from sqlalchemy import Column, Integer, String class teeetttt(Base): __tablename__ = 'testtetetet' id = Column(Integer, primary_key=True) name = Column(String) fullname = Column(String) password = Column(String) print teeetttt.__table__ ed_user = teeetttt(name='ed', fullname='Ed Jones', password='edspassword') from sqlalchemy.orm import sessionmaker Session = sessionmaker(bind=engine) Session = sessionmaker() Session.configure(bind=engine) session = Session() session.add(ed_user) session.commit() print ed_user.name session.query(teeetttt).all() f = session.query(teeetttt).all() print f[0].name print session.query(teeetttt).all() ``` my goals connect to ```engine=create_engine('mssql+adodbapi://sa:a`123456@172.16.20.181:1433\SQLInstance/DB')``` that `SQLInstance` this is my instance and DB is database name Responsible: zzzeek |
From: Michael B. <iss...@bi...> - 2017-04-06 00:59:39
|
New issue 3961: mysql partition options need to be after all the table options https://bitbucket.org/zzzeek/sqlalchemy/issues/3961/mysql-partition-options-need-to-be-after Michael Bayer: e.g. https://dev.mysql.com/doc/refman/5.7/en/create-table.html |
From: Mischa <iss...@bi...> - 2017-04-05 21:30:13
|
New issue 3960: Declarative backref not filtering by polymorphic identity https://bitbucket.org/zzzeek/sqlalchemy/issues/3960/declarative-backref-not-filtering-by Mischa: I am trying to declare a backref relationship between a related table and a single-table polymorphic subclass. I expect the relationship to filter by the polymorphic discriminator column when I select the relationship, but instead it selects ALL related rows, regardless of polymorphic discriminator identity. This is confusing, and does not appear to be documented in http://docs.sqlalchemy.org/en/latest/orm/extensions/declarative/inheritance.html#declarative-inheritance I think this is a common use case, but I see very little documentation in the declarative inheritance relationship documentation. Please let me know if I'm missing something. Attached is a test case that fails to discriminate when selecting the related rows via backref. SQL logs are at: https://0bin.net/paste/P2VTuIUsIDyZ9vw4#zItNB6zFxnNm1usjqzqflA4eJ+JqGELuUq8uIm2pf2i |
From: lv <iss...@bi...> - 2017-04-05 15:48:36
|
New issue 3959: postgresql interval restricted types https://bitbucket.org/zzzeek/sqlalchemy/issues/3959/postgresql-interval-restricted-types lv: as [mentioned Ilja Everilä](http://stackoverflow.com/a/43231662) according to [this code](https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/dialects/postgresql/base.py#L1303), there is only 'interval': INTERVAL, 'interval year to month': INTERVAL, 'interval day to second': INTERVAL, and no [others](https://www.postgresql.org/docs/9.6/static/datatype-datetime.html) YEAR MONTH DAY HOUR MINUTE SECOND YEAR TO MONTH DAY TO HOUR DAY TO MINUTE DAY TO SECOND HOUR TO MINUTE HOUR TO SECOND MINUTE TO SECOND types of interval ? found [460 ticket](https://bitbucket.org/zzzeek/sqlalchemy/issues/460/postgresql-interval-type) but there only two additional types 'interval year to month', 'interval day to second' |
From: anti-social <iss...@bi...> - 2017-04-04 16:19:24
|
New issue 3958: Sqlalchemy 1.1 doesn't remove duplicate rows in some cases while 1.0 does https://bitbucket.org/zzzeek/sqlalchemy/issues/3958/sqlalchemy-11-doesnt-remove-duplicate-rows anti-social: After the [commit](https://bitbucket.org/zzzeek/sqlalchemy/commits/ceeb033054f09db3eccbde3fad1941ec42919a54) next test case begun to fail: ``` #!python class FilterUniqueJoinTest(fixtures.MappedTest, AssertsCompiledSQL): run_setup_mappers = 'once' @classmethod def define_tables(cls, metadata): Table('users', metadata, Column('id', Integer, primary_key=True, test_needs_autoincrement=True), Column('name', String(50)), Column('date_created', DateTime, default=datetime.datetime.now)) Table('orders', metadata, Column('id', Integer, primary_key=True, test_needs_autoincrement=True), Column('user_id', Integer, ForeignKey('users.id'))) @classmethod def setup_classes(cls): users = cls.tables.users orders = cls.tables.orders class User(cls.Comparable): pass class Order(cls.Comparable): pass mapper(User, users, properties={ 'orders': relationship(Order, order_by=users.c.id.desc()) }) mapper(Order, orders) @classmethod def insert_data(cls): User = cls.classes.User Order = cls.classes.Order sess = create_session() o1 = Order() o2 = Order() o3 = Order() u1 = User(name='Mike') u1.orders = [o1, o2] u2 = User(name='John') u2.orders = [o3] sess.add(u1) sess.add(u2) sess.flush() sess.close() def test_filter_unique_column(self): User = self.classes.User sess = create_session() users = ( sess.query( User.date_created.label('dt'), User, ) .options(joinedload(User.orders)) .all() ) assert len(users) == 2 def test_filter_unique_func_column(self): User = self.classes.User sess = create_session() users = ( sess.query( func.date(User.date_created, 'start of month').label('month'), User, ) .options(joinedload(User.orders)) .all() ) assert len(users) == 2 ``` Sqlalchemy 1.0: ``` #!text test/orm/test_joins.py::FilterUniqueJoinTest::test_filter_unique_column PASSED test/orm/test_joins.py::FilterUniqueJoinTest::test_filter_unique_func_column PASSED ``` Sqlalchemy 1.1: ``` #!text test/orm/test_joins.py::FilterUniqueJoinTest::test_filter_unique_column PASSED test/orm/test_joins.py::FilterUniqueJoinTest::test_filter_unique_func_column FAILED ``` Is it expected behaviour? |
From: Michael B. <iss...@bi...> - 2017-04-04 13:48:42
|
New issue 3957: incorrect rendering of sqltext for column-level check constraint https://bitbucket.org/zzzeek/sqlalchemy/issues/3957/incorrect-rendering-of-sqltext-for-column Michael Bayer: ``` #!python from sqlalchemy import * from sqlalchemy.sql.expression import ColumnClause class CodenameConstraint(CheckConstraint): """Check that the column uses a limited alphabet.""" def __init__(self, column_name): column = ColumnClause(column_name) super(CodenameConstraint, self).__init__( column.op("~")("^[A-Za-z0-9_-]+$")) m = MetaData() t = Table( 't', m, Column('x', String, CodenameConstraint('x')), ) from sqlalchemy.schema import CreateTable print CreateTable(t) ``` a string compiler is being invoked separately, needs this: ``` #!diff diff --git a/lib/sqlalchemy/sql/compiler.py b/lib/sqlalchemy/sql/compiler.py index 4415028..b18f903 100644 --- a/lib/sqlalchemy/sql/compiler.py +++ b/lib/sqlalchemy/sql/compiler.py @@ -2602,7 +2602,9 @@ class DDLCompiler(Compiled): formatted_name = self.preparer.format_constraint(constraint) if formatted_name is not None: text += "CONSTRAINT %s " % formatted_name - text += "CHECK (%s)" % constraint.sqltext + text += "CHECK (%s)" % self.sql_compiler.process(constraint.sqltext, + include_table=False, + literal_binds=True) text += self.define_constraint_deferrability(constraint) return text ``` been this way for years so 1.2 to allow for any surprises. |
From: Marcin K. <iss...@bi...> - 2017-04-04 09:27:38
|
New issue 3956: Mutable + column properties broken https://bitbucket.org/zzzeek/sqlalchemy/issues/3956/mutable-column-properties-broken Marcin Kurczewski: Trying to use both mutable and column properties within the same table results in mapper exception. The stacktrace is: ``` #!text Traceback (most recent call last): File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 682, in __getattr__ return getattr(self.comparator, key) AttributeError: 'Comparator' object has no attribute 'info' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/srv/www/tmp.sakuya.pl/f/test", line 25, in <module> session.query(User).all() File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1328, in query return self._query_cls(entities, self, **kwargs) File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 139, in __init__ self._set_entities(entities) File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 150, in _set_entities self._set_entity_selectables(self._entities) File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 180, in _set_entity_selectables ent.setup_entity(*d[entity]) File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3595, in setup_entity self._with_polymorphic = ext_info.with_polymorphic_mappers File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 764, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/orm/mapper.py", line 1948, in _with_polymorphic_mappers configure_mappers() File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/orm/mapper.py", line 2869, in configure_mappers mapper, mapper.class_) File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/event/attr.py", line 218, in __call__ fn(*args, **kw) File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/orm/events.py", line 616, in wrap fn(*arg, **kw) File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/ext/mutable.py", line 603, in listen_for_type prop.columns[0].info.get('_ext_mutable_orig_type') File "/home/rr-/.local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 688, in __getattr__ key) AttributeError: Neither 'Label' object nor 'Comparator' object has an attribute 'info' ``` The code works fine on 1.1.7 and doesn't on 1.1.8. The GitHub hash of the change that introduces this behavior is 07b63894cb8ff9529b406f196b5d7cb9af209e9e: "Track SchemaEventTarget types in as_mutable()". I attach a code sample that triggers this behavior. |
From: Michael B. <iss...@bi...> - 2017-04-03 21:05:22
|
New issue 3955: connectionless execution + autocommit + ON CONFLICT DO NOTHING + no rows == boom https://bitbucket.org/zzzeek/sqlalchemy/issues/3955/connectionless-execution-autocommit-on Michael Bayer: ``` #!diff @@ -96,6 +97,33 @@ class OnConflictTest(fixtures.TablesTest): [(1, 'name1')] ) + def test_on_conflict_do_nothing_connectionless(self): + users = self.tables.users_xtra + + with testing.db.connect() as conn: + result = conn.execute( + insert(users).on_conflict_do_nothing( + constraint='uq_login_email'), + + dict(name='name1', login_email='email1') + ) + eq_(result.inserted_primary_key, [1]) + eq_(result.returned_defaults, (1,)) + + result = testing.db.execute( + insert(users).on_conflict_do_nothing( + constraint='uq_login_email' + ), + dict(name='name2', login_email='email1') + ) + eq_(result.inserted_primary_key, None) + eq_(result.returned_defaults, None) + + eq_( + testing.db.execute(users.select().where(users.c.id == 1)).fetchall(), + [(1, 'name1', 'email1', None)] + ) + @testing.provide_metadata def test_on_conflict_do_nothing_target(self): users = self.tables.users ``` the lack of row returned sends the connection into autoclose via result.fetchone(). then autocommit causes an exception, then it continues to be confused because the connection has been closed. |
From: Michael B. <iss...@bi...> - 2017-04-03 18:02:01
|
New issue 3954: mainstream the "baked" lazyloader https://bitbucket.org/zzzeek/sqlalchemy/issues/3954/mainstream-the-baked-lazyloader Michael Bayer: looking to use "baked" for the two "IN" loaders, this should be mainstreamed in LazyLoader as well. no issues have been reported over the past year so it's time to see if this really holds up. |
From: Michael B. <iss...@bi...> - 2017-04-03 17:01:05
|
New issue 3953: expanding parameter https://bitbucket.org/zzzeek/sqlalchemy/issues/3953/expanding-parameter Michael Bayer: this is for the purpose of IN() in conjunction with query caching. Idea is to allow this to be viable with "select in" loading and "select in" inheritance so that these are compatible w/ "baked". stmt = select([table]).where(table.c.col.in_(bindparam('foo', expanding=True)) conn.execute(stmt, {"foo": [1, 2, 3]}) the string statement is converted at ExecutionContext time with an extra search and replace. this so that all the caching steps can be used against the above query w/ a dynamic list for IN. |
From: Isaku Y. <iss...@bi...> - 2017-04-01 01:24:05
|
New issue 3952: with_variant with dialect type and parameters doesn't work when query. https://bitbucket.org/zzzeek/sqlalchemy/issues/3952/with_variant-with-dialect-type-and Isaku Yamahata: The following code snipet doesn't work as expected from sqlalchemy 1.1.5 The cultprit is the change set of df9b6492e5ca47e26d539d2283fa816a2d5c8ad6 change-id of I7b7b45d604a4ae8d1dc236a5a1248695aab5232e https://bitbucket.org/zzzeek/sqlalchemy/commits/df9b6492e5ca import sqlalchemy as sa from sqlalchemy.dialects import sqlite class Data(Base): created_at = sa.Column( sa.DateTime().with_variant( sqlite.DATETIME(truncate_microseconds=True), 'sqlite'), server_default=sa.func.now()) # insert row and query based on created_at doesn't work as expected. row = Data() session.add(row) session.flush() rows = sessio.query(data.created_at).all() row = rows[0] # query by created_at found = sessoin.squery(Data).filer(Data.created_at == row.created_at).all() found should include added row. but after the change set pointed above, found is empty list. by setting engine.echo=True, the following was gotten. SELECT Data.created_at AS Data_created_at WHERE Data.created_at = ? 2017-03-31 17:42:51,951 INFO sqlalchemy.engine.base.Engine.sqlite@vgluzzatoz ('2017-04-01 00:42:45.000000',) ('2017-04-01 00:42:45.000000',) please notice the it includes microseconds as ".000000" even with truncate_microseconds=True sqlite.DATETIME(truncate_micorseconds=True) should be used when bind_processor() is called, but plain DateTime is used. This is because of type_api.Variant.coerce_compared_value ignores dialects. |
From: Andrew P. R. <iss...@bi...> - 2017-03-31 21:35:39
|
New issue 3951: PostgreSQL Sequence not created by metadata.create_all() from declarative mixin https://bitbucket.org/zzzeek/sqlalchemy/issues/3951/postgresql-sequence-not-created-by Andrew Paulo Robillo: I'd been copying this [snippet](https://bitbucket.org/snippets/shimofuri/bALAz) from my old cookbook to add a surrogate key to a declarative table class: ``` #!python class MixinIntKey(object): @declared_attr def key(cls): return Column( pg.INTEGER, Sequence(f"sqk_{cls.__tablename__}", 1, 1), nullable=False, primary_key=True ) ``` This code was working before (some 3-4 years ago, I think around version 0.8 ): it generates the Sequence when I build the database objects through metadata.create_all(). The only change I made to the snippet was to update it to use format strings (Python 3.6). After some googling around and testing (see attached file/[link](https://bitbucket.org/snippets/shimofuri/bALAz)), the Sequence can be generated only if the metadata is passed during Sequence init: ``` #!python Sequence(f"sqk_{cls.__tablename__}", 1, 1, metadata=metadata) ``` My environment: psycopg2==2.7.1, SQLAlchemy==1.1.8, Python 3.6 64bit on Windows 8.1 64bit, PostgreSQL 9.6 server |
From: Michael B. <iss...@bi...> - 2017-03-30 21:19:37
|
New issue 3950: 1.1 regression due to TypeDecorator copy() from Column.copy(), interferes with mutable https://bitbucket.org/zzzeek/sqlalchemy/issues/3950/11-regression-due-to-typedecorator-copy Michael Bayer: this is a bug that exists since sqlalchemy.ext.mutable started, however it only is apparent when the type object is copied. The copy is uncommon except in the case of SchemaEventTarget, which TypeDecorator becomes in 1.1. ``` #!python from sqlalchemy import * from sqlalchemy.orm import * from sqlalchemy.ext.declarative import declarative_base import json from sqlalchemy.ext import mutable Base = declarative_base() class MyType(TypeDecorator): impl = Text def process_bind_param(self, value, dialect): if value is not None: value = json.dumps(value) return value def process_result_value(self, value, dialect): if value is not None: value = json.loads(value) return value data = Column(mutable.MutableDict.as_mutable(MyType)) class A(Base): __tablename__ = 'a' id = Column(Integer, primary_key=True) # works fine # data = data # fails in 1.1 because TypeDecorator is a SchemaType, # so Column copies the type, so the event we attached # to scan mapper columns can't identify that this is the "data" # we copied data = data.copy() # as happens in declarative e = create_engine("sqlite://", echo=True) Base.metadata.create_all(e) s = Session(e) a1 = A(data={"foo": "bar"}) s.add(a1) s.commit() a1.data["foo"] = "bat" s.commit() assert a1.data["foo"] == "bat" ``` |
From: Jorge S. <iss...@bi...> - 2017-03-29 01:13:04
|
New issue 3949: sqlalchemy compiler.py bug https://bitbucket.org/zzzeek/sqlalchemy/issues/3949/sqlalchemy-compilerpy-bug Jorge Sierra: conn_str = string.Template('db2+ibm_db://$user:$password@localhost:50000/Sample').substitute(user = user , password = password) for t in metadata.sorted_tables: logger.info ("CreateTable %s %s" % (t.name, **CreateTable**(t).compile(db2))) When trying to run **CreateTable** C:\Python27\Lib\site-packages\sqlalchemy\sql\**compiler.py** around line 2588 under function def **visit_foreign_key_constraint**(self, constraint): this function is assuming **f.parent** is never 'None' and in my case it was, same as **f.column** This is the fix below ``` #!python list_text1 = [] for f in constraint.elements: if f.parent is not None: list_text1.append(preparer.quote(f.parent.name)) list_text2 = [] for f in constraint.elements: if hasattr(f, 'column'): if f.column is not None: list_text2.append(preparer.quote(f.column.name)) text += "FOREIGN KEY(%s) REFERENCES %s (%s)" % ( ', '.join(list_text1), self.define_constraint_remote_table( constraint, remote_table, preparer), ', '.join(list_text2)) text += self.define_constraint_match(constraint) text += self.define_constraint_cascades(constraint) text += self.define_constraint_deferrability(constraint) return text ``` |
From: Michael B. <iss...@bi...> - 2017-03-28 14:49:22
|
New issue 3948: post-load of joined-inh subtables https://bitbucket.org/zzzeek/sqlalchemy/issues/3948/post-load-of-joined-inh-subtables Michael Bayer: using the post_load architecture of #3944, add `post_with_polymorphic` argument to mapper. Subclasses will be loaded using a query like query(Subclass).options(defer(everything on superclass), noload(everything on superclass)), as opposed to a custom SELECT statement the way the lazy load works, so that relationship loaders and other loader options on the subclass take effect normally. |
From: Michael B. <iss...@bi...> - 2017-03-26 00:19:58
|
New issue 3947: race condition in joined eager load perf enhancement https://bitbucket.org/zzzeek/sqlalchemy/issues/3947/race-condition-in-joined-eager-load-perf Michael Bayer: e.g. #3915 test: ``` #!python from sqlalchemy import * from sqlalchemy.orm import * from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() class A(Base): __tablename__ = 'a' id = Column(Integer, primary_key=True) bs = relationship("B") class B(Base): __tablename__ = 'b' id = Column(Integer, primary_key=True) a_id = Column(ForeignKey('a.id')) e = create_engine("sqlite:///foo.db", echo=False) Base.metadata.create_all(e) import threading import random import time import traceback def work(e): s = Session(e) while True: try: s.query(A).options(joinedload(A.bs)).all() except Exception as e: traceback.print_exc() print "Exception: %s" % e finally: time.sleep(random.random()) def chaos(): while True: time.sleep(.001) A.bs.property._get_strategy((("lazy", "joined"),))._aliased_class_pool[:] = [] s = Session(e) s.add_all([A(bs=[B(), B(), B()])]) s.commit() threads = [threading.Thread(target=work, args=(e, )) for i in range(20)] chaos = threading.Thread(target=chaos) for t in threads: t.start() chaos.start() ``` traceback ``` #! Traceback (most recent call last): File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/query.py", line 2679, in all File "test.py", line 31, in work s.query(A).options(joinedload(A.bs)).all() File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/query.py", line 2679, in all yes return list(self) return list(self) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/loading.py", line 90, in instances File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/query.py", line 2831, in __iter__ return self._execute_and_instances(context) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/query.py", line 2854, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 945, in execute util.raise_from_cause(err) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/loading.py", line 57, in instances for query_entity in query._entities File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/query.py", line 3686, in row_processor polymorphic_discriminator=self._polymorphic_discriminator File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/loading.py", line 330, in _instance_processor context, path, mapper, result, adapter, populators) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/interfaces.py", line 532, in create_row_processor return meth(self, multiparams, params) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection mapper, result, adapter, populators) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/strategies.py", line 1594, in create_row_processor eager_adapter) return connection._execute_clauseelement(self, multiparams, params) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/loading.py", line 318, in _instance_processor File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement col = adapter.columns[col] compiled_sql, distilled_params File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/sql/util.py", line 713, in __getitem__ File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 1189, in _execute_context context) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception exc_info File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 1182, in _execute_context context) self.parent.include_fn and not self.parent.include_fn(key) File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/orm/util.py", line 326, in _include_fn File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/engine/default.py", line 504, in do_execute entity = elem._annotations.get('parentmapper', None) AttributeError: 'NoneType' object has no attribute '_annotations' Exception: 'NoneType' object has no attribute '_annotations' cursor.execute(statement, parameters) ``` |
From: Daniel E. <iss...@bi...> - 2017-03-24 21:30:36
|
New issue 3946: ROLLBACK outside of transactions https://bitbucket.org/zzzeek/sqlalchemy/issues/3946/rollback-outside-of-transactions Daniel Eisner: Hi, If I try to run a bad "SELECT" statement without starting a transaction, SQLAlchemy tries to roll back the transaction, which fails, and then raises a ProgrammingError execption, obscuring the original exception. I think the problem is with this bit of code in sqlalchemy.engine.base: def _autorollback(self): if not self._root.in_transaction(): self._root._rollback_impl() That logic looks backwards to me. Was this the original intent? Below is an example stack trace from a case like this. --------------------------------- Traceback: /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/pandas/io/sql.pyc in read_sql(sql, con, index_col, coerce_float, params, parse_dates, columns, chunksize) 492 sql, index_col=index_col, params=params, 493 coerce_float=coerce_float, parse_dates=parse_dates, --> 494 chunksize=chunksize) 495 496 /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/pandas/io/sql.pyc in read_query(self, sql, index_col, coerce_float, parse_dates, params, chunksize) 1126 args = _convert_params(sql, params) 1127 -> 1128 result = self.execute(*args) 1129 columns = result.keys() 1130 /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/pandas/io/sql.pyc in execute(self, *args, **kwargs) 1020 def execute(self, *args, **kwargs): 1021 """Simple passthrough to SQLAlchemy engine""" -> 1022 return self.engine.execute(*args, **kwargs) 1023 1024 def read_table(self, table_name, index_col=None, coerce_float=True, /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in execute(self, statement, *multiparams, **params) 2053 2054 connection = self.contextual_connect(close_with_result=True) -> 2055 return connection.execute(statement, *multiparams, **params) 2056 2057 def scalar(self, statement, *multiparams, **params): /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in execute(self, object, *multiparams, **params) 937 """ 938 if isinstance(object, util.string_types[0]): --> 939 return self._execute_text(object, multiparams, params) 940 try: 941 meth = object._execute_on_connection /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in _execute_text(self, statement, multiparams, params) 1095 statement, 1096 parameters, -> 1097 statement, parameters 1098 ) 1099 if self._has_events or self.engine._has_events: /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in _execute_context(self, dialect, constructor, statement, parameters, *args) 1187 parameters, 1188 cursor, -> 1189 context) 1190 1191 if self._has_events or self.engine._has_events: /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in _handle_dbapi_exception(self, e, statement, parameters, cursor, context) 1384 if cursor: 1385 self._safe_close_cursor(cursor) -> 1386 self._autorollback() 1387 1388 if newraise: /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in _autorollback(self) 822 def _autorollback(self): 823 if not self._root.in_transaction(): --> 824 self._root._rollback_impl() 825 826 def close(self): /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in _rollback_impl(self) 701 self.engine.dialect.do_rollback(self.connection) 702 except BaseException as e: --> 703 self._handle_dbapi_exception(e, None, None, None, None) 704 finally: 705 if not self.__invalid and \ /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in _handle_dbapi_exception(self, e, statement, parameters, cursor, context) 1313 self.dialect.dbapi.Error, 1314 dialect=self.dialect), -> 1315 exc_info 1316 ) 1317 self._reentrant_error = True /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/util/compat.pyc in raise_from_cause(exception, exc_info) 201 exc_type, exc_value, exc_tb = exc_info 202 cause = exc_value if exc_value is not exception else None --> 203 reraise(type(exception), exception, tb=exc_tb, cause=cause) 204 205 if py3k: /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/base.pyc in _rollback_impl(self) 699 self.engine.logger.info("ROLLBACK") 700 try: --> 701 self.engine.dialect.do_rollback(self.connection) 702 except BaseException as e: 703 self._handle_dbapi_exception(e, None, None, None, None) /home/developer/.conda/envs/core_runtime/lib/python2.7/site-packages/sqlalchemy/engine/default.pyc in do_rollback(self, dbapi_connection) 438 439 def do_rollback(self, dbapi_connection): --> 440 dbapi_connection.rollback() 441 442 def do_commit(self, dbapi_connection): ProgrammingError: (pyodbc.ProgrammingError) ('25000', '[25000] [FreeTDS][SQL Server]The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION. (3903) (SQLEndTran)') |
From: Charles Bouchard-L. <iss...@bi...> - 2017-03-24 18:08:02
|
New issue 3945: Imported classes confuses intersphinx https://bitbucket.org/zzzeek/sqlalchemy/issues/3945/imported-classes-confuses-intersphinx Charles Bouchard-Légaré: Sphinx (using [intersphinx](http://www.sphinx-doc.org/en/stable/ext/intersphinx.html)) can't find some classes from your objects.inv, resulting in See the [objects.inv](http://docs.sqlalchemy.org/en/latest/objects.inv) ```text WARNING: py:class reference target not found: sqlalchemy.ext.declarative.api.Schema ``` This can be worked around by assigning the `__module__` property of classes imported in any `__init__.py`. See also http://stackoverflow.com/a/24787344 |