[Sqlalchemy-tickets] Issue #4030: Very weird `KeyError` exception (zzzeek/sqlalchemy)

[ddzialak <iss...@bi...>]

New issue 4030: Very weird `KeyError` exception
https://bitbucket.org/zzzeek/sqlalchemy/issues/4030/very-weird-keyerror-exception

ddzialak:

```
Traceback (most recent call last):
  File "site-packages/cherrypy/_cprequest.py", line 670, in respond
  File "site-packages/cherrypy/lib/encoding.py", line 221, in __call__
  File "site-packages/cherrypy/_cpdispatch.py", line 60, in __call__
  File "sfutils/perf_stats.py", line 206, in wrapper
  File "sfutils/base_service.py", line 461, in wrapper
  File "sfutils/base_service.py", line 395, in call_f_and_return_json
  File "sfvolumes/service.py", line 260, in request_save_volume
  File "sfvolumes/service.py", line 229, in _save_volume
  File "sfutils/perf_stats.py", line 206, in wrapper
  File "sfvolumes/dao.py", line 206, in get_volume_by_name
  File "sfvolumes/dao.py", line 190, in _get_volume_by_name
  File "site-packages/sqlalchemy/orm/query.py", line 2814, in one
  File "site-packages/sqlalchemy/orm/query.py", line 2784, in one_or_none
  File "site-packages/sqlalchemy/orm/loading.py", line 90, in instances
  File "site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
  File "site-packages/sqlalchemy/orm/loading.py", line 75, in instances
  File "site-packages/sqlalchemy/orm/loading.py", line 385, in _instance
  File "site-packages/sqlalchemy/orm/identity.py", line 151, in get
KeyError: (<class 'sfutils.pg_api.alchemy_schema.Volume'>, (2L,))
```
It's very weird because code within `orm/identity.py` looks like:
```
[148]    def get(self, key, default=None):
[149]        if key not in self._dict:
[150]            return default
[151]        state = self._dict[key]
```

First time I've though that `session` is used in two threads but session is stored only within `dao` and dao is a local parameter so it's not possible that more than one thread is using the same session.
The other strange thing  is backtrace - can't explain why `raise_from_cause` (from util.py) invoked `instances` (from loading.py).
I've saw that `weakref` is used so is it possible that default `_identity_cls = identity.WeakInstanceDict` from session caused that fired garbage collector freed some objects and removed some key from `self._dict` within identity_map? (just while thread was between `identity.py:149` and `identity.py:151`)?

We use sqlalchemy very intensively and that problem happened three times since last four months (so it seems to be very rare case).

Versions:
SQLAlchemy (1.1.10)
psycopg2 (2.7.1)
postgresql   9.6+184.pgdg14.04+1
python 2.7.12

Unfortunately can't create any Minimal, Complete and Verifiable Example as it seems to be very rare case.

Responsible: zzzeek