[Sqlalchemy-tickets] Issue #3864: Alert user about shortcoming on executemany for psycopg2 (zzzeek/sqlalchemy)

[charles p. <iss...@bi...>]

New issue 3864: Alert user about shortcoming on executemany for psycopg2
https://bitbucket.org/zzzeek/sqlalchemy/issues/3864/alert-user-about-shortcoming-on

charles pierre:

Hello,

Thanks for the fantastic job you did on this library. 

I recently came across a shortcoming of the actual psycopg2 library: apparently the executemany function is not optimized at all and just do a loop over execute.  

In regard to sqlalchemy, this mean that your recommended way of doing massive insert/update statements is actually really slow when working with psycopg2.  

For me, the difference between using execute(insert(), ..) or bulk_insert_mappings (which in the end, just execute a tons of single INSERT statement) and doing .insert().values() (which construct the correct INSERT ... VALUES), was like day and night.

Maybe you could add a warning or note in your documentation regarding the use of psycopg2 in the relevant section of your docs (for instance, here, http://docs.sqlalchemy.org/en/latest/faq/performance.html#i-m-inserting-400-000-rows-with-the-orm-and-it-s-really-slow) ?

Responsible: zzzeek