Menu

#19 sqlite3 performance issue

open
nobody
None
5
2010-12-26
2010-12-26
No

The performance of the SQLite3 backend is many times slower than direct access to SQLite3. This is mainly due querying the database many times for the PRAGMA table_info(SAME_TABLE) during a select statement.

Discussion

  • Holger Freyther

    Holger Freyther - 2010-12-26

    Cosmetic change, make local functions static.

     
  • Holger Freyther

    Holger Freyther - 2010-12-26

    Avoid copying the curr_type, keep the sqlite3_get_table result around longer

     
  • Holger Freyther

    Holger Freyther - 2010-12-26

    The last attachment is giving the speed increase. It implements a global cache of the last result. This is beneficial if more than one column is queried. I am not sure if DBI is making any gurantees about thread safety or such.

    A proper fix should save these values somewhere else but I don't know where this should be.

     
  • Markus Hoenicka

    Markus Hoenicka - 2011-02-20

    I've applied patch #1 which is absolutely reasonable even if it does not speed up anything.

    As far as I understand your patch #2, this is not going to help as it creates memory leaks. The for loop may go through several iterations before curr_type is non-zero. Your patch would leave all SQLite table results but the last one in memory.

    As for patch #3, it seems to be built against a fairly outdated version of dbd_sqlite3.c. Could you please rework this patch for the current cvs revision?

     

Log in to post a comment.

MongoDB Logo MongoDB