Re: [libdbi-users] Found and fixed: dbd_mysql thread unsafe problem

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Markus,
Unfortunately, I have no experience maintaining code for packages.  The
best I can do is a few lines in a Makefile, and I don't understand
automake scripts at all. I can fix all sorts of things in c source
code, but I would likely break the test and build process. So I
wouldn't trust me as a maintainer.
Thanks for your work on libdbi over the years.  I have been using it
for well over a decade.
Ethan...
On Tue, 2019-11-19 at 00:20 +0100, Markus Hoenicka wrote:
> Hi Ethan and anyone else who might be listening,
> I'm sorry for the sluggish response time. I'm afraid I'm the one
> you're looking for. I used to maintain libdbi and the drivers for a
> couple of years while I was able to do some programming. I continued
> as sort of a caretaker for another while even though there was no
> time left for pretty much anything outside my dayjob. I still use
> libdbi everyday but I'd have to read the manual how to check in a
> patch these days. So long story short - if anyone is willing to take
> over development, or at least to act as a caretaker, please step
> forward. If you have the skills to check in and test your patch, let
> me know. I can add you as a developer to the project.
> regards,Markus
> On 2019-11-16 21:09, Ethan Funk was heard to say:
> > I found and tested a fix for the mysql thread safety problem I
> > havebeen troubleshooting.
> > Commenting out line 154 of dbd_mysql.c succeeds in fixing the
> > problemwhen libdbi-mysql is build against recent libmysqlclient
> > libraries.This is the were mysql_server_end() is called after each
> > dbi instanceif finished. In a multi-threaded application, multiple
> > dbi instancescan come and go in various threads. With recent
> > libmysqlclient libraryimplementations, mysql_server_end() should
> > only be called ONCE, at theend of the entire program/process, not
> > multiple times when a thread isfinished with a dbi intsnace.
> > libmysqlclient's mysql_server_end() function performs some
> > finallibmysqlclient memory clean-up. So if it isn't called, and
> > theapplication ends, it's not really a problem - the OS will clean
> > up thememory allocation.
> > A few other thread safety notes that might be helpful to others. To
> > bethread safe:
> > 1. Each program thread must use it's own libdbi instance
> > (asdocumented in libdbi).
> > 2. The dbi_conn_connect function must be protected by a mutex,
> > sinceit results in an underlying libmysqlclient call that is not
> > threadsafe, using memory objects allocated once by the
> > libmysqlclientlibrary when it was first initialized. This is
> > exactly the memory thatwas causing trouble (freed) when
> > mysql_server_end() is called bylibdbi_shutdown_r in onw thread when
> > the libdbi library is still inuse by other threads prior to program
> > termination.
> > So... is libdbi and the database drivers still maintained by
> > someone?How can I get this fix included so it will be present when
> > folksinstall the driver for mysql on their various systems?
> > Thanks,Ethan...
> > On Fri, 2019-11-15 at 09:35 -0700, Ethan Funk wrote:
> > > I think I found the problem in the mysql dbd driver code.:
> > > dbd_finalize() calls mysql_server_end(), which is deprecated in
> > > the
> > > current mysqlclient21 library, but results in a call to
> > > areplacement
> > > function which is intended to be called only once, at the end of
> > > a
> > > process.  With the call being made inside dbd_finalize(), the
> > > mysql_server_end() gets called each time my application
> > > codefinished
> > > with a libdbi instance.  Reading between the lines in the MySQL
> > > documentation, it looks think the mysql_server_end() call to
> > > it'snew
> > > counter part frees memory which is allocated by a pthread_once()
> > > function triggered by mysql_library_init(), which itself is
> > > calledby
> > > mysql_init() via the libdbi driver on each connection
> > > attempt.  Somy
> > > code's first libdbi connection results in the memory
> > > beingallocated,
> > > with subsequent calls in other threads being ignored.  But
> > > dbd_finalize() gets called at the end of EVERY libdbi
> > > instanceclosure,
> > > so the memory if freed over and over again.  I don't see a
> > > mechanism
> > > for a once-per-process driver shutdown callback in libdbi, so I
> > > wantto
> > > try commenting out mysql_servefr_end() in dbd_finalize().  This
> > > will
> > > leak memory which is never freed at the end of the process, but
> > > so
> > > what.  It is only allocated once per process, so the
> > > operatingsystem
> > > process shutdown will free the memory.
> > > Unfortunately, I can't seem to build my modified code for
> > > testingwith
> > > the libdi_dbd project code. "./configure" is giving me the
> > > following
> > > error:
> > > configure: error: Invalid libdbi directory - include files
> > > notfound.
> > > This is after configure has already indicated that yes, it
> > > haslocated
> > > the dbi header files, which I installed with the libdbi-dev
> > > package
> > > with the Ubuntu apt command.
> > > How might I be able to fix this error, and proceed with testing
> > > mycode
> > > fix?
> > > Thanks,
> > > Ethan...
> > > On Wed, 2019-11-13 at 14:19 -0700, Ethan Funk wrote:
> > > > I am having a problem with libdbd_mysql on linux (Ubuntu 19.10)
> > > > reagding thread fasety.
> > > > There used to be two versions of libmysqlclient, one was
> > > > threadsafe,
> > > > one was not. However,
> > > > now there seems to be only one client library, which I
> > > > understoodwas
> > > > thread safe: libmysqlclient.so.21.
> > > > Any insight into the dump below would be greatly appreciated!
> > > > Ethan...
> > > > Here is a valgrind dump of a read-already-freed error that
> > > > occursin
> > > > what I will call thread #3
> > > > when it attempts to connect to MySQL.  This thread has it's
> > > > ownlibdbi
> > > > instance, yet mysql_real_connect
> > > > is trying to read memory allocated by the first thread (thread
> > > > #1)that
> > > > first called mysql_real_connect,
> > > > after that same memory was freed by another thread (thread
> > > > #2)that
> > > > called mysql_server_end as it was
> > > > finishing. Each thread has it's own libdbi instance, so why is
> > > > libmysqlclient sharring memory between
> > > > threads? It's looking like the libmysqlclient isn't thread
> > > > safe.
> > > > ==2828== Invalid read of size 1
> > > > ==2828==    at 0x4A3A042: strlen (vg_replace_strmem.c:461)
> > > > ==2828==    by 0x141A46C1: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A5468: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A54A8: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A373E: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x4FE041E: __pthread_once_slow
> > > > (pthread_once.c:116)
> > > > ==2828==    by 0x141A320D: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A41B2: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A429E: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x1415472B: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x1415492A: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141577A8: mysql_real_connect (in
> > > > /usr/lib/x86_64-
> > > > linux-gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x12F47435: dbd_connect (in /usr/lib/x86_64-
> > > > linux-
> > > > gnu/dbd/libdbdmysql.so)
> > > > ==2828==    by 0x4DC37AE: dbi_conn_connect (in/usr/lib/x86_64-
> > > > linux-
> > > > gnu/libdbi.so.1.1.0)
> > > > ==2828==    by 0x134BFA: dbSetupConnection (database.c:211)
> > > > ==2828==    by 0x1357F6: DeleteLogEntry (database.c:440)
> > > > ==2828==    by 0x131ECE: taskCallProcInThread (tasks.c:73)
> > > > ==2828==    by 0x4FD7668: start_thread (pthread_create.c:479)
> > > > ==2828==    by 0x567B322: clone (clone.S:95)
> > > > Memory was freed in a thread (lets call it thread #2) that ran
> > > > and
> > > > completed. It created it's own
> > > > dbi instance, did it's thing, and then closed it's
> > > > MySQLconnection and
> > > > shutdown the instance.
> > > > So why is it freeing memory deep in the mysql driver that it
> > > > didnot
> > > > allocate?
> > > > ==2828==  Address 0x1343e698 is 424 bytes inside a block of
> > > > size4,088
> > > > free'd
> > > > ==2828==    at 0x4A37F90: free (vg_replace_malloc.c:540)
> > > > ==2828==    by 0x141A9B8A: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A8756: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x14148982: mysql_server_end (in/usr/lib/x86_64-
> > > > linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x12F4684C: dbd_finalize (in /usr/lib/x86_64-
> > > > linux-
> > > > gnu/dbd/libdbdmysql.so)
> > > > ==2828==    by 0x4DC2E4D: dbi_shutdown_r (in/usr/lib/x86_64-
> > > > linux-
> > > > gnu/libdbi.so.1.1.0)
> > > > ==2828==    by 0x134440: dbi_instance_free (database.c:56)
> > > > ==2828==    by 0x4FD6600: __nptl_deallocate_tsd.part.0
> > > > (pthread_create.c:301)
> > > > ==2828==    by 0x4FD7689:
> > > > __nptl_deallocate_tsd(pthread_create.c:256)
> > > > ==2828==    by 0x4FD7689: start_thread (pthread_create.c:490)
> > > > ==2828==    by 0x567B322: clone (clone.S:95)
> > > > This is in the first thread in the program to make use
> > > > oflibdbi/MySQL,
> > > > and it is still
> > > > running/connected to MySQL using a unique libdbi intance.
> > > > ==2828==  Block was alloc'd at
> > > > ==2828==    at 0x4A36E96: malloc (vg_replace_malloc.c:309)
> > > > ==2828==    by 0x141A9A30: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A9AEB: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A4E1C: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x1419CFA0: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x1419D95E: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x1416FD29: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A2A66: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A3771: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x4FE041E: __pthread_once_slow
> > > > (pthread_once.c:116)
> > > > ==2828==    by 0x141A320D: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A41B2: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141A429E: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x1415472B: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x1415492A: ??? (in /usr/lib/x86_64-linux-
> > > > gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x141577A8: mysql_real_connect (in
> > > > /usr/lib/x86_64-
> > > > linux-gnu/libmysqlclient.so.21.1.17)
> > > > ==2828==    by 0x12F47435: dbd_connect (in /usr/lib/x86_64-
> > > > linux-
> > > > gnu/dbd/libdbdmysql.so)
> > > > ==2828==    by 0x4DC37AE: dbi_conn_connect (in/usr/lib/x86_64-
> > > > linux-
> > > > gnu/libdbi.so.1.1.0)
> > > > ==2828==    by 0x134BFA: dbSetupConnection (database.c:211)
> > > > ==2828==    by 0x13D2F4: GetItemMetaData (database.c:2960)
> > > > ==2828==
> > _______________________________________________libdbi-users mailing
> > lis...@li...
> > https://lists.sourceforge.net/lists/listinfo/libdbi-users