|
From: Jiří Č. <ji...@ci...> - 2016-09-26 10:02:58
|
Hi *, I've had quite a few occurrences of transaction being stuck on Firebird 2.5.6 in last 7 days. Even trying to close the connection from monitoring tables was not working. I had to kill the process. I was able to take a dump (actually 2, from different processes), but given it's a production system with real data, I would rather not upload it publicly. It's 2.5.6.27020, 64bit, Classic. It happened in 2.5.5 as well. Bellow are the call stacks of threads in fb_inet_server (with PDB for Firebird loaded). I suppose the first thread here is the key. Looks like it's spinning in "while (owner->own_ast_count)" forever (at least couple of hours before I found out, because of a lot of deadlock timeouts in SQL). Anybody interested in the dumps? ntdll.dll!00007ffbe6a7151a() KERNELBASE.dll!00007ffbe3f5121a() fb_inet_server.exe!Jrd::LockManager::shutdownOwner(Jrd::thread_db * tdbb, long * owner_offset) Line 450 fb_inet_server.exe!shutdown_database(Jrd::Database * dbb, const bool release_pools) Line 5950 fb_inet_server.exe!purge_attachment(Jrd::thread_db * tdbb, Jrd::Attachment * attachment, const bool force_flag) Line 6391 fb_inet_server.exe!jrd8_detach_database(__int64 * user_status, Jrd::Attachment * * handle) Line 2459 fb_inet_server.exe!detach_or_drop_database(__int64 * user_status, unsigned int * handle, const int proc, const __int64 specCode) Line 2262 fb_inet_server.exe!EDS::IscConnection::doDetach(Jrd::thread_db * tdbb) Line 189 fb_inet_server.exe!EDS::Connection::detach(Jrd::thread_db * tdbb) Line 524 fb_inet_server.exe!EDS::Connection::deleteConnection(Jrd::thread_db * tdbb, EDS::Connection * conn) Line 310 fb_inet_server.exe!EDS::Provider::releaseConnection(Jrd::thread_db * tdbb, EDS::Connection & conn, bool __formal) Line 249 fb_inet_server.exe!EDS::Connection::deleteTransaction(EDS::Transaction * tran) Line 413 fb_inet_server.exe!EDS::Transaction::commit(Jrd::thread_db * tdbb, bool retain) Line 670 fb_inet_server.exe!EDS::Transaction::jrdTransactionEnd(Jrd::thread_db * tdbb, Jrd::jrd_tra * transaction, bool commit, bool retain, bool force) Line 765 fb_inet_server.exe!TRA_commit(Jrd::thread_db * tdbb, Jrd::jrd_tra * transaction, const bool retaining_flag) Line 418 fb_inet_server.exe!commit(Jrd::thread_db * tdbb, Jrd::jrd_tra * transaction, const bool retaining_flag) Line 4595 fb_inet_server.exe!JRD_commit_transaction(Jrd::thread_db * tdbb, Jrd::jrd_tra * * transaction) Line 7034 fb_inet_server.exe!jrd8_commit_transaction(__int64 * user_status, Jrd::jrd_tra * * tra_handle) Line 1801 fb_inet_server.exe!isc_commit_transaction(__int64 * user_status, unsigned int * tra_handle) Line 1749 fb_inet_server.exe!rem_port::end_transaction(P_OP operation, p_rlse * release, packet * sendL) Line 2107 fb_inet_server.exe!process_packet(rem_port * port, packet * sendL, packet * receive, rem_port * * result) Line 3442 fb_inet_server.exe!loopThread(void * __formal) Line 5266 fb_inet_server.exe!`anonymous namespace'::threadStart(void * arg) Line 139 msvcr80.dll!0000000068b437d7() msvcr80.dll!0000000068b43894() kernel32.dll!00007ffbe68a13d2() ntdll.dll!00007ffbe69f5454() ntdll.dll!00007ffbe6a7121a() KERNELBASE.dll!00007ffbe3f51118() fb_inet_server.exe!Jrd::ConfigStorage::touchThreadFunc() Line 352 fb_inet_server.exe!Jrd::ConfigStorage::touchThread(void * arg) Line 339 fb_inet_server.exe!`anonymous namespace'::threadStart(void * arg) Line 139 msvcr80.dll!0000000068b437d7() msvcr80.dll!0000000068b43894() kernel32.dll!00007ffbe68a13d2() ntdll.dll!00007ffbe69f5454() ntdll.dll!00007ffbe6a7121a() KERNELBASE.dll!00007ffbe3f51118() fb_inet_server.exe!ISC_event_wait(event_t * event, const long value, const long micro_seconds) Line 2012 fb_inet_server.exe!Jrd::LockManager::blocking_action_thread() Line 1581 fb_inet_server.exe!Jrd::LockManager::blocking_action_thread(void * arg) Line 469 fb_inet_server.exe!`anonymous namespace'::threadStart(void * arg) Line 139 msvcr80.dll!0000000068b437d7() msvcr80.dll!0000000068b43894() kernel32.dll!00007ffbe68a13d2() ntdll.dll!00007ffbe69f5454() ntdll.dll!00007ffbe6a7121a() KERNELBASE.dll!00007ffbe3f51118() fb_inet_server.exe!ISC_event_wait(event_t * event, const long value, const long micro_seconds) Line 2012 fb_inet_server.exe!Jrd::LockManager::blocking_action_thread() Line 1581 fb_inet_server.exe!Jrd::LockManager::blocking_action_thread(void * arg) Line 469 fb_inet_server.exe!`anonymous namespace'::threadStart(void * arg) Line 139 msvcr80.dll!0000000068b437d7() msvcr80.dll!0000000068b43894() kernel32.dll!00007ffbe68a13d2() ntdll.dll!00007ffbe69f5454() ntdll.dll!00007ffbe6a7121a() KERNELBASE.dll!00007ffbe3f51118() fb_inet_server.exe!ISC_event_wait(event_t * event, const long value, const long micro_seconds) Line 2012 fb_inet_server.exe!Jrd::LockManager::blocking_action_thread() Line 1581 fb_inet_server.exe!Jrd::LockManager::blocking_action_thread(void * arg) Line 469 fb_inet_server.exe!`anonymous namespace'::threadStart(void * arg) Line 139 msvcr80.dll!0000000068b437d7() msvcr80.dll!0000000068b43894() kernel32.dll!00007ffbe68a13d2() ntdll.dll!00007ffbe69f5454() ntdll.dll!00007ffbe6a7121a() ntdll.dll!00007ffbe6a1d899() ntdll.dll!00007ffbe6a1b264() fb_inet_server.exe!Firebird::PublicHandleHolder::hold(Firebird::PublicHandle * handle, const char * from) Line 109 fb_inet_server.exe!`anonymous namespace'::AttachmentHolder::validateHandle(Jrd::thread_db * tdbb, Jrd::Attachment * const attachment, const char * from) Line 318 fb_inet_server.exe!jrd8_detach_database(__int64 * user_status, Jrd::Attachment * * handle) Line 2449 fb_inet_server.exe!fb_ping(__int64 * user_status, unsigned int * db_handle) Line 6083 fb_inet_server.exe!`anonymous namespace'::attachmentShutdownThread(void * arg) Line 7396 fb_inet_server.exe!`anonymous namespace'::threadStart(void * arg) Line 139 msvcr80.dll!0000000068b437d7() msvcr80.dll!0000000068b43894() kernel32.dll!00007ffbe68a13d2() ntdll.dll!00007ffbe69f5454() -- Mgr. Jiří Činčura Independent IT Specialist |
|
From: Vlad K. <hv...@us...> - 2016-09-27 15:08:17
|
26.09.2016 13:02, Jiří Činčura wrote: > Hi *, > > I've had quite a few occurrences of transaction being stuck on Firebird > 2.5.6 in last 7 days. Even trying to close the connection from > monitoring tables was not working. I had to kill the process. > > I was able to take a dump (actually 2, from different processes), but > given it's a production system with real data, I would rather not upload > it publicly. It's 2.5.6.27020, 64bit, Classic. It happened in 2.5.5 as > well. Bellow are the call stacks of threads in fb_inet_server (with PDB > for Firebird loaded). > > I suppose the first thread here is the key. Looks like it's spinning in > "while (owner->own_ast_count)" forever (at least couple of hours before > I found out, because of a lot of deadlock timeouts in SQL). > > Anybody interested in the dumps? Yes, send it to me. But better is to have reproducible example, or, at least, memory dump before attachments are killed - i.e. when transaction stuck is detected. Regards, Vlad PS Stack shows that EXECUTE STATEMENT is used, probably with autonomous transactions - it could block itself if used not correctly (for example: update record and then update it again in autonomus tx with "wait" option) |
|
From: Jiří Č. <ji...@ci...> - 2016-09-27 16:12:52
|
> Yes, send it to me. But better is to have reproducible example, or, at > least, memory dump before attachments are killed - i.e. when transaction > stuck is detected. Sending link privately. When that happens again I'll try to take the dump early. > PS Stack shows that EXECUTE STATEMENT is used, probably with autonomous > transactions - it could block itself if used not correctly (for example: > update record and then update it again in autonomus tx with "wait" > option) I'll check for that. It's a huge database, so it will take a while. -- Mgr. Jiří Činčura Independent IT Specialist |