Since worker/4.6.0 (?), worker either crashes when opening bookmarks window through shortkey, or hangs and has to be xkill-ed
Error message on console :
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
Abort
using strace :
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=35922, si_uid=20325, si_status=SIGABRT, si_utime=12, si_stime=13} ---
rt_sigreturn({mask=[INT CHLD]}) = -1 EINTR (Interrupted system call)
wait4(-1, [{WIFSIGNALED(s) && WTERMSIG(s) == SIGABRT}], WNOHANG|WSTOPPED, {ru_utime={tv_sec=0, tv_usec=132005}, ru_stime={tv_sec=0, tv_usec=147153}, ...}) = 35922
wait4(-1, 0x7ffd4d510750, WNOHANG|WSTOPPED, 0x7ffd4d510760) = -1 ECHILD (No child processes)
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
write(18, "Abort\n", 6Abort
) = 6
any possibility to get some more debugging output ?
Hi,
Worker uses asynchronous background tasks to check for existence of bookmark entries. I can understand a hang if the path to check is on some file system that is temporarily not available (like a NFS mount), but it should not crash. Do you have a special file in your bookmarks which is not a regular local file (either on a mounted FS, or even some special file like a pipe or block device or so)?
You can run Worker with gdb to get more info:
gdb worker
(gdb) run
....
(gdb) bt
Hi Ralf,
sorry, took quite a long time to bring worker to crash. Thought it would never fail when running inside gdb :-)
here's the stack trace - do you want me to select a special frame to get more information ? I kept gdb open, so I can get just any information
Regarding your question : yes, I have many NFS-mounted paths in my bookmarks, several of them not always mounted. So far worker was showing them a strikethrough, was fine. And there was no delay as the directory was simply non-existent (bookmark is to a subdirectory of the main mountpoint)
unfortunately
1) I oversaw the IT service time this evening -> all hosts will be rebooted, gdb session is lost
2) worker executable has no debug information (stripped executable ?)
I could only extract following info prior to reboot, I assume it's poor information content
will relaunch worker on monday, using a non-striped executable, and hope that the crash will reproduce in shorter time frame
Hi,
thanks for the debug output. I'm now able to reproduce the issue. It looks like there are too many async jobs open so eventually there will the exception "Resource temporarily unavailable". I think there must be at least one entry for which the test for existence blocks more or less forever.
I will look into ways to implement a workaround.
Maybe, for verification, you could run worker within gdb, open the bookmark dialog a couple of times, scroll through all the items to trigger the existence test. Then interrupt the execution within gdb and look how many threads are existing (info threads). Normally, it should just be around 10. If you see a significant amount of additional threads, you can switch into any of those (thread xyz) and do a backtrace there (bt). My guess is that it hangs in some system call (probably stat).
Of course, having debugging symbols helps a lot.
OK, running anew, but for now no errors reported (v4.7.0)
Did I understand it correctly that the issue should vanish if I reduce the amount of bookmarks ?
No, it's not the amount of bookmarks. It's just that there must be some entries for which stat() never returns so there are more and more background jobs pending until the thread limit is reached. Since the blocking stat is uninterruptible, there is not much I can do. I will add a job limit so eventually Worker will block noticeable until the stat returns. Usually it happens with mounted network devices which are unavailable. I have reproduce the issue with a sshfs mounted file system. After disconnecting from the network, stat() will block uninterruptible until the network comes up again. I will also add an option to disable the existence test so Worker can skip those directories.
Hi Ralf,
I think we can close this topic. Couldn't reproduce using 4.7.0 and you have anyhow provided a fix in 4.8.0 (also no crashes seen so far)