#254 Infinite loop during SSH session

[Rob Swindell]

This is actually in Linux Wayland mode (not Windows-GDI). Can't change the drop down in Firefox for some reason.

Just happened while trying to respond to a message (coincidentally, about SyncTERM). Backtrace of threads after attaching with gdb:

(gdb) thread apply all bt

Thread 10 (Thread 0x7022d8dc46c0 (LWP 13738) "WL Events"):
#0  0x00007022d9698d71 in __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7022d401c3bc) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7022d401c3bc) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7022d401c3bc, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
#3  0x00007022d969b7ed in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7022d401c368, cond=0x7022d401c390) at ./nptl/pthread_cond_wait.c:503
#4  ___pthread_cond_wait (cond=0x7022d401c390, mutex=0x7022d401c368) at ./nptl/pthread_cond_wait.c:627
#5  0x00005a5e3065698d in xp_sem_wait (sem=0x5a5e30c40160 <sample_complete_sem>) at xpsem.c:164
#6  0x00005a5e30650af8 in xp_play_sample_locked (sample=0x7022d40b0360 "\200\200\200\200\200\200\200\200\200\200\177\177~~~}}}}~~}}}}}", '|' <repeats 12 times>, "{{zzyyyyyzz{|~\177~\200\202\202\202\200~|zxvuttttuuuttsrqoonnnnooonmljhfcbaabdgjnwz~\200\202\202\201\200~|zxvuttttuutttsrqponnnnooonmljhfdbaabdfjmvz}\200\202\202\202\201\177|zxwuttttuuuttsrqponnnnooonnljhfdbaabcfi"..., size=2205, background=false) at xpbeep.c:1252
#7  0x00005a5e30650bdb in xp_play_sample (sample=0x7022d40b0360 "\200\200\200\200\200\200\200\200\200\200\177\177~~~}}}}~~}}}}}", '|' <repeats 12 times>, "{{zzyyyyyzz{|~\177~\200\202\202\202\200~|zxvuttttuuuttsrqoonnnnooonmljhfcbaabdgjnwz~\200\202\202\201\200~|zxvuttttuutttsrqponnnnooonmljhfdbaabdfjmvz}\200\202\202\202\201\177|zxwuttttuuuttsrqponnnnooonnljhfdbaabcfi"..., size=2205, background=false) at xpbeep.c:1270
#8  0x00005a5e30650df4 in xptone (freq=440, duration=100, shape=6) at xpbeep.c:1332
#9  0x00005a5e30650e46 in xpbeep (freq=440, duration=100) at xpbeep.c:1388
#10 0x00005a5e3060e817 in wl_event_thread (args=0xe) at wl_events.c:1788
#11 0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#12 0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 9 (Thread 0x7022d8ca06c0 (LWP 13739) "Blinker"):
#0  0x00007022d96ecadf in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=0x7022d8c9fea0, rem=0x7022d8c9fea0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:78
#1  0x00007022d96f9a27 in __GI___nanosleep (req=<optimized out>, rem=<optimized out>) at ../sysdeps/unix/sysv/linux/nanosleep.c:25
#2  0x00005a5e306190aa in blinker_thread (data=0x0) at bitmap_con.c:1085
#3  0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#4  0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 8 (Thread 0x7022d99996c0 (LWP 13740) "WL Mouse"):
#0  0x00007022d9698d71 in __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7022c8000c3c) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7022c8000c3c) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7022c8000c3c, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
#3  0x00007022d969b7ed in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7022c8000be8, cond=0x7022c8000c10) at ./nptl/pthread_cond_wait.c:503
#4  ___pthread_cond_wait (cond=0x7022c8000c10, mutex=0x7022c8000be8) at ./nptl/pthread_cond_wait.c:627
#5  0x00005a5e3065698d in xp_sem_wait (sem=0x5a5e30bfd1c8 <state+392>) at xpsem.c:164
#6  0x00005a5e3064933e in listSemWait (list=0x5a5e30bfd178 <state+312>) at link_list.c:203
#7  0x00005a5e30600ef1 in mouse_wait () at mouse.c:584
#8  0x00005a5e30608f21 in wl_mouse_thread (data=0x0) at wl_cio.c:283
#9  0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#10 0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 7 (Thread 0x7022d3fff6c0 (LWP 13741) "Mouse"):
#0  0x00007022d9698d71 in __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7022c8000bc8) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7022c8000bc8) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7022c8000bc8, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
#3  0x00007022d969b7ed in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7022c8000b78, cond=0x7022c8000ba0) at ./nptl/pthread_cond_wait.c:503
#4  ___pthread_cond_wait (cond=0x7022c8000ba0, mutex=0x7022c8000b78) at ./nptl/pthread_cond_wait.c:627
--Type <RET> for more, q to quit, c to continue without paging--c
#5  0x00005a5e3065698d in xp_sem_wait (sem=0x5a5e30bfd170 <state+304>) at xpsem.c:164
#6  0x00005a5e3064933e in listSemWait (list=0x5a5e30bfd120 <state+224>) at link_list.c:203
#7  0x00005a5e305ff12f in ciolib_mouse_thread (data=0x0) at mouse.c:314
#8  0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#9  0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 6 (Thread 0x7022d205b6c0 (LWP 39797) "SSH Output"):
#0  0x00007022d9698d71 in __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x7022d205adc0, op=393, expected=0, futex_word=0x5a5e6b45963c) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x7022d205adc0, clockid=23134, expected=0, futex_word=0x5a5e6b45963c) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x5a5e6b45963c, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x7022d205adc0, private=private@entry=0) at ./nptl/futex-internal.c:139
#3  0x00007022d969bc8e in __pthread_cond_wait_common (abstime=0x7022d205adc0, clockid=0, mutex=0x5a5e6b4595e8, cond=0x5a5e6b459610) at ./nptl/pthread_cond_wait.c:503
#4  ___pthread_cond_timedwait64 (cond=0x5a5e6b459610, mutex=0x5a5e6b4595e8, abstime=0x7022d205adc0) at ./nptl/pthread_cond_wait.c:652
#5  0x00005a5e30656e98 in xp_sem_timedwait (sem=0x5a5e309de530 <conn_outbuf+80>, abs_timeout=0x7022d205adc0) at xpsem.c:279
#6  0x00005a5e3064adc3 in sem_trywait_block (sem=0x5a5e309de530 <conn_outbuf+80>, timeout=99) at semwrap.c:41
#7  0x00005a5e3044b730 in conn_buf_wait_cond (buf=0x5a5e309de4e0 <conn_outbuf>, bcount=1, timeout=100, do_free=0) at conn.c:265
#8  0x00005a5e3047a9db in ssh_output_thread (args=0x0) at ssh.c:351
#9  0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#10 0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 5 (Thread 0x7022d15e86c0 (LWP 39798) "SSH Input"):
#0  0x00007022d971b4fd in __GI___poll (fds=0x7022d15e7e50, nfds=1, timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00005a5e3064b821 in socket_check (sock=9, rd_p=0x7022d15e7e82, wr_p=0x0, timeout=100) at sockwrap.c:207
#2  0x00005a5e3047a1b5 in ssh_input_thread (args=0x0) at ssh.c:225
#3  0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#4  0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 4 (Thread 0x7022d37fe6c0 (LWP 39799) "PlayNote"):
#0  0x00007022d9698d71 in __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x5a5e6b427258) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x5a5e6b427258) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x5a5e6b427258, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
#3  0x00007022d969b7ed in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5a5e6b427208, cond=0x5a5e6b427230) at ./nptl/pthread_cond_wait.c:503
#4  ___pthread_cond_wait (cond=0x5a5e6b427230, mutex=0x5a5e6b427208) at ./nptl/pthread_cond_wait.c:627
#5  0x00005a5e3065698d in xp_sem_wait (sem=0x5a5e6b42c0f8) at xpsem.c:164
#6  0x00005a5e3064933e in listSemWait (list=0x5a5e6b42c0a8) at link_list.c:203
#7  0x00005a5e305e0c87 in playnote_thread (args=0x5a5e6b42b3e0) at cterm.c:832
#8  0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#9  0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 3 (Thread 0x7022bffff6c0 (LWP 54454) "Sample Play"):
#0  0x00007022d9698d71 in __futex_abstimed_wait_common64 (private=28706, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7022c000ea9c) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=28706, abstime=0x0, clockid=0, expected=0, futex_word=0x7022c000ea9c) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7022c000ea9c, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
#3  0x00007022d969b7ed in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7022c00112b0, cond=0x7022c000ea70) at ./nptl/pthread_cond_wait.c:503
#4  ___pthread_cond_wait (cond=0x7022c000ea70, mutex=0x7022c00112b0) at ./nptl/pthread_cond_wait.c:627
#5  0x00007022d0dcc604 in pa_threaded_mainloop_wait () from /lib/x86_64-linux-gnu/libpulse.so.0
#6  0x00007022d804fc58 in pa_simple_new () from /lib/x86_64-linux-gnu/libpulse-simple.so.0
#7  0x00005a5e3065035b in do_xp_play_sample (sampo=0x7022c000df90 "\200\200\200\200\200\200\200\200\200\200\177\177~~~}}}}~~}}}}}", '|' <repeats 12 times>, "{{zzyyyyyzz{|~\177~\200\202\202\202\200~|zxvuttttuuuttsrqoonnnnooonmljhfcbaabdgjnwz~\200\202\202\201\200~|zxvuttttuutttsrqponnnnooonmljhfdbaabdfjmvz}\200\202\202\202\201\177|zxwuttttuuuttsrqponnnnooonnljhfdbaabcfi"..., sz=2205, freed=0x7022bfffeea4) at xpbeep.c:1025
#8  0x00005a5e306507c5 in xp_play_sample_thread (data=0x0) at xpbeep.c:1185
#9  0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#10 0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 2 (Thread 0x7022bb6de6c0 (LWP 54581) "threaded-ml"):
#0  0x00007022d971b4fd in __GI___poll (fds=0x7022b4007a50, nfds=2, timeout=30000) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007022d0dcb596 in ?? () from /lib/x86_64-linux-gnu/libpulse.so.0
#2  0x00007022d0db46e4 in pa_mainloop_poll () from /lib/x86_64-linux-gnu/libpulse.so.0
#3  0x00007022d0dbf1b3 in pa_mainloop_iterate () from /lib/x86_64-linux-gnu/libpulse.so.0
#4  0x00007022d0dbf278 in pa_mainloop_run () from /lib/x86_64-linux-gnu/libpulse.so.0
#5  0x00007022d0dcf7c1 in ?? () from /lib/x86_64-linux-gnu/libpulse.so.0
#6  0x00007022d09dd67b in ?? () from /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-16.1.so
#7  0x00007022d969caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#8  0x00007022d9729c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 1 (Thread 0x7022d9829680 (LWP 13734) "syncterm"):
#0  0x00007022d96ecadf in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=0x7fff678dd8f0, rem=0x7fff678dd8f0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:78
#1  0x00007022d96f9a27 in __GI___nanosleep (req=<optimized out>, rem=<optimized out>) at ../sysdeps/unix/sysv/linux/nanosleep.c:25
#2  0x00005a5e3045f0ee in doterm (bbs=0x5a5e309d8760 <retlist>) at term.c:6221
#3  0x00005a5e30470c0f in main (argc=1, argv=0x7fff678e6a18) at syncterm.c:2489

[Rob Swindell]

After lots of waiting, it finally updated the local display, but then just threw-up the hang-application notice again. Not accepting keyboard input.

[Rob Swindell]

Happened to me again today, while editing a message.

[Rob Swindell]

It appears that hitting Ctrl-Y to delete a line in fseditor.js triggers this hang in SyncTERM 1.9rc2.

[Stephen James Hurd]

The main interesting thing I see here is that the Wayland event thread is trying to beep... suggesting it's actually a PulseAudio issue...

[Stephen James Hurd]

Also, all the threads appear to be blocked... curious why you're classifying this as an "infinite loop" instead of a "hang".

[Stephen James Hurd]

Assuming it's near-100% CPU, using top -Hp <pid> and then f to add %CPU when this is occurring would help nail down the exact culprit... I expect it to be threads 2 or 3.

[Rob Swindell]

I can confirm it appears to be audio-related. Any received of ^G from the BBS appears to cause this hang (in Wayland mode only).

[Stephen James Hurd]

Ok, I fiddled with the preferred audio backends in the syncterm-1.9 branch, as long as portaudio is also enabled, I think it's more likely to work. If your build has portaudio enabled, you can disable PulseAudio in the config, no need to rebuild to test it.

[Stephen James Hurd]

This appears to be an issue with some versions of PipeWire causing some versions pa_simple() to hang rather than return an error. The new ordering should help, and future versions of SyncTERM won't use the pa_simple() API.

[Rob Swindell]

I saw the issue again today

[Rob Swindell]

With PulseAudio disabled