Re: [TCLCORE] PURIFY and unloading extensions issue (?)

[Neophytos D. <neo...@gm...>]

AppExitHandler in threadPoolCmd attempts to release the pool while
unloading. When a TpoolWorker exits, it attempts to Tcl_DeleteInterp at
line 1274, which ends up calling TclpThreadGetGlobalTSD but tsdGlobal.key
is NULL by that time as described in my original message.

In the given example, one can "::tpool::release $pool" and wait/sleep for a
while for all TpoolWorker to exit and there would be no issue.

But, still, I think freeing the TSD before unloading extensions seems to be
in the wrong place. I leave that to those who know the code better to
figure out if any change is required here.

Please note that without "clock format", I could not get it to fail but
that's most likely because "clock format" just references some Tcl_Obj.

I hope it helps.

Kind regards,
Neophytos

PS. Please note that, I realized later, that the example did not have to be
so convoluted. A simple "::tpool::post $pool [list sayhi]" in test.tcl and
a "clock format 1722260433" in somefile.tcl (without requiring the Thread
package again) still fails.

On Mon, Jul 29, 2024 at 9:51 AM Neophytos Demetriou <neo...@gm...>
wrote:

> The following consistently fails with Thread package 3.0b2 on linux using
> tcl 9.0b2 when you run "tclsh9.0 test.tcl"
>
> == test.tcl ==
> package require Thread
> set pool [::tpool::create -minworkers 1 -maxworkers 4 -idletime 5 -initcmd
> [list source somefile.tcl]]
> ::tpool::post -detached -nowait $pool [list sayhi]
> after 10000 [list set v 1]
> vwait v
>
> == somefile.tcl ==
> package require Thread
> proc sayhi {} {
>     set fmt {[%d/%b/%Y:%H:%M:%S %z]}
>     puts str=[clock format 1722260433 -format $fmt]
> }
>
> == Here is the message from the AddressSanitizer ==
>
> /home/phi/build-projects/tcl9.0b2/unix/tclUnixThrd.c:915:12: runtime
> error: load of null pointer of type 'pthread_key_t'
> AddressSanitizer:DEADLYSIGNAL
> =================================================================
> ==71907==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000
> (pc 0x7f27d6a09c7f bp 0x7f27cc225910 sp 0x7f27cc2258f0 T1)
> ==71907==The signal is caused by a READ memory access.
> ==71907==Hint: address points to the zero page.
> AddressSanitizer:DEADLYSIGNAL
> AddressSanitizer: nested bug in the same thread, aborting.
>
> == Here is how I compiled tcl 9.0b2 ==
>
> CFLAGS="-fsanitize=address -fsanitize=undefined -DPURIFY"
> LDFLAGS="-fsanitize=address -fsanitize=undefined" ./configure
> --enable-symbols=all
>
>
> On Mon, Jul 29, 2024 at 7:57 AM Neophytos Demetriou <neo...@gm...>
> wrote:
>
>> hmm, I disabled all above extensions and it still occurs, which reminded
>> me I also use Thread package 3.0b2 and some preliminary investigation seems
>> to show that it only happens when Thread package is loaded (3.0b2)
>>
>> more tomorrow and again sorry for wasting your time
>>
>> - Neophytos
>>
>>
>> On Mon, Jul 29, 2024 at 7:22 AM Neophytos Demetriou <neo...@gm...>
>> wrote:
>>
>>> Hi Sergey,
>>>
>>> Thanks for your message. Answers below.
>>>
>>> did you test it before or after [96910cd8822ffb8f]
>>>> <https://core.tcl-lang.org/tcl/info/96910cd8822ffb8f>? Just to ensure
>>>> it is still acute...
>>>>
>>>
>>> It was with tcl 9.0b2, so before that change.
>>>
>>>
>>>> The whole "unload" thing was initially introduced by Nathan in
>>>> [3b0e92b198b86975]
>>>> <https://core.tcl-lang.org/tcl/info/3b0e92b198b86975>, however it
>>>> seemed to have some issues (like [34870ab5756911d1]
>>>> <https://core.tcl-lang.org/tcl/info/34870ab5756911d1> or
>>>> [ae09f6b190ceec31]
>>>> <https://core.tcl-lang.org/tcl/info/ae09f6b190ceec31>), which I tried
>>>> to fix in the last time as good as I was able to do that.
>>>>
>>>
>>> I cannot tell if it is relevant or not. It does not look like it.
>>>
>>>
>>>> Which versions (and platforms) are affected or do you mean exactly
>>>> here? Is it stock tcl or some application of you, that uses Tcl as a
>>>> subsystem (in particular, it is Tcl_Main* or your own main)?
>>>>
>>>
>>> It was a tcl file that loaded twebserver in the main thread and
>>> twebserver, tjson and thtml in the spawn twebserver threads. All of these
>>> extensions were using Tcl_CreateThreadExitHandler in their initialization
>>> function. They did not have to but I did not notice that was the case
>>> before this issue. Changing them all to use CreateExitHandler seems to have
>>> resolved the issue.
>>>
>>> I will investigate further tomorrow and report back but my guess is that
>>> there is a TCL_TSD_INIT used somewhere when the thread exit handler runs
>>> that causes this issue during unload.
>>>
>>> Regards,
>>> Neophytos
>>>
>>> PS. My apologies but I have to run right now. I will check further
>>> tomorrow and report back.
>>>
>>> Regards,
>>>> Sergey.
>>>>
>>>> 29.07.2024 10:29, Neophytos Demetriou wrote:
>>>>
>>>> Hi,
>>>>
>>>> When the PURIFY flag is used Tcl_Exit calls Tcl_Finalize that in turn
>>>> calls TclFinalizeSynchronization, which sets tsdGlobal.key to NULL via
>>>> TclFinalizeThreadStorage.
>>>>
>>>> Then, when it afterwards tries to unload extensions, it attempts to use
>>>> the TSD (thread-specific data) again and because tsdGlobal.key is NULL, the
>>>> AddressSanitizer complains about a NULL pthread key.
>>>>
>>>> I was not able to track where the TSD is used during unloading or if
>>>> this is an issue with the extensions being used and whether they have to
>>>> unload in a specific way. As a consequence I stopped using the PURIFY flag
>>>> but if anyone has any suggestions, please let us know.
>>>>
>>>> Kind regards,
>>>> Neophytos
>>>>
>>>> _______________________________________________
>>>> Tcl-Core mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/tcl-core
>>>>
>>>>