Re: [Mingw-w64-public] winpthreads and GCC's posix-threads only features (eg C++0x std::thread)

[K. Frank <kfrank29.c@gm...>]

Hi Ruben!

On Mon, Jun 13, 2011 at 5:00 PM, Ruben Van Boxem
<vanboxem.ruben@...> wrote:
> 2011/6/13 K. Frank <kfrank29.c@...>:
>> Hi Ruben!
>> On Mon, Jun 13, 2011 at 2:54 PM, Ruben Van Boxem
>> <vanboxem.ruben@...> wrote:
>>>>> ...
>>>> 2. C++0x std::thread is not automagically enabled.
>>>> ...
>>>>  - And that ends up in an undefined reference to "std::thread::join()".
>>>> ...
>> Anyway, std::thread is not pure header, but has some (compiled)
>> implementation somewhere.  (I assume that it's in libstdc++, but
>> I never tried to verify that.)  The source code for the implementation
>> for std::thread, thread.cc (and other files), is basically entirely
>> protected by:
>>
>>   #if defined(_GLIBCXX_HAS_GTHREADS) && defined(_GLIBCXX_USE_C99_STDINT_TR1)
>> ...
>> Anyway, this is just a suggestion -- maybe Ruben is seeing a
>> different problem that happens to have a similar symptom.
>
> No I think you're spot on. I'm sure Kai has some other patches in GCC
> trunk that enable this (he assured he had the std::thread stuff
> working). If he would be so kind as to provide a date/revision I'll
> cherry-pick that change into my GCC 4.6 and see where that gets me.
> ...
>>> Maybe the
>>> std::thread things aren't properly exported from the shared libstdc++
>>> build?
>>
>> This suggestion -- that libstdc++ does contain std::thread::join(), but
>> that it is not being exported -- seems plausible.
>> ...
> I think I'm wrong here, the macro isn't defined (even in the final g++
> I have I have to define it explicitely...). I'm sure these classes are
> defined just like any other libstdc++ class or function. Anyway, I'm
> using vanilla GCC, except for two patches that allowed me to use posix
> threads by Kai from trunk.

Okay.  Assuming that you have the export stuff set up properly, I'm
guessing that you will want to define

   _POSIX_TIMEOUTS
   _GLIBCXX__PTHREADS   (note the double underscore, "__")
   _GLIBCXX_HAS_GTHREADS

when you build the library.

Note that _GLIBCXX_USE_C99_STDINT_TR1 should also be defined,
but I'm guessing that it gets defined automatically in the "standard"
configuration.

Now the above macros must get defined somehow when g++ is
built for linux.  How, I don't know.  It might be nice to figure out
the "official" way to activate these macros, but when I was poking
around, it was completely opaque to me, so just defining them by
hand might be the most practical way to go.

> ...
> Thanks for the continued assistance.

Not at all.  I'm eager to get a stable build with integrated std::thread.

Happy hacking!


K. Frank

Re: [Mingw-w64-public] winpthreads and GCC's posix-threads only features (eg C++0x std::thread)

[Ruben Van Boxem <vanboxem.ruben@gm...>]

2011/6/13 K. Frank <kfrank29.c@...>:
> Hi Ruben!
>
> On Mon, Jun 13, 2011 at 2:54 PM, Ruben Van Boxem
> <vanboxem.ruben@...> wrote:
>> 2011/6/13 Ruben Van Boxem <vanboxem.ruben@...>:
>>> 2011/6/13 Ruben Van Boxem <vanboxem.ruben@...>:
>>>> Ok, I ran into some problems with "--enable-threads=posix"...
>>>> ...
>>> 2. C++0x std::thread is not automagically enabled. Specifically, I
>>> needed to perform the following manual steps (taken from K. Frank's
>>> howto here: http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTikwrRkzOuUdKOO2ShYSG7zqt8qq3QmXPfZsYhGa%40mail.gmail.com&forum_name=mingw-w64-public):
>>>  - Add "-D_POSIX_TIMEOUTS -D_GLIBCXX__PTHREADS
>>> -D_GLIBCXX_HAS_GTHREADS" to the commandline, along with "-std=c++0x"
>>> of course.
>>>  - Uncomment the line in bits/error_constants.h (strange, Explorer
>>> won't show this file, but grep found it anyways... it's not hidden
>>> either)
>>>  - And that ends up in an undefined reference to "std::thread::join()".
>>>
>>> I must be doing something wrong then :(
>
> I had the same problem (undefined reference to "std::thread::join()")
> getting std::thread to work.  Of course, in my case, I wasn't building
> g++ -- I was using a pre-built g++, and tweaking the environment to
> get std::thread to work.  (That's a long way of saying that even though
> Ruben and I see the same symptom, we may have different causes.)
>
> Anyway, std::thread is not pure header, but has some (compiled)
> implementation somewhere.  (I assume that it's in libstdc++, but
> I never tried to verify that.)  The source code for the implementation
> for std::thread, thread.cc (and other files), is basically entirely
> protected by:
>
>   #if defined(_GLIBCXX_HAS_GTHREADS) && defined(_GLIBCXX_USE_C99_STDINT_TR1)
>
> In my case -- linking against a pre-built libstdc++ (or whatever
> library has the std::thread implementation) -- I got the undefined
> reference, presumably because when the implementation
> library was built, _GLIBCXX_HAS_GTHREADS was not defined.
>
> So in my case, I simply added the source file thread.cc (and
> some others) to the compile line when I compiled my program
> that used std::thread.  Because that compile command had
> _GLIBCXX_HAS_GTHREADS explicitly defined, the content of
> thread.cc got compiled (no longer #ifdef'ed out) and provided the
> function std::thread::join() for my program at link time.
>
> However, if you're building a new g++ from the ground up (including
> the standard libraries), then I would think that having
>
>   _GLIBCXX_HAS_GTHREADS
>
> (and other required macros) defined at build time would cause the
> implementation for std::thread::join() contained in thread.cc to
> be compiled and linked into libstdc++ (or whatever the right library
> is), and therefore std::thread::join() should be available when you
> compile and link a test program using your newly built g++.
>
> Anyway, this is just a suggestion -- maybe Ruben is seeing a
> different problem that happens to have a similar symptom.

No I think you're spot on. I'm sure Kai has some other patches in GCC
trunk that enable this (he assured he had the std::thread stuff
working). If he would be so kind as to provide a date/revision I'll
cherry-pick that change into my GCC 4.6 and see where that gets me.

>
>>
>> Like for example adding "--disable-static" to the GCC configure line.
>> This probably caused the linker error, although the libstdc++.dll.a
>> import library should've been used, no? Qmake also wouldn't build,
>> because libstdc++.a was missing, that's how I found out. Maybe the
>> std::thread things aren't properly exported from the shared libstdc++
>> build?
>
> This suggestion -- that libstdc++ does contain std::thread::join(), but
> that it is not being exported -- seems plausible.
>
> My copy of thread.cc (where std::thread::join() is implemented) came,
> I believe, from the non-mingw-ized gnu source.  In any event, it does not
> have any of that dllexport / dllimport stuff that I've never truly understood
> the need for, but that seems to be necessary in the windows world.

I think I'm wrong here, the macro isn't defined (even in the final g++
I have I have to define it explicitely...). I'm sure these classes are
defined just like any other libstdc++ class or function. Anyway, I'm
using vanilla GCC, except for two patches that allowed me to use posix
threads by Kai from trunk.

>
> Let's say that you're compiling thread.cc with
>
>   _GLIBCXX_HAS_GTHREADS
>
> defined.  Then the std::thread::join() implementation should be in the
> object file and presumably also the library.  But if the mingw-w64 source
> has a "bug" in that std::thread::join() isn't getting exported, then you
> would get your undefined reference.  And the "bug" would never be
> noticed until someone tried to build the library with
>
>   _GLIBCXX_HAS_GTHREADS
>
> defined.
>
>>
>> I'm rebuilding now to see where I can get.
>> ...
>>> Ruben
>
> Thanks again for your effort and for making your builds available.
>
> Please let me know if there is any more detail I can offer that
> might be helpful.

Thanks for the continued assistance.

>
> Best.
>
>
> K. Frank
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Mingw-w64-public mailing list
> Mingw-w64-public@...
> https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
>

Re: [Mingw-w64-public] winpthreads and GCC's posix-threads only features (eg C++0x std::thread)

[K. Frank <kfrank29.c@gm...>]

Hi Ruben!

On Mon, Jun 13, 2011 at 2:54 PM, Ruben Van Boxem
<vanboxem.ruben@...> wrote:
> 2011/6/13 Ruben Van Boxem <vanboxem.ruben@...>:
>> 2011/6/13 Ruben Van Boxem <vanboxem.ruben@...>:
>>> Ok, I ran into some problems with "--enable-threads=posix"...
>>> ...
>> 2. C++0x std::thread is not automagically enabled. Specifically, I
>> needed to perform the following manual steps (taken from K. Frank's
>> howto here: http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTikwrRkzOuUdKOO2ShYSG7zqt8qq3QmXPfZsYhGa%40mail.gmail.com&forum_name=mingw-w64-public):
>>  - Add "-D_POSIX_TIMEOUTS -D_GLIBCXX__PTHREADS
>> -D_GLIBCXX_HAS_GTHREADS" to the commandline, along with "-std=c++0x"
>> of course.
>>  - Uncomment the line in bits/error_constants.h (strange, Explorer
>> won't show this file, but grep found it anyways... it's not hidden
>> either)
>>  - And that ends up in an undefined reference to "std::thread::join()".
>>
>> I must be doing something wrong then :(

I had the same problem (undefined reference to "std::thread::join()")
getting std::thread to work.  Of course, in my case, I wasn't building
g++ -- I was using a pre-built g++, and tweaking the environment to
get std::thread to work.  (That's a long way of saying that even though
Ruben and I see the same symptom, we may have different causes.)

Anyway, std::thread is not pure header, but has some (compiled)
implementation somewhere.  (I assume that it's in libstdc++, but
I never tried to verify that.)  The source code for the implementation
for std::thread, thread.cc (and other files), is basically entirely
protected by:

   #if defined(_GLIBCXX_HAS_GTHREADS) && defined(_GLIBCXX_USE_C99_STDINT_TR1)

In my case -- linking against a pre-built libstdc++ (or whatever
library has the std::thread implementation) -- I got the undefined
reference, presumably because when the implementation
library was built, _GLIBCXX_HAS_GTHREADS was not defined.

So in my case, I simply added the source file thread.cc (and
some others) to the compile line when I compiled my program
that used std::thread.  Because that compile command had
_GLIBCXX_HAS_GTHREADS explicitly defined, the content of
thread.cc got compiled (no longer #ifdef'ed out) and provided the
function std::thread::join() for my program at link time.

However, if you're building a new g++ from the ground up (including
the standard libraries), then I would think that having

   _GLIBCXX_HAS_GTHREADS

(and other required macros) defined at build time would cause the
implementation for std::thread::join() contained in thread.cc to
be compiled and linked into libstdc++ (or whatever the right library
is), and therefore std::thread::join() should be available when you
compile and link a test program using your newly built g++.

Anyway, this is just a suggestion -- maybe Ruben is seeing a
different problem that happens to have a similar symptom.

>
> Like for example adding "--disable-static" to the GCC configure line.
> This probably caused the linker error, although the libstdc++.dll.a
> import library should've been used, no? Qmake also wouldn't build,
> because libstdc++.a was missing, that's how I found out. Maybe the
> std::thread things aren't properly exported from the shared libstdc++
> build?

This suggestion -- that libstdc++ does contain std::thread::join(), but
that it is not being exported -- seems plausible.

My copy of thread.cc (where std::thread::join() is implemented) came,
I believe, from the non-mingw-ized gnu source.  In any event, it does not
have any of that dllexport / dllimport stuff that I've never truly understood
the need for, but that seems to be necessary in the windows world.

Let's say that you're compiling thread.cc with

   _GLIBCXX_HAS_GTHREADS

defined.  Then the std::thread::join() implementation should be in the
object file and presumably also the library.  But if the mingw-w64 source
has a "bug" in that std::thread::join() isn't getting exported, then you
would get your undefined reference.  And the "bug" would never be
noticed until someone tried to build the library with

   _GLIBCXX_HAS_GTHREADS

defined.

>
> I'm rebuilding now to see where I can get.
> ...
>> Ruben

Thanks again for your effort and for making your builds available.

Please let me know if there is any more detail I can offer that
might be helpful.

Best.


K. Frank

Re: [Mingw-w64-public] winpthreads and GCC's posix-threads only features (eg C++0x std::thread)

[Ruben Van Boxem <vanboxem.ruben@gm...>]

2011/6/13 Ruben Van Boxem <vanboxem.ruben@...>:
> 2011/6/13 Ruben Van Boxem <vanboxem.ruben@...>:
>> Ok, I ran into some problems with "--enable-threads=posix"...
>>
>> There are undefined references to
>>
>> pthread_mutex_lock
>> pthread_mutex_unlock
>>
>> and pretty much every other pthread function. I fear gcc isn't linking
>> to libpthread.a. This seems like a bug in the build system. I'll add
>> "-lpthread" to the LDFLAGS and see where that gets me. But doesn't gcc
>> link to libpthread on linux? Or are the pthread functions part of
>> glibc (and don't need explicit linking)?
>
> Thanks to Kai's help on IRC, I now have built a GCC with posix
> threading model. Of course, jon_y helped a lot too with the build
> steps, which are quite complicated (winpthreads depends on libgcc).
> First observations:
>
> 1. Qt Creator (which was built with a "normal" mingw-w64 toolchain,
> namely, my 4.6.1-1 release, took 30 seconds to start up, with 100%
> dual core load. Not normal, and probably either related to me not
> rebuilding Qt/Qt Creator or a bug in winpthreads.
>
> 2. C++0x std::thread is not automagically enabled. Specifically, I
> needed to perform the following manual steps (taken from K. Frank's
> howto here: http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTikwrRkzOuUdKOO2ShYSG7zqt8qq3QmXPfZsYhGa%40mail.gmail.com&forum_name=mingw-w64-public):
>  - Add "-D_POSIX_TIMEOUTS -D_GLIBCXX__PTHREADS
> -D_GLIBCXX_HAS_GTHREADS" to the commandline, along with "-std=c++0x"
> of course.
>  - Uncomment the line in bits/error_constants.h (strange, Explorer
> won't show this file, but grep found it anyways... it's not hidden
> either)
>  - And that ends up in an undefined reference to "std::thread::join()".
>
> I must be doing something wrong then :(

Like for example adding "--disable-static" to the GCC configure line.
This probably caused the linker error, although the libstdc++.dll.a
import library should've been used, no? Qmake also wouldn't build,
because libstdc++.a was missing, that's how I found out. Maybe the
std::thread things aren't properly exported from the shared libstdc++
build?

I'm rebuilding now to see where I can get.

>
> The first two need fixing though, but the patch Kai put into trunk
> works as far as building GCC goes. I'll recompile Qt and Qt Creator to
> see if the 30 second full load thing is a matter of recompiling or a
> nifty bug in winpthreads.
>
> Ruben
>

Re: [Mingw-w64-public] winpthreads and GCC's posix-threads only features (eg C++0x std::thread)

[Ruben Van Boxem <vanboxem.ruben@gm...>]

2011/6/13 Ruben Van Boxem <vanboxem.ruben@...>:
> Ok, I ran into some problems with "--enable-threads=posix"...
>
> There are undefined references to
>
> pthread_mutex_lock
> pthread_mutex_unlock
>
> and pretty much every other pthread function. I fear gcc isn't linking
> to libpthread.a. This seems like a bug in the build system. I'll add
> "-lpthread" to the LDFLAGS and see where that gets me. But doesn't gcc
> link to libpthread on linux? Or are the pthread functions part of
> glibc (and don't need explicit linking)?

Thanks to Kai's help on IRC, I now have built a GCC with posix
threading model. Of course, jon_y helped a lot too with the build
steps, which are quite complicated (winpthreads depends on libgcc).
First observations:

1. Qt Creator (which was built with a "normal" mingw-w64 toolchain,
namely, my 4.6.1-1 release, took 30 seconds to start up, with 100%
dual core load. Not normal, and probably either related to me not
rebuilding Qt/Qt Creator or a bug in winpthreads.

2. C++0x std::thread is not automagically enabled. Specifically, I
needed to perform the following manual steps (taken from K. Frank's
howto here: http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTikwrRkzOuUdKOO2ShYSG7zqt8qq3QmXPfZsYhGa%40mail.gmail.com&forum_name=mingw-w64-public):
 - Add "-D_POSIX_TIMEOUTS -D_GLIBCXX__PTHREADS
-D_GLIBCXX_HAS_GTHREADS" to the commandline, along with "-std=c++0x"
of course.
 - Uncomment the line in bits/error_constants.h (strange, Explorer
won't show this file, but grep found it anyways... it's not hidden
either)
 - And that ends up in an undefined reference to "std::thread::join()".

I must be doing something wrong then :(

The first two need fixing though, but the patch Kai put into trunk
works as far as building GCC goes. I'll recompile Qt and Qt Creator to
see if the 30 second full load thing is a matter of recompiling or a
nifty bug in winpthreads.

Ruben

Re: [Mingw-w64-public] winpthreads and GCC's posix-threads only features (eg C++0x std::thread)

[Ruben Van Boxem <vanboxem.ruben@gm...>]

Ok, I ran into some problems with "--enable-threads=posix"...

There are undefined references to

pthread_mutex_lock
pthread_mutex_unlock

and pretty much every other pthread function. I fear gcc isn't linking
to libpthread.a. This seems like a bug in the build system. I'll add
"-lpthread" to the LDFLAGS and see where that gets me. But doesn't gcc
link to libpthread on linux? Or are the pthread functions part of
glibc (and don't need explicit linking)?

Thanks,

Ruben

2011/6/12 K. Frank <kfrank29.c@...>:
> Hello Ruben!
>
> On Sun, Jun 12, 2011 at 3:32 PM, Ruben Van Boxem
> <vanboxem.ruben@...> wrote:
>> Hi,
>>
>> I'm upgrading my personal build to include Kai's winpthreads instead
>> of the ancient pthreads-win32. Now, Kai already said on IRC that this
>> would allow me to build all of GCC with posix threading, and I thought
>> that maybe this would enable me to enable cool features available on
>> Linux (like C++0x std::threads).
>
> That would be wonderful!  I would be especially eager to see a g++ 4.6
> version that had Kai's new winpthreads and std::thread support built in.
>
>> My question is a simple one (eurhm, two): would passing
>> --enable-threads=posix automagically enable the features missing with
>> win32 threads? Are there any other GCC/runtime features hidden behind
>> the posix threading model?
>
> I don't know the details of what "--enable-threads=posix" does.  As far
> as getting std::thread to work with pthreads on windows, I believe that
> there are some very minor changes you would need to make.  (I think
> that g++'s std::thread works "out of the box" using pthreads on linux.)
>
> I posted some notes to this list detailing the tweaks I needed to make
> to get std::thread working with pthreads:
>
>   "Pthreads-recipe for using std::thread with mingw (mingw32 / mingw-w64)"
>
>   http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTikwrRkzOuUdKOO2ShYSG7zqt8qq3QmXPfZsYhGa%40mail.gmail.com&forum_name=mingw-w64-public
>
> I would think that the std::thread / gthread side of the equation should
> be the same.  Kai's winpthreads could well be a little different than the
> pthreads-w64 that I used.  As I remember it, the gthread wrapper that
> std::thread uses to access pthreads had some modest dependencies
> on the pthreads implementation, so it's possible that gthread would
> need to be tweaked a bit to work with Kai's implementation.
>
> I think std::thread will probably not work automagically, but I do think
> that it should be very easy to get it working.

These are the steps you need to take if GCC is configured with win32
threads, I think. And exactly what I'm trying to avoid when I use
posix thread completely. If I think wrong, I'll need to try this, but
I don't like the aspect of messing with the code that much.

>
>> I understand using this option will theoretically reduce performance
>> in stuff like openmp and GCC's internal threading, but as both are
>> pretty limited anyways, I don't really see this impacting real-world
>> performance much. Also, this will provide a larger test bed for Kai's
>> implementation.
>
> I think that shooting std::thread at winpthreads would be a good test
> of some of its functionality.  Also, if you know how to run the g++ test
> suites (I don't know how to run them automatically, but I did run one or
> two manually.), the g++ std::thread-related test suites would be a good
> test for winpthreads, as well.
>
>> Thanks!
>>
>> Ruben
>
> On the contrary, thanks to you (and Kai)!
>
>
> Best.
>
>
> K. Frank
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Mingw-w64-public mailing list
> Mingw-w64-public@...
> https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
>