state-threads-users — State Threads users' list

Re: [ST-users] Help applying print_stk patch

[Yang Z. <yan...@gm...>]

Mike Abbott wrote:
>> $ CPATH=/tmp/st-1.8/LINUX_2.6.27-7-generic_DBG/ 
>> LIBRARY_PATH=/tmp/st-1.8/LINUX_2.6.27-7-generic_DBG/ gcc stbt.c -lst
>> /tmp/ccoL3i05.o: In function `h':
>> stbt.c:(.text+0x2e): undefined reference to `_st_print_thread_stacks'
>> collect2: ld returned 1 exit status
> 
> I get exactly the same error when I use the wrong path
> (the wrong LINUX_whatever_DBG), but when I use the right path the gcc
> command works.  Then when I run
> "LD_LIBRARY_PATH=/tmp/st-1.8/LINUX_whatever_DBG ./a.out" it runs fine.
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> State-threads-users mailing list
> Sta...@li...
> https://lists.sourceforge.net/lists/listinfo/state-threads-users

Whoops.  Turned out I had two libst.a's on my system, and the other one 
was getting in the way.
-- 
Yang Zhang
http://www.mit.edu/~y_z/

Re: [ST-users] Help applying print_stk patch

[Mike A. <ya...@ya...>]

> $ CPATH=/tmp/st-1.8/LINUX_2.6.27-7-generic_DBG/ 
> LIBRARY_PATH=/tmp/st-1.8/LINUX_2.6.27-7-generic_DBG/ gcc stbt.c -lst
> /tmp/ccoL3i05.o: In function `h':
> stbt.c:(.text+0x2e): undefined reference to `_st_print_thread_stacks'
> collect2: ld returned 1 exit status

I get exactly the same error when I use the wrong path
(the wrong LINUX_whatever_DBG), but when I use the right path the gcc
command works.  Then when I run
"LD_LIBRARY_PATH=/tmp/st-1.8/LINUX_whatever_DBG ./a.out" it runs fine.

Re: [ST-users] Waiting on multiple blocking events

[Mike A. <ya...@ya...>]

> Is there a way to "compose" together multiple blocking operations? 

Sounds interesting, kinda like what kqueue()/kevent() do:  one-stop
event processing.  "Block this thread until one of this laundry list of
disparate events fires."  Do you have a prototype implementation you'd
like to contribute to the project, or wrappers you'd contribute to its
"extensions" library?

Re: [ST-users] C++ stack unwinding

[Mike A. <ya...@ya...>]

> I'd like stack unwinding to take place in *all* spawned threads

As you discovered, ST has no knowledge of C++ exceptions.

However, I think you may be relying on platform-specific behavior.  My
understanding of uncaught exceptions is that they result in terminate()
being called, which defaults to abort(), which means no destructors are
called for any objects.  If you remove all the ST functions from your
test program and simply throw an uncaught exception:

[...]
int main() {
   cleanup c;
//   st_init();
//   st_thread_t t = st_thread_create(&f, (void*)"abc", 1, 0);
//   st_thread_join(t, NULL);
   f(NULL);
   return 0;
}

then c's destructor is not called either.  This matches what my copy of
Stroustrup says.  Does something different happen on your system?

On the other hand, if you were trying to have thread A *catch* an
exception thrown by thread B, then I could understand your desire to
have stack unwinding work properly and have object c's destructor
called.  But ST does not support propagating C++ exceptions across
threads.  ST does work just fine with C++ -- I have done so myself --
just not when it comes to exceptions crossing threads.

[ST-users] Help applying print_stk patch

[Yang Z. <yan...@gm...>]

I did:

patch < print_stk.patch # manually patched Makefile
CONFIG_GUESS_PATH=/usr/share/misc/ make default-debug

then built the following program, stbt.c:

#define DEBUG
#include <st.h>
#include <signal.h>
void *f(void *p) {
   st_sleep(10);
   return NULL;
}
void h(int s) {
   _st_print_thread_stacks(0);
}
int main() {
   struct sigaction sa;
   sa.sa_handler = h;
   sigemptyset(&sa.sa_mask);
   sa.sa_flags = 0;
   sigaction(SIGINT, &sa, NULL);

   st_init();
   st_thread_t ts[3];
   int i; 

   for (i = 0; i < 3; i++)
     ts[i] = st_thread_create(&f, NULL, 1, 0);
   for (i = 0; i < 3; i++)
     st_thread_join(ts[i], NULL);
   return 0;
}

but then I got:

$ CPATH=/tmp/st-1.8/LINUX_2.6.27-7-generic_DBG/ 
LIBRARY_PATH=/tmp/st-1.8/LINUX_2.6.27-7-generic_DBG/ gcc stbt.c -lst
/tmp/ccoL3i05.o: In function `h':
stbt.c:(.text+0x2e): undefined reference to `_st_print_thread_stacks'
collect2: ld returned 1 exit status

objdump shows that _st_print_thread_stacks is there in the .a file and 
non-static.  Any hints?
-- 
Yang Zhang
http://www.mit.edu/~y_z/

[ST-users] Waiting on multiple blocking events

[Yang Z. <yan...@gm...>]

Is there a way to "compose" together multiple blocking operations? 
E.g., for the following blocking operations:

   st_read_fully(...)
   st_read_fully(...)
   st_thread_join(...)
   st_cond_wait(...)
   st_sleep(...)

I'd like to be able to:

- wait for all of them to finish
- wait for any one of them to finish

The former is easy - spawn a thread per operation and join on all of 
them (in any order).

The latter is tricky.  It becomes especially so if you want to guarantee 
that *exactly* one of them successfully completes.  A two-phase commit 
must take place to select a winner and to have the loser clean up after 
itself (e.g. so the read operations do not simply drop incoming data).

Such composition would be powerful.  E.g., being able to compose with 
st_sleep(...) would render timeout parameters unnecessary.

Again, if I were to build my own API atop ST, then as long as the user 
only uses my API and not ST directly, then I can write the rules. 
However, I'm wondering if there's a simpler approach (less work), esp. 
one that allows ST to continue to be used.

The following is a Python implementation of this idea, using Python 2.5 
generators:

svn://scripts.mit.edu:36900/mit/golem/Share/svn/af
-- 
Yang Zhang
http://www.mit.edu/~y_z/

Re: [ST-users] C++ stack unwinding

[Yang Z. <yan...@gm...>]

Just to be more clear: I'd like stack unwinding to take place in *all* 
spawned threads, ideally without another set of bookkeeping state that's 
separate from what st already has to manage internally, and without 
requiring the user to use another API above ST (and only that API).

The closest solution I've come to so far involves creating another API 
on top of ST that will wrap all st_* calls (blocking ones in particular) 
in wrappers that throw exceptions on an error.  I then also wrap 
st_thread_create() in a wrapper that puts the new thread into a global 
list of threads, and also wraps the target function in a wrapper that 
calls st_thread_interrupt() on all other threads in the global list 
whenever an uncaught exception is thrown.

Yang Zhang wrote:
> The following simple C++ program does not end up calling the destructor 
> because the stack does not get unwound for the threads that aren't part 
> of the thread from which SIGABRT is called (by the uncaught exception). 
>  Is there a simple way to get stack unwinding to happen properly? Thanks 
> in advance.
> 
> #include <exception>
> #include <iostream>
> #include <st.h>
> using namespace std;
> 
> class cleanup {
>   public:
>     ~cleanup() { cout << "clean up" << endl; }
> };
> 
> void *f(void *) {
>   throw exception();
>   return NULL;
> }
> 
> int main() {
>   cleanup c;
>   st_init();
>   st_thread_t t = st_thread_create(&f, (void*)"abc", 1, 0);
>   st_thread_join(t, NULL);
>   return 0;
> }
> 


-- 
Yang Zhang
http://www.mit.edu/~y_z/

[ST-users] C++ stack unwinding

[Yang Z. <yan...@gm...>]

The following simple C++ program does not end up calling the destructor 
because the stack does not get unwound for the threads that aren't part 
of the thread from which SIGABRT is called (by the uncaught exception). 
  Is there a simple way to get stack unwinding to happen properly? 
Thanks in advance.

#include <exception>
#include <iostream>
#include <st.h>
using namespace std;

class cleanup {
   public:
     ~cleanup() { cout << "clean up" << endl; }
};

void *f(void *) {
   throw exception();
   return NULL;
}

int main() {
   cleanup c;
   st_init();
   st_thread_t t = st_thread_create(&f, (void*)"abc", 1, 0);
   st_thread_join(t, NULL);
   return 0;
}

-- 
Yang Zhang
http://www.mit.edu/~y_z/

Re: [ST-users] Scheduler

[Mike A. <ya...@ya...>]

> How does ST's scheduler work?  Is it a simple round-robin queue?

In a word, yes.

The scheduler maintains a queue of runnable threads.  At each scheduling
opportunity it simply pulls the head element off the queue and runs it.
So the real question is, in what order does ST add threads to the run
queue?  Unfortunately that is completely unpredictable.  Some of the
actions that append threads to the run queue include:
	- select, poll, kevent etc. indicate that one or more
	  descriptors are ready for I/O, in which case the threads are
	  added to the run queue in the order they appear in the
	  I/O-pending queue
	- a timer expires
	- a zombie thread is joined
	- a sleeping thread is interrupted by st_thread_interrupt()
	- a thread is created
	- a mutex is unlocked or a condvar signalled

> Is it possible to exercise any control over the scheduler (such as
> dropping in my own), short of explicitly using synchronization
> primitives everywhere to control what threads are eligible for
> execution?

Why do you need such control over thread execution order?  If you really
do need such control then just use the synchronizers; that's what
they're there for.

Re: [ST-users] Memory leaks

[Mike A. <ya...@ya...>]

> valgrind complains about memory leaks:

The leaks you show are legitimate.  The State Threads library has no
"de-initialize" function to clean up, by design.  If such cleanup is
very important to you, please feel free to code up a de-init routine and
contribute it to the project.  Until then you'll just have to ignore
them.
-- 
Mike Abbott    ma...@us...
State Threads Project co-administrator

[ST-users] Scheduler

[Yang Z. <yan...@gm...>]

How does ST's scheduler work?  Is it a simple round-robin queue?

Is it possible to exercise any control over the scheduler (such as 
dropping in my own), short of explicitly using synchronization 
primitives everywhere to control what threads are eligible for execution?
-- 
Yang Zhang
http://www.mit.edu/~y_z/

[ST-users] Memory leaks

[Yang Z. <yan...@gm...>]

When running this simple program:

#include <st.h>
int main() {
   st_init();
   return 0;
}

valgrind complains about memory leaks:

==745== 56 bytes in 1 blocks are still reachable in loss record 1 of 3
==745==    at 0x4C230BC: calloc (vg_replace_malloc.c:397)
==745==    by 0x401C3F: _st_stack_new (in /tmp/a.out)
==745==    by 0x40170F: st_thread_create (in /tmp/a.out)
==745==    by 0x4018DD: st_init (in /tmp/a.out)
==745==    by 0x401144: main (in /tmp/a.out)
==745==
==745==
==745== 448 bytes in 1 blocks are still reachable in loss record 2 of 3
==745==    at 0x4C230BC: calloc (vg_replace_malloc.c:397)
==745==    by 0x40192B: st_init (in /tmp/a.out)
==745==    by 0x401144: main (in /tmp/a.out)
==745==
==745==
==745== 12,680 bytes in 1 blocks are still reachable in loss record 3 of 3
==745==    at 0x4C23FAB: malloc (vg_replace_malloc.c:207)
==745==    by 0x403F59: _st_select_init (in /tmp/a.out)
==745==    by 0x4018A5: st_init (in /tmp/a.out)
==745==    by 0x401144: main (in /tmp/a.out)

More arise as I introduce netfds into the program.  As far as I can 
tell, it's not an enormous issue, but is annoying when valgrind is being 
used as a continuous integration tool for monitoring code health.  Any 
fixes?
-- 
Yang Zhang
http://www.mit.edu/~y_z/

Re: [ST-users] System threads (pthreads)

[Mike A. <ya...@ya...>]

> I'm interested in as many details as you can spare.

On 2002-11-26 Gene wrote this:
| I did some digging in the linuxthreads source.  I think the
| culprit is in the inlined function thread_self() which is called
| by __errno_location().  That function relies on stack pointer to
| get pthread's identity.  The "stack pointer" is just an address
| of a local variable:
| 
| #define CURRENT_STACK_FRAME  ({ char __csf; &__csf; })
| char *sp = CURRENT_STACK_FRAME;
| 
| In other words, libpthread assumes that there is only one stack
| per each pthread and thus thread id can be inferred from the
| stack pointer.  If there are stack segments other than created
| by libpthread, thread_self() may return bogus thread id.
| 
| The bottom line, I guess, is that current pthreads implementation
| cannot be mixed with any other thread library.
| 
| --Gene

I remember discussing other issues, in particular a collision on some
special memory location, but I don't recall the details.

Interop between ST and pthreads comes up repeatedly though, so if you
find a fix we'd love to receive it.

Re: [ST-users] System threads (pthreads)

[Yang Z. <yan...@gm...>]

Michael Abbott wrote:
>> Anyone?
> 
> Hmm, I responded earlier today to the list, I wonder if it wound up in
> your spam folder?

Sorry, I did indeed miss the reply.

> 
> The pthreads issue, IIRC, had to do more with clobbering memory.  That
> is, both state threads and pthreads want to use the same memory
> location to store some private data.  We never worked out a way to
> solve it.  (Although I think my idea would work, I haven't tried it. 
> Gene thinks it won't work.)

Why do you want to use the same memory location as pthreads?  I'm 
interested in as many details as you can spare.

What was your idea?

And are the valgrind problems attributed to this same problem of 
conflicting storage location?

Thanks,

Yang

Re: [ST-users] System threads (pthreads)

[Michael A. <ya...@ya...>]

> Anyone?

Hmm, I responded earlier today to the list, I wonder if it wound up in
your spam folder?

The pthreads issue, IIRC, had to do more with clobbering memory.  That
is, both state threads and pthreads want to use the same memory
location to store some private data.  We never worked out a way to
solve it.  (Although I think my idea would work, I haven't tried it. 
Gene thinks it won't work.)

Re: [ST-users] System threads (pthreads)

[Yang Z. <yan...@gm...>]

Anyone?  I'm mainly wondering what were the difficulties people 
encountered in making ST interop with threads (why it's not just a 
matter of interrupting the blocking select() call).

A more intrusive (but potentially more efficient) way to implement 
interop with threads is to use pipes (requires modifying the reactor 
itself; done in Twisted) or use small timeouts on the select (done in 
libasync-smp).

Thanks, for any answers.


Yang Zhang wrote:
> Also, has interop with valgrind been fixed?
> 
> If not, does anybody understand precisely what the problems are with 
> pthreads and valgrind?
> 
> Thanks!
> 
> 
> Yang Zhang wrote:
>> Hi, I've only been able to dig up some old mailing list posts about 
>> system threads (pthreads) not being usable in a st-based program.  Has 
>> there been any progress on this front?
>>
>> One approach suggested by the FAQ is to adopt a multi-process 
>> approach, but IPC overhead (incl. shared memory) is substantial 
>> compared to threads, and the approach generally makes things more 
>> complex.
>>
>> (FWIW, in the past, I've added thread support to other async 
>> programming frameworks by creating a lightweight thread that listens 
>> on a socket for connections from other system threads.)
>>
>> Thanks in advance for any answers.
> 
> 


-- 
Yang Zhang
http://www.mit.edu/~y_z/

Re: [ST-users] System threads (pthreads)

[Michael A. <ya...@ya...>]

As you probably inferred from the lack of response so far, I believe
nobody has yet made State Threads interoperate with pthreads or
valgrind.  But like all open source projects, if you figure these out
we will welcome your code contribution.

Re: [ST-users] System threads (pthreads)

[Yang Z. <yan...@gm...>]

Also, has interop with valgrind been fixed?

If not, does anybody understand precisely what the problems are with 
pthreads and valgrind?

Thanks!


Yang Zhang wrote:
> Hi, I've only been able to dig up some old mailing list posts about 
> system threads (pthreads) not being usable in a st-based program.  Has 
> there been any progress on this front?
> 
> One approach suggested by the FAQ is to adopt a multi-process approach, 
> but IPC overhead (incl. shared memory) is substantial compared to 
> threads, and the approach generally makes things more complex.
> 
> (FWIW, in the past, I've added thread support to other async programming 
> frameworks by creating a lightweight thread that listens on a socket for 
> connections from other system threads.)
> 
> Thanks in advance for any answers.

[ST-users] System threads (pthreads)

[Yang Z. <yan...@gm...>]

Hi, I've only been able to dig up some old mailing list posts about 
system threads (pthreads) not being usable in a st-based program.  Has 
there been any progress on this front?

One approach suggested by the FAQ is to adopt a multi-process approach, 
but IPC overhead (incl. shared memory) is substantial compared to 
threads, and the approach generally makes things more complex.

(FWIW, in the past, I've added thread support to other async programming 
frameworks by creating a lightweight thread that listens on a socket for 
connections from other system threads.)

Thanks in advance for any answers.
-- 
Yang Zhang
http://www.mit.edu/~y_z/

Re: [ST-users] coexist with pthread

[<ge...@co...>]

General answer is "state-threads can't coexist with pthreads". Note that there is no such thing as "generic pthreads". POSIX threads is just an API and the implementation is platform-dependent. For example, on Linux 2.4.* threads are kernel entities and on FreeBSD 4.* pthreads are implemented in user-land (libc_r threads are very similar to state-threads so there is no reason whatsoever to combine them).
So the real question is: why would you want to combine different thread models in a single app?

-------------- Original message -------------- 
From: "Koji Namekata" <nam...@gm...> 

> Hi, 
> 
> Is it inherently impossible that state-threads coexist with pthread? 
> I am trying to execute a test program shown below. Is there anyone to 
> make a similar attempt? 
> 
> ----- 
> Results: 
> 
> Linux 2.4.* Linuxthreads: SEGV 
> FreeBSD 4.* libc_r: Fatal error '_longjmp()ing between thread 
> contexts is undefined by POSIX 1003.1' 
> WinXP Cygwin: Runnable(!) 
> 
> Program: 
> 
> #include 
> #include 
> #include 
> 
> static void *p_thread(void *arg) 
> { 
> while (1) { 
> printf("child pthread [%p]\n", pthread_self()); 
> sleep(5); 
> } 
> return NULL; 
> } 
> 
> static void *s_thread(void *arg) 
> { 
> while (1) { 
> printf("child sthread [%p]\n", st_thread_self()); 
> st_sleep(5); 
> } 
> return NULL; 
> } 
> 
> int main(int argc, char *argv[]) 
> { 
> pthread_t pth; 
> st_thread_t sth1, sth2; 
> 
> pthread_create(&pth, NULL, p_thread, NULL); 
> printf("parent pthread [%p]\n", pthread_self()); 
> 
> st_init(); 
> sth1 = st_thread_create(s_thread, NULL, 0, 0); 
> sth2 = st_thread_create(s_thread, NULL, 0, 0); 
> st_sleep(100); 
> return 0; 
> } 
> ----- 
> 
> Regards, 
> 
> -- 
> Koji Namekata 
> 
> 
> ------------------------------------------------------- 
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language 
> that extends applications into web and mobile media. Attend the live webcast 
> and join the prime developer group breaking into this new coding territory! 
> http://sel.as-us.falkag.net/sel?cmdk&kid0944&bid$1720&dat1642 
> _______________________________________________ 
> State-threads-users mailing list 
> Sta...@li... 
> https://lists.sourceforge.net/lists/listinfo/state-threads-users 

Re: [ST-users] coexist with pthread

[Mike A. <mj...@co...>]

> state-threads coexist with pthread? 

There was some debate on this a while back, with no resolution.  AFAIK
ST and PT can not yet coexist.  We will welcome any patches you
contribute to make them coexist though :).

[ST-users] coexist with pthread

[Koji N. <nam...@gm...>]

Hi,

Is it inherently impossible that state-threads coexist with pthread?=20
I am trying to execute a test program shown below.  Is there anyone to
make a similar attempt?

-----
Results:

   Linux 2.4.* Linuxthreads: SEGV
   FreeBSD 4.* libc_r: Fatal error '_longjmp()ing between thread
contexts is undefined by POSIX 1003.1'
   WinXP Cygwin: Runnable(!)

Program:

#include <stdio.h>
#include <pthread.h>
#include <st.h>

static void *p_thread(void *arg)
{
=09while (1) {
=09=09printf("child pthread [%p]\n", pthread_self());
=09=09sleep(5);
=09}
=09return NULL;
}

static void *s_thread(void *arg)
{
=09while (1) {
=09=09printf("child sthread [%p]\n", st_thread_self());
=09=09st_sleep(5);
=09}
=09return NULL;
}

int main(int argc, char *argv[])
{
=09pthread_t pth;
=09st_thread_t sth1, sth2;

=09pthread_create(&pth, NULL, p_thread, NULL);
=09printf("parent pthread [%p]\n", pthread_self());

=09st_init();
=09sth1 =3D st_thread_create(s_thread, NULL, 0, 0);
=09sth2 =3D st_thread_create(s_thread, NULL, 0, 0);
=09st_sleep(100);
=09return 0;
}
-----

Regards,

--
Koji Namekata

Re: [ST-users] MySQL++ memory corruption, valgrind

[Mike A. <mj...@co...>]

> I'm using the MySQL++ C++ wrapper for the standard mysql library. 
> I am experiencing some memory corruption issues.

Perhaps your mysql client library is compiled for pthreads?  It is well
known that pthreads and state threads do not mix.  Ditto valgrind and
state threads.

Re: [ST-users] c++ exceptions in threads?

[Mike A. <mj...@co...>]

> I wonder, how safe it is to use in threads c++ objects which can throw
> exceptions?

I mix C++ exceptions and State Threads with g++ on FreeBSD, and it works
great.

[ST-users] c++ exceptions in threads?

[Jim S. <jim...@gm...>]

I wonder, how safe it is to use in threads c++ objects which can throw
exceptions? Are i'm asking for trouble when i use in thread start
function c++ exceptions when i catch them also there and do not allow
them to leave the thread start function bondaries?

Happy New Year to you all!