Menu
▾
▴
Re: [Xsb-development] Bug in multi-threaded XSB?
|
From: K. A. <k_a...@ya...> - 2012-08-19 23:06:20
|
Hi Terry,
Thanks for the reply.
>You're absolutely right that sharing tables can
>(and most probably would) lead to problems with
>concurrently executing threads.
>>In principle, it shouldn't. In local evaluation, there is a
>>mechanism to share concurrently evaluated shared tables which
>>I think is described in the manual, and is described in gory
>>detail in Marques and Swift, 2008 ICLP.
It shouldn't, as long as there are no insertions while concurrent
queries are being answered. But if there are such insertions
(as is almost always the case in question-answering systems),
then these will not occur with abolished tables but rather
with whatever tables happen to be in effect at insertion time,
which will in turn depend on the state of the various active
queries at the time. This will most likely invalidate the insertions
and lead to wrong results. It seems to me that what one really
needs is the ability to spawn a query at time t with a *fresh* (empty)
set of tables and with whatever data happens to be in the
database at t. I don't know if, with private predicates/tables,
there is a way to forcibly copy the contents of one thread's
predicates/tables into another. If so, then this could be
readily implemented in XSB as things stand. But if not, then
I don't see how XSB's tabling can be used to implement a
system that does concurrent updates/queries.
>>However if you are using MT XSB, I strongly recommend first
>>testing out the programming idioms you use from the command-line
>>shell, then putting things into the C-XSB interface. XSB's
>>MT C-Prolog interface is quite ambitious, but as a result it
>>may have some undiagnosed bugs. Just as importantly, XSB-only
>>code will be far, far easier to debug.
I'm sure that's the case, but I would think that the point
of having a C<-->XSB interface is to facilitate things that,
for some reason or other, cannot be easily done exclusively
in XSB. If I could program the whole thing in XSB by itself,
I could, but I can't. Moreover, it may well be that some bugs
are peculiar to the C<-->XSB interface and are not reproducible
when the code is expressed purely in XSB. What happens to them?
>So I tried declaring
>tables to be private, *but* it seems you cannot have
>shared dynamic predicates with private tables. The
>predicates themselves (the facts in the database) have
>to be "dynamic as shared", because otherwise a new thread
>spawned to answer a query would not have any usable facts
>to work with. But again, the combination "table p/2 as private"
>with "dynamic p/2 as shared" does not seem to be allowed by XSB
>(perhaps someone can set me straight if I'm missing something).
>>I haven't looked at your code, but in principle you can do this.
>>See the attached file, though I'm sure that your program is much
>>more complex than the attached file.
I didn't see an attached file - could you please resend?
I'd be very interested to see how this could be done.
>At any rate, we're still left with the more narrow question
>of why in the world the query threads in this particular
>example are not exiting when they are explicitly killed.
>>XSB threads should exit when they are killed, although there are cases ?>>where this doesn't happen. For instance if the thread is waiting on I/O, >>or waiting on a mutex, etc. the waiting thread needs to be signaled, to >>wake up, quit doing whatever it is
>>doing, and exit. XSB has a lot of OS interfaces, and I haven't yet >>handled this in every single case. In addition, when a thread is killed >>it must clean up after itself, and free what ever mutexes, db_cursors, >>etc. that it has (but may not be currently working
>> on). While XSB can and does do this for some resources, it doesn't >>always have full knowledge of all of the resources a thread has taken. >>So getting thread cancellation to work in any system is awkward, and >>requires a lot from both the system and the user.
>>In pretty much any MT system, its better to have the thread itself exit, >>and save thread cancellation for special cases.
Well, these are all special cases unfortunately - these threads
*never* exit by themselves because for some reason which I don't
quite understand, they go into a read-eval loop *after* they
are finished evaluating the goal that they were intended
to answer. As far as I can see there is no mechanism in
the C<-->XSB interface to create a thread to do a specific
job and then quit. The only way to do that would be to
execute embedded XSB code using thread_create, but then
keeping track of the thread id and getting the thread
to communicate properly becomes very difficult because,
again, we are not in XSB proper but in the embedded C/XSB
world. So xsb_ccall_thread_create seems to be the only viable
option, but again, these threads don't quit by themselves,
so killing them is the only option (and an absolute must,
because it seems that when multiple threads are active,
tables cannot be abolished, resources cannot be reclaimed,
etc.).
>So if you can show me how your program works from the command-line >interface, I'll help debug at that level. Once we are sure the program is >doing what you want at that level, we can see how to add the C-XSB >interface.
Like I said earlier, this seems to assume that the bug
will be reproducible in pure XSB, and I'm not at all
sure about that. The C code sample is very small so
I hope someone will be able to take a look at it and
reach a (tentative, at least) verdict. In the meanwhile
I'll try to take the client/server I've written in C and
re-express it in pure XSB using the samples given in the
/examples/sockets directory of the distribution. Is there
anyone in particular in this list to whom I should direct
questions pertaining to that client/server code?
Many thanks,
Konstantine
--- On Sat, 8/18/12, David Warren <wa...@cs...> wrote:
> From: David Warren <wa...@cs...>
> Subject: RE: Bug in multi-threaded XSB?
> To: "K. A." <k_a...@ya...>, "Xsb...@li..." <Xsb...@li...>
> Cc: "Terrance Swift" <ts...@cs...>
> Date: Saturday, August 18, 2012, 12:17 PM
> I strongly suspect it is because you
> are sharing the tables and that other thread is not exiting
> (or at least XSB doesn't know it has exited.)
> I would suggest that you don't use shared tables but private
> tables. If they are not causing this problem now, they
> certainly will if you run a multithreaded query service
> where you have a number of queries and some can update the
> underlying data.
> -David
>
> -----Original Message-----
> From: K. A. [mailto:k_a...@ya...]
>
> Sent: Friday, August 17, 2012 5:24 PM
> To: Xsb...@li...
> Cc: David Warren; Terrance Swift
> Subject: Bug in multi-threaded XSB?
>
> Could someone please explain to me why the following C code
> fails to find any answers to the last query in the main
> function - the parentOf(X,Y,L) query?
>
> Note that every insertion does an abolish_all_tables first,
> and that right before each insertion or query in main, there
> should be (as far as I can see) only one XSB thread active -
> the main one.
>
> I assume this is a bug in multi-threaded XSB unless someone
> can provide an alternative explanation.
>
> If you put the C code below in a file test.c, you can make
> the program as follows:
>
> gcc -c -I/home/.../XSB/emu
> -I/home/.../XSB/config/x86_64-unknown-linux-gnu-mt -O3
> -fno-strict-aliasing -Wall -pipe -D_GNU_SOURCE test.c
>
> gcc -o test.out -lm -ldl -Wl -export-dynamic -lpthread
> /home/.../XSB/config/x86_64-unknown-linux-gnu-mt/saved.o/xsb.o
> test.o
>
> #include <stdio.h>
> #include <unistd.h>
> #include <stdlib.h>
> #include <string.h>
> #include <sys/types.h>
> #include <pthread.h>
> #include <ctype.h>
> #include "cinterf.h"
> #include "varstring_xsb.h"
> #include "context.h"
>
> void doInsert(char* insertion_command,th_context* th){ int
> res = xsb_command_string(th, "abolish_all_tables."); res =
> xsb_command_string(th, insertion_command);}
>
> void doCommand(char* cmd,th_context* th){ int res =
> xsb_command_string(th, cmd);}
>
> struct QueryArgs {
> char* query;
> th_context* th;};
>
> void* doQuery(void* args) {
> struct QueryArgs * pt = (struct QueryArgs *) args;
> char* query = (*pt).query;
> th_context* th = (*pt).th;
> XSB_StrDefine(retstr);
> th_context* new_query_thread;
> xsb_ccall_thread_create(th,&new_query_thread);
> int rc =
> xsb_query_string_string(new_query_thread,query,&retstr,"|");
> int answer_count = 0;
> while ((rc == XSB_SUCCESS) && (++answer_count))
> {
> printf("\nAnswer: %s\n",retstr.string);
> rc = xsb_next_string(new_query_thread,
> &retstr,"|");
> }
> printf("\n%d answers for this query:
> %s\n",answer_count,query);
> xsb_kill_thread(new_query_thread);
> return NULL;}
>
> void answerQuery(char* query,th_context* th){
> void* exit_status;
> pthread_t new_thread;
> struct QueryArgs args = {.query = query, .th = th};
> pthread_create(&new_thread,NULL,doQuery,(void*)
> &args);
> pthread_join(new_thread,&exit_status);
> }
>
> int main() {
> char init_string[MAXPATHLEN];
> char* xsbHome = getenv("XSB_HOME");
> strcpy(init_string, xsbHome);
> xsb_init_string(init_string);
> th_context *main_th = xsb_get_main_thread();
>
> doCommand("consult('prelude.P').",xsb_get_main_thread());
> doCommand("load_dyn('main.P').",xsb_get_main_thread());
>
> doInsert("assert(auntOf(woman9,foo,[])).",xsb_get_main_thread());
> answerQuery("familyOf(X,Y,L).",xsb_get_main_thread());
>
> doInsert("assertAll([parentOf(man2,man3,[1/555]),parentOf(man1,man2,[7/982,1/34])]).",xsb_get_main_thread());
> answerQuery("parentOf(X,Y,L).",xsb_get_main_thread());
> xsb_close(xsb_get_main_thread());
> return 0;}
>
> The contents of the other 2 files are as follows:
>
> /************* Contents of prelude.P : ***************/
> assertAll([]).
> assertAll([H|T]) :- asserta(H),assertAll(T).
>
> /************* Contents of main.P : ***************/
> :- import append/3 from basics.
> :- table motherOf/3 as shared.
> :- table fatherOf/3 as shared.
> :- table parentOf/3 as shared.
> :- table ancestorOf/3 as shared.
> :- table auntOf/3 as shared.
> :- table familyOf/3 as shared.
> :- dynamic motherOf/3 as shared.
> :- dynamic fatherOf/3 as shared.
> :- dynamic parentOf/3 as shared.
> :- dynamic ancestorOf/3 as shared.
> :- dynamic auntOf/3 as shared.
> :- dynamic familyOf/3 as shared.
>
> familyOf(X,Y,L) :- ancestorOf(X,Y,L).
> ancestorOf(X,Y,L) :- parentOf(X,Y,L).
> parentOf(X,Y,L) :- motherOf(X,Y,L).
> parentOf(X,Y,L) :- fatherOf(X,Y,L).
> familyOf(X,Y,L) :- auntOf(X,Y,L).
> familyOf(X,Y,L) :- familyOf(Y,X,L).
> ancestorOf(X,Y,L) :- ancestorOf(X,Z,L1), ancestorOf(Z,Y,L2),
> append(L1,L2,L).
>
>
|