From: K. A. <k_a...@ya...> - 2012-08-21 02:08:15
|
Hi Terry, Thanks for the suggestion. One question: Can a private thread table be explicitly abolished on that thread while other threads are running? If so, then your suggestion could work with the unkillable threads created by xsb_ccall_thread_create - I can maintain a queue of 20 or so such threads and reuse them to answer queries, since I can't kill them. (This would obviously be unworkable in general, but in this case it's unlikely that I'll have more than 20 concurrent queries at any given time, so it might be viable.) But this presupposes that I can abolish their private tables while other threads are running. If not, then I'd have to continually spawn new threads to answer new queries, and if these threads never quit then I'll be out of memory before too long. Thanks, Konstantine --- On Mon, 8/20/12, Terrance Swift <ts...@cs...> wrote: > From: Terrance Swift <ts...@cs...> > Subject: RE: Bug in multi-threaded XSB? > To: "K. A." <k_a...@ya...>, "Xsb...@li..." <Xsb...@li...>, "David Warren" <wa...@cs...> > Date: Monday, August 20, 2012, 9:34 AM > Here is the simple attached file. > > A couple of people have written large applications with > MT-XSB but they probably did not use all the features you > are trying to use, and MT-XSB is certainly less stable than > single-threaded XSB. I wish I had sufficient time to > help every user, but looking very briefly at your code, its > impossible to understand. > > 1) Why you can't remove the C/XSB interface, at least for > debugging > 2) Why you can't use thread exiting, at least for debugging > (actually its not an issue from the prolog level, threads > exit once they have been joined.) > 3) How many of the problems in XSB and how many are > multi-programming errors of yours. No offense, > everybody makes MT-programming errors. > > I hope you'll take seriously the suggestions I gave you to > start breaking down the program to help isolate the > bugs. I'll try to fix the bugs I can once they become > clear to me. > > Terry > ________________________________________ > From: K. A. [k_a...@ya...] > Sent: Sunday, August 19, 2012 7:06 PM > To: Xsb...@li...; > David Warren; Terrance Swift > Subject: RE: Bug in multi-threaded XSB? > > Hi Terry, > > Thanks for the reply. > > >You're absolutely right that sharing tables can > >(and most probably would) lead to problems with > >concurrently executing threads. > > >>In principle, it shouldn't. In local > evaluation, there is a > >>mechanism to share concurrently evaluated shared > tables which > >>I think is described in the manual, and is described > in gory > >>detail in Marques and Swift, 2008 ICLP. > > It shouldn't, as long as there are no insertions while > concurrent > queries are being answered. But if there are such > insertions > (as is almost always the case in question-answering > systems), > then these will not occur with abolished tables but rather > with whatever tables happen to be in effect at insertion > time, > which will in turn depend on the state of the various > active > queries at the time. This will most likely invalidate the > insertions > and lead to wrong results. It seems to me that what one > really > needs is the ability to spawn a query at time t with a > *fresh* (empty) > set of tables and with whatever data happens to be in the > database at t. I don't know if, with private > predicates/tables, > there is a way to forcibly copy the contents of one > thread's > predicates/tables into another. If so, then this could be > readily implemented in XSB as things stand. But if not, > then > I don't see how XSB's tabling can be used to implement a > system that does concurrent updates/queries. > > >>However if you are using MT XSB, I strongly > recommend first > >>testing out the programming idioms you use from the > command-line > >>shell, then putting things into the C-XSB > interface. XSB's > >>MT C-Prolog interface is quite ambitious, but as a > result it > >>may have some undiagnosed bugs. Just as > importantly, XSB-only > >>code will be far, far easier to debug. > > I'm sure that's the case, but I would think that the point > of having a C<-->XSB interface is to facilitate things > that, > for some reason or other, cannot be easily done exclusively > in XSB. If I could program the whole thing in XSB by > itself, > I could, but I can't. Moreover, it may well be that some > bugs > are peculiar to the C<-->XSB interface and are not > reproducible > when the code is expressed purely in XSB. What happens to > them? > > >So I tried declaring > >tables to be private, *but* it seems you cannot have > >shared dynamic predicates with private tables. The > >predicates themselves (the facts in the database) have > >to be "dynamic as shared", because otherwise a new > thread > >spawned to answer a query would not have any usable > facts > >to work with. But again, the combination "table p/2 as > private" > >with "dynamic p/2 as shared" does not seem to be allowed > by XSB > >(perhaps someone can set me straight if I'm missing > something). > > >>I haven't looked at your code, but in principle you > can do this. > >>See the attached file, though I'm sure that your > program is much > >>more complex than the attached file. > > I didn't see an attached file - could you please resend? > I'd be very interested to see how this could be done. > > >At any rate, we're still left with the more narrow > question > >of why in the world the query threads in this > particular > >example are not exiting when they are explicitly > killed. > > >>XSB threads should exit when they are killed, > although there are cases ?>>where this doesn't > happen. For instance if the thread is waiting on I/O, > >>or waiting on a mutex, etc. the waiting thread needs > to be signaled, to >>wake up, quit doing whatever it > is > >>doing, and exit. XSB has a lot of OS > interfaces, and I haven't yet >>handled this in every > single case. In addition, when a thread is killed > >>it must clean up after itself, and free what ever > mutexes, db_cursors, >>etc. that it has (but may not > be currently working > >> on). While XSB can and does do this for some > resources, it doesn't >>always have full knowledge of > all of the resources a thread has taken. >>So > getting thread cancellation to work in any system is > awkward, and >>requires a lot from both the system and > the user. > >>In pretty much any MT system, its better to have the > thread itself exit, >>and save thread cancellation for > special cases. > > Well, these are all special cases unfortunately - these > threads > *never* exit by themselves because for some reason which I > don't > quite understand, they go into a read-eval loop *after* > they > are finished evaluating the goal that they were intended > to answer. As far as I can see there is no mechanism in > the C<-->XSB interface to create a thread to do a > specific > job and then quit. The only way to do that would be to > execute embedded XSB code using thread_create, but then > keeping track of the thread id and getting the thread > to communicate properly becomes very difficult because, > again, we are not in XSB proper but in the embedded C/XSB > world. So xsb_ccall_thread_create seems to be the only > viable > option, but again, these threads don't quit by themselves, > so killing them is the only option (and an absolute must, > because it seems that when multiple threads are active, > tables cannot be abolished, resources cannot be reclaimed, > etc.). > > >So if you can show me how your program works from the > command-line >interface, I'll help debug at that > level. Once we are sure the program is >doing what > you want at that level, we can see how to add the C-XSB > >interface. > > Like I said earlier, this seems to assume that the bug > will be reproducible in pure XSB, and I'm not at all > sure about that. The C code sample is very small so > I hope someone will be able to take a look at it and > reach a (tentative, at least) verdict. In the meanwhile > I'll try to take the client/server I've written in C and > re-express it in pure XSB using the samples given in the > /examples/sockets directory of the distribution. Is there > anyone in particular in this list to whom I should direct > questions pertaining to that client/server code? > > Many thanks, > > Konstantine > > > --- On Sat, 8/18/12, David Warren <wa...@cs...> > wrote: > > > > > From: David Warren <wa...@cs...> > > > Subject: RE: Bug in multi-threaded XSB? > > > To: "K. A." <k_a...@ya...>, > "Xsb...@li..." > <Xsb...@li...> > > > Cc: "Terrance Swift" <ts...@cs...> > > > Date: Saturday, August 18, 2012, 12:17 PM > > > I strongly suspect it is because you > > > are sharing the tables and that other thread is not > exiting > > > (or at least XSB doesn't know it has exited.) > > > I would suggest that you don't use shared tables but > private > > > tables. If they are not causing this problem now, they > > > certainly will if you run a multithreaded query > service > > > where you have a number of queries and some can update > the > > > underlying data. > > > -David > > > > > > -----Original Message----- > > > From: K. A. [mailto:k_a...@ya...] > > > > > > Sent: Friday, August 17, 2012 5:24 PM > > > To: Xsb...@li... > > > Cc: David Warren; Terrance Swift > > > Subject: Bug in multi-threaded XSB? > > > > > > Could someone please explain to me why the following C > code > > > fails to find any answers to the last query in the > main > > > function - the parentOf(X,Y,L) query? > > > > > > Note that every insertion does an abolish_all_tables > first, > > > and that right before each insertion or query in main, > there > > > should be (as far as I can see) only one XSB thread > active - > > > the main one. > > > > > > I assume this is a bug in multi-threaded XSB unless > someone > > > can provide an alternative explanation. > > > > > > If you put the C code below in a file test.c, you can > make > > > the program as follows: > > > > > > gcc -c -I/home/.../XSB/emu > > > -I/home/.../XSB/config/x86_64-unknown-linux-gnu-mt -O3 > > > -fno-strict-aliasing -Wall -pipe -D_GNU_SOURCE test.c > > > > > > gcc -o test.out -lm -ldl -Wl -export-dynamic -lpthread > > > > /home/.../XSB/config/x86_64-unknown-linux-gnu-mt/saved.o/xsb.o > > > test.o > > > > > > #include <stdio.h> > > > #include <unistd.h> > > > #include <stdlib.h> > > > #include <string.h> > > > #include <sys/types.h> > > > #include <pthread.h> > > > #include <ctype.h> > > > #include "cinterf.h" > > > #include "varstring_xsb.h" > > > #include "context.h" > > > > > > void doInsert(char* insertion_command,th_context* th){ > int > > > res = xsb_command_string(th, "abolish_all_tables."); > res = > > > xsb_command_string(th, insertion_command);} > > > > > > void doCommand(char* cmd,th_context* th){ int res = > > > xsb_command_string(th, cmd);} > > > > > > struct QueryArgs { > > > char* query; > > > th_context* th;}; > > > > > > void* doQuery(void* args) { > > > struct QueryArgs * pt = (struct QueryArgs *) args; > > > char* query = (*pt).query; > > > th_context* th = (*pt).th; > > > XSB_StrDefine(retstr); > > > th_context* new_query_thread; > > > xsb_ccall_thread_create(th,&new_query_thread); > > > int rc = > > > > xsb_query_string_string(new_query_thread,query,&retstr,"|"); > > > int answer_count = 0; > > > while ((rc == XSB_SUCCESS) && > (++answer_count)) > > > { > > > printf("\nAnswer: %s\n",retstr.string); > > > rc = xsb_next_string(new_query_thread, > > > &retstr,"|"); > > > } > > > printf("\n%d answers for this query: > > > %s\n",answer_count,query); > > > xsb_kill_thread(new_query_thread); > > > return NULL;} > > > > > > void answerQuery(char* query,th_context* th){ > > > void* exit_status; > > > pthread_t new_thread; > > > struct QueryArgs args = {.query = query, .th = th}; > > > pthread_create(&new_thread,NULL,doQuery,(void*) > > > &args); > > > pthread_join(new_thread,&exit_status); > > > } > > > > > > int main() { > > > char init_string[MAXPATHLEN]; > > > char* xsbHome = getenv("XSB_HOME"); > > > strcpy(init_string, xsbHome); > > > xsb_init_string(init_string); > > > th_context *main_th = xsb_get_main_thread(); > > > > > > > doCommand("consult('prelude.P').",xsb_get_main_thread()); > > > > doCommand("load_dyn('main.P').",xsb_get_main_thread()); > > > > > > > doInsert("assert(auntOf(woman9,foo,[])).",xsb_get_main_thread()); > > > answerQuery("familyOf(X,Y,L).",xsb_get_main_thread()); > > > > > > > doInsert("assertAll([parentOf(man2,man3,[1/555]),parentOf(man1,man2,[7/982,1/34])]).",xsb_get_main_thread()); > > > answerQuery("parentOf(X,Y,L).",xsb_get_main_thread()); > > > xsb_close(xsb_get_main_thread()); > > > return 0;} > > > > > > The contents of the other 2 files are as follows: > > > > > > /************* Contents of prelude.P : > ***************/ > > > assertAll([]). > > > assertAll([H|T]) :- asserta(H),assertAll(T). > > > > > > /************* Contents of main.P : ***************/ > > > :- import append/3 from basics. > > > :- table motherOf/3 as shared. > > > :- table fatherOf/3 as shared. > > > :- table parentOf/3 as shared. > > > :- table ancestorOf/3 as shared. > > > :- table auntOf/3 as shared. > > > :- table familyOf/3 as shared. > > > :- dynamic motherOf/3 as shared. > > > :- dynamic fatherOf/3 as shared. > > > :- dynamic parentOf/3 as shared. > > > :- dynamic ancestorOf/3 as shared. > > > :- dynamic auntOf/3 as shared. > > > :- dynamic familyOf/3 as shared. > > > > > > familyOf(X,Y,L) :- ancestorOf(X,Y,L). > > > ancestorOf(X,Y,L) :- parentOf(X,Y,L). > > > parentOf(X,Y,L) :- motherOf(X,Y,L). > > > parentOf(X,Y,L) :- fatherOf(X,Y,L). > > > familyOf(X,Y,L) :- auntOf(X,Y,L). > > > familyOf(X,Y,L) :- familyOf(Y,X,L). > > > ancestorOf(X,Y,L) :- ancestorOf(X,Z,L1), > ancestorOf(Z,Y,L2), > > > append(L1,L2,L). > > > > > > > > > > > > > |