From: Alan W. I. <ir...@be...> - 2016-02-22 22:48:58
|
@Phil: I am particularly interested in your reaction here because you had the idea before that you could implement PLplot in a thread-safe way by using C++ as the core language, i.e., rewriting PLplot in C++. I don't rule out the possibility, but one intermediate step toward that goal might be to implement the idea below in C, and and then change our C++ binding so that it automatically uses a PLplotContext (which would presumably go a long way toward solving your current issues with PLplot thread safety and wxwidgets). @Everybody: now on to my C idea for thread safety. The PLplot core C library is currently not thread safe, and I think we are all agreed that it is important to address that issue since it is an important barrier to entry for some. For example, if I recall correctly it was Ruby on Rails developers who publicly expressed that they wanted no part of PLplot because of this thread safety issue, and presumably other library developers are silently avoiding PLplot for this same reason. In addition, once it becomes possible to use PLplot in a thread-safe way, it should make life much easier for Phil's development work on the wxwidgets device. We would go a long way toward thread safety if we got rid of all static variables so the rest of this post focuses on that issue from the C perspective. My idea for implementing that (closely following what was done for the C ephcom library case where David Howells implemented an ephcom context to help provide an API that did not depend on static variables) is to eliminate all static variables by using a PLplotContext struct that contains all data that is currently stored as static variables. Once that is implemented then the proper non-static way to use PLplot would be to do something like the following: PLplotContext * context; context = CreatePLplotContext(); plparseopts(..., context); plinit(context); // other ordinary PLplot calls as usual but with // context as last argument, e.g., plline(..., context); plend(context); where CreatePLplotContext would malloc a PLplotContext, and all the PLplot API calls would refer to that extra context argument whenever they needed access to any part of what were previously static variables (e.g., such as plsc). The context-sensitive plend would do everything that plend currently does plus destroy the context by freeing it. So far this follows pretty exactly what David Howells implemented for ephcom. Does everybody agree this general idea (a comprehensive API change where a context argument was added to every function call) would go a long way toward making PLplot thread safe? We found in the ephcom case that both the Python and Fortran bindings for ephcom could pass the C pointer to a context as arguments. So we could implement those bindings in a non-static way using the ephcom equivalent of CreatePLplotContext above. But we retained the static version of the API just in case some future binding of ephcom was for a language that could not pass C pointer arguments, and we would also want to retain the static API for similar reasons in the PLplot case. For static versions of plinit, plinit variants (e.g., plstar), and/or all PLplot routines that can legitimately be called before plinit would call CreatePLplotContext internally if the static variable PLplotContextAddress was non-NULL and store that pointer in PLplotContextAddress which would be referenced by every static PLplot routine (but PLplotContextAddress would be completely ignored by the non-static API). For backwards compatibility (e.g., to support those who don't care about thread safety and who do not want to change their code) we would want to retain the same name for the static API that are used now. But I would like to use the same names for the non-static cases as well if that is possible (say by following the approach discussed at <http://stackoverflow.com/questions/1472138/c-default-arguments>.) Further discussion is encouraged and welcome! Also, I am well aware I have glossed over lots of details here. That is because I frankly don't completely understand all those details! :-) Nevertheless, assuming I have expressed the overview correctly of what would be required, I hope someone will be inspired by that overview to go ahead and implement the non-static C alternative as a very large step toward the ability to use our library in a thread-safe way. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ ---------- Forwarded message ---------- Date: Tue, 10 Dec 2002 02:52:43 -0600 (CST) From: Maurice LeBrun <mj...@ga...> To: Alan W. Irwin <ir...@be...> Cc: PLplot development list <Plp...@li...> Subject: Re: [Plplot-devel] plinit, plend, plinit sequence now works, but I am having second thoughts I can't tell you the specific answer to your questions due to how maniacally busy I am these days (having just joined Lightspeed Semiconductor), but I can elucidate some of the plplot design ideas that have historically been un-documented. And, I can give it in an object-oriented context, which (because it is "canonical") is a lot nicer than the "this is the way it should work" approach I've used historically. :) This also includes proposals for change to how we do it now -- i.e. the behavior of plinit(). I've always been somewhat bothered by the way stream 0 vs stream N is handled (this bugged me back in '94 but I wasn't exactly swimming in free time then either). When plplot starts, you have the statically pre-allocated stream, stream 0. Yes the stream 0 that I hate b/c it's not allocated on the heap like a proper data-structure/object (in the plframe widget I automatically start from stream 1). So stream 0 is like a class definition and an instance rolled into one. What I think we need to do is get rid of the "instance" part of this and leave stream 0 as a "class definition" only. In this case all the command line arguments and initial pls...() calls (before plinit) serve to override the default initializion of the class variables, i.e. set stream 0 parameters only In other words, stream 0 becomes the template for all plplot streams. You can use it, but once you've called plinit() you have your own "instance" of the plplot "object" -- i.e. you have a new stream with all the relevant state info copied from stream 0. If you change it, it dies when your stream dies with plend1(). If you really want to change stream 0 (i.e. the "plplot object" "class data") you can always set your stream number to 0 and fire away. To summarize: plinit() creates a new stream, copied from stream 0 plend1() deletes that stream plend() deletes all streams, except of course stream 0 which is "class data" Let me know if any of this helps. -- Maurice LeBrun mj...@ga... Research Organization for Information Science and Technology of Japan (RIST) |
From: Jim D. <ji...@di...> - 2016-02-23 06:07:36
|
> On Feb 22, 2016, at 5:48 PM, Alan W. Irwin <ir...@be...> wrote: > > @Everybody: now on to my C idea for thread safety. > > My idea for implementing that (closely following what was done for the > C ephcom library case where David Howells implemented an ephcom > context to help provide an API that did not depend on static > variables) is to eliminate all static variables by using a > PLplotContext struct that contains all data that is currently stored > as static variables. > > Once that is implemented then the proper non-static way to use PLplot > would be to do something like the following: > > PLplotContext * context; > > context = CreatePLplotContext(); > plparseopts(..., context); > plinit(context); > // other ordinary PLplot calls as usual but with > // context as last argument, e.g., > plline(..., context); > > plend(context); > > where CreatePLplotContext would malloc a PLplotContext, and all the > PLplot API calls would refer to that extra context argument whenever > they needed access to any part of what were previously static variables > (e.g., such as plsc). > > The context-sensitive plend would do everything that plend currently does plus > destroy the context by freeing it. > > So far this follows pretty exactly what David Howells implemented for > ephcom. Does everybody agree this general idea (a comprehensive API > change where a context argument was added to every function call) would > go a long way toward making PLplot thread safe? > I think that is a good approach. I have done something similar in a mixed C/Fortran environment and it appeared to work, though I did not rigorously test the implementation. The hard part will be chasing down all the static allocations—they abound in the string handling. I had a patch that I put together that removed all the static char arrays used to format information/error messages, but it never made it into the code. I still have the patch and can update and push it to repository now that i can commit. > We found in the ephcom case that both the Python and Fortran bindings > for ephcom could pass the C pointer to a context as arguments. So we > could implement those bindings in a non-static way using the ephcom > equivalent of CreatePLplotContext above. But we retained the static > version of the API just in case some future binding of ephcom was for > a language that could not pass C pointer arguments, and we would also > want to retain the static API for similar reasons in the PLplot case. > > For static versions of plinit, plinit variants (e.g., plstar), and/or > all PLplot routines that can legitimately be called before plinit > would call CreatePLplotContext internally if the static variable > PLplotContextAddress was non-NULL and store that pointer in > PLplotContextAddress which would be referenced by every static PLplot > routine (but PLplotContextAddress would be completely ignored by the > non-static API). > I seem to recall there is a way to get per-thread initialization of variables in C. Is that the direction you want to go or do you want the context address to be shared by all threads? > For backwards compatibility (e.g., to support those who don't care > about thread safety and who do not want to change their code) we would > want to retain the same name for the static API that are used > now. But I would like to use the same names for the non-static cases > as well if that is possible (say by following the approach discussed > at <http://stackoverflow.com/questions/1472138/c-default-arguments>.) > > Further discussion is encouraged and welcome! Also, I am well aware I > have glossed over lots of details here. That is because I frankly > don't completely understand all those details! :-) Nevertheless, > assuming I have expressed the overview correctly of what would be > required, I hope someone will be inspired by that overview to go ahead > and implement the non-static C alternative as a very large step toward > the ability to use our library in a thread-safe way. > > Alan > __________________________ > Alan W. Irwin > > Astronomical research affiliation with Department of Physics and Astronomy, > University of Victoria (astrowww.phys.uvic.ca). > > Programming affiliations with the FreeEOS equation-of-state > implementation for stellar interiors (freeeos.sf.net); the Time > Ephemerides project (timeephem.sf.net); PLplot scientific plotting > software package (plplot.sf.net); the libLASi project > (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); > and the Linux Brochure Project (lbproject.sf.net). > __________________________ > > Linux-powered Science > __________________________ > > ---------- Forwarded message ---------- > Date: Tue, 10 Dec 2002 02:52:43 -0600 (CST) > From: Maurice LeBrun <mj...@ga...> > To: Alan W. Irwin <ir...@be...> > Cc: PLplot development list <Plp...@li...> > Subject: Re: [Plplot-devel] plinit, plend, plinit sequence now works, > but I am having second thoughts > > I can't tell you the specific answer to your questions due to how maniacally > busy I am these days (having just joined Lightspeed Semiconductor), but I can > elucidate some of the plplot design ideas that have historically been > un-documented. And, I can give it in an object-oriented context, which > (because it is "canonical") is a lot nicer than the "this is the way it should > work" approach I've used historically. :) > > This also includes proposals for change to how we do it now -- i.e. the > behavior of plinit(). I've always been somewhat bothered by the way stream 0 > vs stream N is handled (this bugged me back in '94 but I wasn't exactly > swimming in free time then either). > > When plplot starts, you have the statically pre-allocated stream, stream 0. > Yes the stream 0 that I hate b/c it's not allocated on the heap like a proper > data-structure/object (in the plframe widget I automatically start from stream > 1). > > So stream 0 is like a class definition and an instance rolled into one. What > I think we need to do is get rid of the "instance" part of this and leave > stream 0 as a "class definition" only. In this case all the command line > arguments and initial pls...() calls (before plinit) serve to override the > default initializion of the class variables, i.e. set stream 0 parameters only > > In other words, stream 0 becomes the template for all plplot streams. You can > use it, but once you've called plinit() you have your own "instance" of the > plplot "object" -- i.e. you have a new stream with all the relevant state info > copied from stream 0. If you change it, it dies when your stream dies with > plend1(). > > If you really want to change stream 0 (i.e. the "plplot object" "class data") > you can always set your stream number to 0 and fire away. > > To summarize: > > plinit() creates a new stream, copied from stream 0 > plend1() deletes that stream > plend() deletes all streams, except of course stream 0 which is "class data" > > Let me know if any of this helps. > > -- > Maurice LeBrun mj...@ga... > Research Organization for Information Science and Technology of Japan (RIST) > > ------------------------------------------------------------------------------ > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > _______________________________________________ > Plplot-devel mailing list > Plp...@li... > https://lists.sourceforge.net/lists/listinfo/plplot-devel |
From: Alan W. I. <ir...@be...> - 2016-02-23 18:10:51
|
On 2016-02-23 00:50-0500 Jim Dishaw wrote: > >> On Feb 22, 2016, at 5:48 PM, Alan W. Irwin <ir...@be...> wrote: >> >> @Everybody: now on to my C idea for thread safety. >> >> My idea for implementing that (closely following what was done for the >> C ephcom library case where David Howells implemented an ephcom >> context to help provide an API that did not depend on static >> variables) is to eliminate all static variables by using a >> PLplotContext struct that contains all data that is currently stored >> as static variables. >> >> Once that is implemented then the proper non-static way to use PLplot >> would be to do something like the following: >> >> PLplotContext * context; >> >> context = CreatePLplotContext(); >> plparseopts(..., context); >> plinit(context); >> // other ordinary PLplot calls as usual but with >> // context as last argument, e.g., >> plline(..., context); >> >> plend(context); >> >> where CreatePLplotContext would malloc a PLplotContext, and all the >> PLplot API calls would refer to that extra context argument whenever >> they needed access to any part of what were previously static variables >> (e.g., such as plsc). >> >> The context-sensitive plend would do everything that plend currently does plus >> destroy the context by freeing it. >> >> So far this follows pretty exactly what David Howells implemented for >> ephcom. Does everybody agree this general idea (a comprehensive API >> change where a context argument was added to every function call) would >> go a long way toward making PLplot thread safe? >> > > I think that is a good approach. I have done something similar in a mixed C/Fortran environment and it appeared to work, though I did not rigorously test the implementation. The hard part will be chasing down all the static allocations—they abound in the string handling. I had a patch that I put together that removed all the static char arrays used to format information/error messages, but it never made it into the code. I still have the patch and can update and push it to repository now that i can commit. > >> We found in the ephcom case that both the Python and Fortran bindings >> for ephcom could pass the C pointer to a context as arguments. So we >> could implement those bindings in a non-static way using the ephcom >> equivalent of CreatePLplotContext above. But we retained the static >> version of the API just in case some future binding of ephcom was for >> a language that could not pass C pointer arguments, and we would also >> want to retain the static API for similar reasons in the PLplot case. >> >> For static versions of plinit, plinit variants (e.g., plstar), and/or >> all PLplot routines that can legitimately be called before plinit >> would call CreatePLplotContext internally if the static variable >> PLplotContextAddress was non-NULL and store that pointer in >> PLplotContextAddress which would be referenced by every static PLplot >> routine (but PLplotContextAddress would be completely ignored by the >> non-static API). >> > > I seem to recall there is a way to get per-thread initialization of variables in C. Is that the direction you want to go or do you want the context address to be shared by all threads? Thanks for that idea which I frankly had never heard of before, but now you have drawn my attention to that possibility (e.g., <https://en.wikipedia.org/wiki/Thread-local_storage>) it would be a vast simplification since we would no longer need to carry the context argument to each routine because instead you could store context as a static thread-local variable. In other words, if I am interpreting what that article said properly, what I have described as the static case could be made thread safe by simply changing the static variable PLplotContextAddress to a static thread-local variable. One complication for this exciting prospect for PLplot thread safety is according to the article above this C capability was only introduced in C11. So it is apparently going to take a while for C compilers to support thread-local capability in a standard way. For example, according to <https://gcc.gnu.org/onlinedocs/gcc-3.3/gcc/Thread-Local.html> gcc does support thread local storage but currently in a non-standard way using the "__thread" attribute. On my system the following compiles without issues for gcc-4.9.2: irwin@raven> cat test_thread_local.c static __thread int foo = 0; irwin@raven> gcc -c test_thread_local.c -o test.o So that support for the __thread attribute looks promising. Anyhow, I think the next step forward is to follow up with the "static" implementation I described above with PLplotContextAddress initially declared for testing purposes as static __thread PLplotContext * PLplotContextAddress; for gcc (4.9.2 and above) and static PLplotContext * PLplotContextAddress; otherwise. I hope someone here is keen to give this idea a try. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Phil R. <p.d...@gm...> - 2016-02-23 11:22:00
|
Hi Alan and Jim I entirely advocate this. This is the same model that libcurl uses too. In libcurl the "context" variable is a typecast void* so is entirely opaque to the user, but it gets cast to a structure internally. I presume the idea would be that the user would use one context per thread? There are two other items highly related items that go hand in hand with this and they are error reporting and removing the exit calls. Actually this is where I feel C++ has an advantage over C, but things can be done in C too. Basically when we hit an error that we cannot deal with such as a memory allocation fail, we need to return all the way back to the API entry point and somehow make an error code available to the user. At the same time we need to make sure we don't generate a segfault by accessing our failed allocation and we need to free any memory along the way to avoid memory leaks. The naïve, error prone and extremely labour intensive way to do this is to return error codes from all our internal functions, make sure we check them and do clean up at every stage. This will never ever work. The library is just too complex. C++ does this very well. We can use an array class for our memory allocations which has a destructor that frees the memory. This means that when an array goes out of scope the memory is automatically freed. C++ also has an error reporting mechanism called exception throwing. If we have code which can throw an exception then we put it inside a special block called a try-catch block. If at any point no matter how many layers down the function call stack, the code hits an error that it cannot deal with it throws an exception using the throw() function. At this point the code will return all the way back to the try catch block and execute the code in the catch section. Otherwise the catch code is ignored. The amazing thing about this is that along the return path all scopes are exited "cleanly" causing the destructors of all objects to be called and freeing all memory. This is an incredibly clean way to do things. However if we wish to stick with C, which is probably a valid thing to do, then there are some similar things we can do. Instead of exception throwing C has longjmp(), which does a similar thing. The main problem is that we still need to free our memory. We can do this by creating a plmalloc and plfree function which will keep track of what memory is allocated/deallocated. If long_jmp gets called part of the error handling that we will do is check if any memory is still allocated and deallocate it. From my understanding this longjmp and memory pool model is pretty standard in C error reporting. Here is an example of how it would work: void c_plline( PLINT n, const PLFLT *x, const PLFLT *y, PLplotContext context ) { plstream *plsc = (plstream *)context; int val; val = setjmp( plsc->env ); //this returns 0, however if we call longjmp //using the same value for env, we return //here (a bit like a goto with a nonzero return value. if( val == 0 ) //run our code if we haven't had a longjmp call { if ( plsc->level < 3 ) { longjmp( PLERR_INITLEVEL3, plsc->env ); } plP_drawor_poly( x, y, n, plsc ); //somewhere in the depths here we could call longjmp and //we would end up back at the setjmp call } //do cleanup, just in case we forgot to do it in the code and save our error code and perhaps report //an error message plfreeall(); reporterror( val, plsc ); //this could do any number of things } Things do get a little more complicated than this, for example if internally we call an API function we need to push the env variable onto a stack to avoid overwriting it and we must avoid longjmping over C++ code otherwise we get undefined behaviour. There are some libraries that implement all this already, e.g. https://github.com/guillermocalvo/exceptions4c So in summary: Pass in a context for helping thread safety - definitely Decide how we wish to report an error, could be return val, callback or a plgeterr( context ) call. If we want to stick with C: Create a memory pool for allocating memory and ensuring we avoid memory leaks Use longjmp for reporting errors back to the API entry point, but bear in mind issues with jumping over C++ code If we would be interested in using C++: use an array object for memory allocation and automatic freeing use exceptions to report errors back to the API entry point This would be much much much (I could add a lot of muches here) more robust than doing things with C, but it does require a philosophical change to how plplot is written. My personal view is that the error reporting and removing exit calls is actually more important than thread safety. Note that even if we decide to use C++ internally we must maintain a C interface. C++ at a library interface can be very bad. Phil On 23 February 2016 at 05:50, Jim Dishaw <ji...@di...> wrote: > >> On Feb 22, 2016, at 5:48 PM, Alan W. Irwin <ir...@be...> wrote: >> >> @Everybody: now on to my C idea for thread safety. >> >> My idea for implementing that (closely following what was done for the >> C ephcom library case where David Howells implemented an ephcom >> context to help provide an API that did not depend on static >> variables) is to eliminate all static variables by using a >> PLplotContext struct that contains all data that is currently stored >> as static variables. >> >> Once that is implemented then the proper non-static way to use PLplot >> would be to do something like the following: >> >> PLplotContext * context; >> >> context = CreatePLplotContext(); >> plparseopts(..., context); >> plinit(context); >> // other ordinary PLplot calls as usual but with >> // context as last argument, e.g., >> plline(..., context); >> >> plend(context); >> >> where CreatePLplotContext would malloc a PLplotContext, and all the >> PLplot API calls would refer to that extra context argument whenever >> they needed access to any part of what were previously static variables >> (e.g., such as plsc). >> >> The context-sensitive plend would do everything that plend currently does plus >> destroy the context by freeing it. >> >> So far this follows pretty exactly what David Howells implemented for >> ephcom. Does everybody agree this general idea (a comprehensive API >> change where a context argument was added to every function call) would >> go a long way toward making PLplot thread safe? >> > > I think that is a good approach. I have done something similar in a mixed C/Fortran environment and it appeared to work, though I did not rigorously test the implementation. The hard part will be chasing down all the static allocations—they abound in the string handling. I had a patch that I put together that removed all the static char arrays used to format information/error messages, but it never made it into the code. I still have the patch and can update and push it to repository now that i can commit. > >> We found in the ephcom case that both the Python and Fortran bindings >> for ephcom could pass the C pointer to a context as arguments. So we >> could implement those bindings in a non-static way using the ephcom >> equivalent of CreatePLplotContext above. But we retained the static >> version of the API just in case some future binding of ephcom was for >> a language that could not pass C pointer arguments, and we would also >> want to retain the static API for similar reasons in the PLplot case. >> >> For static versions of plinit, plinit variants (e.g., plstar), and/or >> all PLplot routines that can legitimately be called before plinit >> would call CreatePLplotContext internally if the static variable >> PLplotContextAddress was non-NULL and store that pointer in >> PLplotContextAddress which would be referenced by every static PLplot >> routine (but PLplotContextAddress would be completely ignored by the >> non-static API). >> > > I seem to recall there is a way to get per-thread initialization of variables in C. Is that the direction you want to go or do you want the context address to be shared by all threads? > >> For backwards compatibility (e.g., to support those who don't care >> about thread safety and who do not want to change their code) we would >> want to retain the same name for the static API that are used >> now. But I would like to use the same names for the non-static cases >> as well if that is possible (say by following the approach discussed >> at <http://stackoverflow.com/questions/1472138/c-default-arguments>.) >> >> Further discussion is encouraged and welcome! Also, I am well aware I >> have glossed over lots of details here. That is because I frankly >> don't completely understand all those details! :-) Nevertheless, >> assuming I have expressed the overview correctly of what would be >> required, I hope someone will be inspired by that overview to go ahead >> and implement the non-static C alternative as a very large step toward >> the ability to use our library in a thread-safe way. >> >> Alan >> __________________________ >> Alan W. Irwin >> >> Astronomical research affiliation with Department of Physics and Astronomy, >> University of Victoria (astrowww.phys.uvic.ca). >> >> Programming affiliations with the FreeEOS equation-of-state >> implementation for stellar interiors (freeeos.sf.net); the Time >> Ephemerides project (timeephem.sf.net); PLplot scientific plotting >> software package (plplot.sf.net); the libLASi project >> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); >> and the Linux Brochure Project (lbproject.sf.net). >> __________________________ >> >> Linux-powered Science >> __________________________ >> >> ---------- Forwarded message ---------- >> Date: Tue, 10 Dec 2002 02:52:43 -0600 (CST) >> From: Maurice LeBrun <mj...@ga...> >> To: Alan W. Irwin <ir...@be...> >> Cc: PLplot development list <Plp...@li...> >> Subject: Re: [Plplot-devel] plinit, plend, plinit sequence now works, >> but I am having second thoughts >> >> I can't tell you the specific answer to your questions due to how maniacally >> busy I am these days (having just joined Lightspeed Semiconductor), but I can >> elucidate some of the plplot design ideas that have historically been >> un-documented. And, I can give it in an object-oriented context, which >> (because it is "canonical") is a lot nicer than the "this is the way it should >> work" approach I've used historically. :) >> >> This also includes proposals for change to how we do it now -- i.e. the >> behavior of plinit(). I've always been somewhat bothered by the way stream 0 >> vs stream N is handled (this bugged me back in '94 but I wasn't exactly >> swimming in free time then either). >> >> When plplot starts, you have the statically pre-allocated stream, stream 0. >> Yes the stream 0 that I hate b/c it's not allocated on the heap like a proper >> data-structure/object (in the plframe widget I automatically start from stream >> 1). >> >> So stream 0 is like a class definition and an instance rolled into one. What >> I think we need to do is get rid of the "instance" part of this and leave >> stream 0 as a "class definition" only. In this case all the command line >> arguments and initial pls...() calls (before plinit) serve to override the >> default initializion of the class variables, i.e. set stream 0 parameters only >> >> In other words, stream 0 becomes the template for all plplot streams. You can >> use it, but once you've called plinit() you have your own "instance" of the >> plplot "object" -- i.e. you have a new stream with all the relevant state info >> copied from stream 0. If you change it, it dies when your stream dies with >> plend1(). >> >> If you really want to change stream 0 (i.e. the "plplot object" "class data") >> you can always set your stream number to 0 and fire away. >> >> To summarize: >> >> plinit() creates a new stream, copied from stream 0 >> plend1() deletes that stream >> plend() deletes all streams, except of course stream 0 which is "class data" >> >> Let me know if any of this helps. >> >> -- >> Maurice LeBrun mj...@ga... >> Research Organization for Information Science and Technology of Japan (RIST) >> >> ------------------------------------------------------------------------------ >> Site24x7 APM Insight: Get Deep Visibility into Application Performance >> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month >> Monitor end-to-end web transactions and take corrective actions now >> Troubleshoot faster and improve end-user experience. Signup Now! >> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 >> _______________________________________________ >> Plplot-devel mailing list >> Plp...@li... >> https://lists.sourceforge.net/lists/listinfo/plplot-devel > > > ------------------------------------------------------------------------------ > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > _______________________________________________ > Plplot-devel mailing list > Plp...@li... > https://lists.sourceforge.net/lists/listinfo/plplot-devel |
From: Alan W. I. <ir...@be...> - 2016-02-23 19:00:34
|
Hi Phil: On 2016-02-23 11:21-0000 Phil Rosenberg wrote: > Hi Alan and Jim > I entirely advocate this. This is the same model that libcurl uses > too. In libcurl the "context" variable is a typecast void* so is > entirely opaque to the user, but it gets cast to a structure > internally. I presume the idea would be that the user would use one > context per thread? > [...] > So in summary: > > Pass in a context for helping thread safety - definitely It appears everyone so far is happy with the context approach. Furthermore, it appears now with thread-local storage that the address of the context does not need to be an argument which makes life a lot simpler for our users. > Decide how we wish to report an error, could be return val, callback > or a plgeterr( context ) call. In the ephcom case, David Howells was kind enough to implement both thread safety and an error reporting system. The implementation of the latter uses return values (simply but correctly propagated by each internal call of an ephcom routine checking for a non-zero error return and making an immediate return if one of those is detected). That implementation also uses some C tricks to make it convenient for a routine encountering an error to print out an error message in standard form and return with non-zero exit status with one statement. Furthermore, virtually all PLplot routines right now do not have a return value which allows us to use return values as part of our error reporting system. So I lean toward following the ephcom implementation of an error reporting system based on return values. Also note that if an application or a library that depends on PLplot wants to ignore the returned status code for all our C API he can do so with the above model. So it appears to me that we can implement an error reporting system and also thread safety without disrupting our users in any way which is a huge plus in my book. > If we want to stick with C: > Create a memory pool for allocating memory and ensuring we avoid memory leaks > Use longjmp for reporting errors back to the API entry point, but bear > in mind issues with jumping over C++ code I prefer the return value approach, see above. > My personal view is that the error reporting and removing exit calls > is actually more important than thread safety. I think we are in basic agreement on this, but I would phrase it slightly differently; I won't consider PLplot to be a first-class library until we have both a good error reporting system AND thread safety implemented. So I am greedy for both. :-) Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Phil R. <p.d...@gm...> - 2016-02-23 23:36:51
|
Hi Alan Sorry for top posting, I'm replying from my phone. Just to be clear the strategy I am suggesting is only internal, so that at the API boundary plplot just returns an error code. I obviously don't have any idea of the size of ephcom, but Plplot must have thousands of internal function calls that would need the return value checking. I'm not sure there is any way we can reliably catch them all and the time investment would be huge. However, discussions about this shouldn't necessarily impact the api change. We can still move to a context variable and a error return code and the internals can be changed as we need. Phil -----Original Message----- From: "Alan W. Irwin" <ir...@be...> Sent: 23/02/2016 19:00 To: "Phil Rosenberg" <p.d...@gm...> Cc: "Jim Dishaw" <ji...@di...>; "PLplot development list" <Plp...@li...> Subject: Error report system plus thread safety Hi Phil: On 2016-02-23 11:21-0000 Phil Rosenberg wrote: > Hi Alan and Jim > I entirely advocate this. This is the same model that libcurl uses > too. In libcurl the "context" variable is a typecast void* so is > entirely opaque to the user, but it gets cast to a structure > internally. I presume the idea would be that the user would use one > context per thread? > [...] > So in summary: > > Pass in a context for helping thread safety - definitely It appears everyone so far is happy with the context approach. Furthermore, it appears now with thread-local storage that the address of the context does not need to be an argument which makes life a lot simpler for our users. > Decide how we wish to report an error, could be return val, callback > or a plgeterr( context ) call. In the ephcom case, David Howells was kind enough to implement both thread safety and an error reporting system. The implementation of the latter uses return values (simply but correctly propagated by each internal call of an ephcom routine checking for a non-zero error return and making an immediate return if one of those is detected). That implementation also uses some C tricks to make it convenient for a routine encountering an error to print out an error message in standard form and return with non-zero exit status with one statement. Furthermore, virtually all PLplot routines right now do not have a return value which allows us to use return values as part of our error reporting system. So I lean toward following the ephcom implementation of an error reporting system based on return values. Also note that if an application or a library that depends on PLplot wants to ignore the returned status code for all our C API he can do so with the above model. So it appears to me that we can implement an error reporting system and also thread safety without disrupting our users in any way which is a huge plus in my book. > If we want to stick with C: > Create a memory pool for allocating memory and ensuring we avoid memory leaks > Use longjmp for reporting errors back to the API entry point, but bear > in mind issues with jumping over C++ code I prefer the return value approach, see above. > My personal view is that the error reporting and removing exit calls > is actually more important than thread safety. I think we are in basic agreement on this, but I would phrase it slightly differently; I won't consider PLplot to be a first-class library until we have both a good error reporting system AND thread safety implemented. So I am greedy for both. :-) Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2016-02-24 03:17:46
|
Hi Phil: On 2016-02-23 23:36-0000 Phil Rosenberg wrote: > I obviously don't have any idea of the size of ephcom, but Plplot must have thousands of internal function calls that would need the return value checking. I believe there are some 300 functions in our public and private API, and not all of those will be affected by this (because there are no error conditions they generate or they need to propagate). So I would estimate "hundreds" of functions rather than "thousands" would be touched by this propagation effort. Also, I think the editing task for propagating return codes would be a similar effort to the editing task for replacing the call to plexit by the generation of the equivalent message to stderr and an immediate return with non-zero return code. Note, the number of plexit calls is 220, from the results of find . -name "*.c" |xargs grep plexit | wc -l That number is roughly equivalent to the estimate I made above of the number of functions that would be affected by the propagation effort. That is the basis of my claim that the two efforts would be comparable. > I'm not sure there is any way we can reliably catch them all I am glad you asked that really important question. You stumped me for a while, but I have now thought of one test method which I believe would be almost completely reliable. Here is how you would perform that test. For each replacement of a "plexit" call you simply check (by testing with a temporary fake error message and temporary fake non-zero return code) that the fake return code propagates correctly. That check would be done by running all our C examples with the appropriate device. Normally, that device would just be the svg or psc one, but you would need to be more specific about the device whenever you are testing propagation of return codes from a particular device to libplplot. The point is that if the return code propagation is working correctly, the fake error message (generated right in the routine where the error occurs) should always be accompanied by a non-zero return code from one of the C examples. So this method tests both the propagation of the non-zero return code throughout our devices and C library and also through the C examples (which is a good thing). Of course, there is one obvious caveat about the reliability of this test method which is that our C examples might never call (directly or indirectly) the function where the fake error message and fake non-zero return code is being used for testing propagation. So extra caution would have to be used for the case where no C example prints out the temporary fake error message. But I don't think that case will happen very often because our C examples test libplplot pretty thoroughly. Anyhow, I am convinced by my above estimate that the propagation part of the work will require similar effort to the rest of the project, I do like the simplicity of the return code method that has proved to be so useful in the ephcom case, and I think the above testing method will insure the whole effort will give reliable return code results. Also, I think we are talking a few weeks of one-man effort rather than months to get this done completely. Therefore, I stand willing (early in the next release cycle) to make this happen. Of course, many hands make light work so I would welcome some help with this project. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Jonathan W. <jw...@ju...> - 2016-02-24 03:49:35
|
Hi Alan On Tue, Feb 23, 2016 at 11:00:24AM -0800, Alan W. Irwin wrote: > Hi Phil: > On 2016-02-23 11:21-0000 Phil Rosenberg wrote: > > > Hi Alan and Jim > > I entirely advocate this. This is the same model that libcurl uses > > too. In libcurl the "context" variable is a typecast void* so is > > entirely opaque to the user, but it gets cast to a structure > > internally. I presume the idea would be that the user would use one > > context per thread? > > > [...] > > So in summary: > > > > Pass in a context for helping thread safety - definitely > > It appears everyone so far is happy with the context approach. > Furthermore, it appears now with thread-local storage that the address of > the context does not need to be an argument which makes life a lot > simpler for our users. >From my perspective as a user of plplot I think having the explicit context argument in the C API has distinct advantages. I have written several programs where multiple plot streams were being maintained in parallel from a common data source. I got it working but the need to deal with a single global plot context made this more difficult than it might otherwise have been. Wrapping this context up into a context structure is a great first step. However, the full benefit won't be available if this is just stored in thread-local storage because a single thread will still be dealing with what is essentially a global context. I certainly understand the need to keep API changes to a minimum to avoid inflicting undue pain on users. At the same time, while thread-local global context storage may provide thread safety it means that people who require multiple plot contexts could not achieve this unless they split their work across multiple threads. The inclusion of an explicit context argument addresses both considerations. On a related matter, the placement of the context argument at the end of the argument list in the original proposal was intriging. Most libraries I've used which include such an argument include it as the first argument and this makes more sense to my brain at least. What is the rationale for having it at the end of the list? Does it make it easier to support earlier API versions which lack the new argument? While there's talk about APIs, I also encountered what seemed to be a slight oddity in the C++ API under plplot 5.10.0. In particular there doesn't appear to be any plstream method equivalent to plsetqtdev(). Instead one must call plsetqtdev() which relies on the global "current stream" variable plsc. The practical upshot is that one must ensure that no new plstream is created between the creation of a plstream and it's companion call to plsetqtdev(). It would be much neater and more consistent if one could just do plstream *pls = new plstream; pls->sdev("extqt"); pls->plsetqtdev(...); Regards jonathan |
From: Alan W. I. <ir...@be...> - 2016-02-24 09:11:13
|
On 2016-02-24 14:18+1030 Jonathan Woithe wrote: > Hi Alan > > On Tue, Feb 23, 2016 at 11:00:24AM -0800, Alan W. Irwin wrote: [..] >> It appears everyone so far is happy with the context approach. >> Furthermore, it appears now with thread-local storage that the address of >> the context does not need to be an argument which makes life a lot >> simpler for our users. > > From my perspective as a user of plplot I think having the explicit context > argument in the C API has distinct advantages. [...] > I certainly understand the need to keep API changes to a minimum to avoid > inflicting undue pain on users. At the same time, while thread-local global > context storage may provide thread safety it means that people who require > multiple plot contexts could not achieve this unless they split their work > across multiple threads. True. However, isn't being forced to use separate threads in order to (automatically) have separate PLplot contexts for each thread a relatively minor inconvenience compared to the very much larger user inconvenience required by adding a context address to most API calls? > On a related matter, the placement of the context argument at the end of the > argument list in the original proposal was intriging. Most libraries I've > used which include such an argument include it as the first argument and > this makes more sense to my brain at least. What is the rationale for > having it at the end of the list? Does it make it easier to support earlier > API versions which lack the new argument? If we did end up having to include a context address as an argument to every PLplot call, then I agree my initial choice of making that argument last is likely wrong, and it should be first instead. However, I also think this question is now likely moot (see my response to your question above). You also asked a question about our C++ API, but I will leave that answer to our C++ experts. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Phil R. <p.d...@gm...> - 2016-02-24 10:33:26
|
Hi Alan I've cut and pasted some bits together of your last couple of emails > However, isn't being forced to use separate threads in order to > (automatically) have separate PLplot contexts for each thread a > relatively minor inconvenience compared to the very much larger user > inconvenience required by adding a context address to most API calls? > Not necessarily. This may force someone to restructure their code as they would have to have separate contexts in separate functions and may actually generate race conditions that people will need to deal with using mutexes. >I believe there are some 300 functions in our public and private API, >and not all of those will be affected by this (because there are no >error conditions they generate or they need to propagate). So I would >estimate "hundreds" of functions rather than "thousands" would be >touched by this propagation effort. Note that some functions are called tens of times within plplot, and with an error code return mechanism it is each call that needs a check. Also some functions return values; this would need restructuring with the potential for introducing bugs - possibly very hard to find bugs. In each function there would need to be an examination of all the memory allocations and code would need to be introduced at each check point to do freeing to avoid memory leaks. >Also, I think the editing task for propagating return codes would be a >similar effort to the editing task for replacing the call to plexit by >the generation of the equivalent message to stderr and an immediate >return with non-zero return code. Note, the number of plexit calls is >220, By contrast with an exception model we have less than 1 change to make per plexit call. These can almost be done by find and replace. You may wonder why this is less than. I imagine almost all of those are memory allocation failures. If we create a plmalloc function which allocates and tracks memory then this function can generate the exception if the memory allocation fails. This removes all requirements to even check whether a memory allocation has succeeded. Almost all the plexit calls vanish by writing just one function, but even better all the cases where we were lazy and didn't bother to check if our allocation succeeded (I wonder how many of those exist in plplot right now?) get automatically fixed. I am a fairly recent convert to exceptions, but I now use them throughout my C++ code, because using exceptions totally frees me from having to worry about errors. I don't have to worry about checking return values or anything like that, I am free to concentrate on the interesting aspects of the code writing, rather than getting bogged down in error checking and avoiding memory leaks. But more importantly, if used in well styled code, exceptions are bomb proof, they actually make it impossible to miss or fail to deal with an error. However I used to think exceptions were just a feature that I didn't really understand I probably didn't need to. The thing that changed my mind was some reading about coding style. Can I strongly recommend that you have a look at chapter 10 of C++ In Action: Industrial-Strength Programming Techniques. It has been published online for free and you can find this chapter at http://www.relisoft.com/book/tech/5resource.html. The important bit is from the beginning to the subsection Ownership of Resources. I know you are not big into C++, but all you have to know to understand this part of the chapter is that in C++ new is basically like malloc, delete is basically like free, when you create an object (either statically or using new) a function called its constructor is automatically called and when it goes out of scope (or if you delete it after creating it with new) then a function called its destructor is called. It may help to know that earlier in the book they create a calculator project as an example which gets briefly referred to. Reading this chapter was a revelation to me! >For each replacement of a "plexit" call you simply check >(by testing with a temporary fake error message and temporary fake >non-zero return code) that the fake return code propagates correctly. I'm not sure how you practically do this? Nor am I sure this is maintainable. Don't forget that a program may reach many potential plexit calls with a single API call and it may reach the same plexit call multiple times, but in different states. We also need to check for memory leaks. The testing required sounds huge to me. Anyway, sorry if this sounds negative, it's not meant to be. As I said I've found exceptions quite recently and they have revolutionised my coding style, so I feel quite passionate about them. Phil |
From: Alan W. I. <ir...@be...> - 2016-02-24 17:18:46
|
On 2016-02-24 10:33-0000 Phil Rosenberg wrote: > Hi Alan > I've cut and pasted some bits together of your last couple of emails > > > >> However, isn't being forced to use separate threads in order to >> (automatically) have separate PLplot contexts for each thread a >> relatively minor inconvenience compared to the very much larger user >> inconvenience required by adding a context address to most API calls? >> > Not necessarily. This may force someone to restructure their code as > they would have to have separate contexts in separate functions and > may actually generate race conditions that people will need to deal > with using mutexes. Without introducing backwards incompatibilities in our entire API we could certainly add a getter, plgcontext and setter, plscontext, for PLplotContextAddress to our API similarly to the way that plgstrm and plsstrm work now. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2016-02-24 19:40:36
|
On 2016-02-24 10:33-0000 Phil Rosenberg wrote: [Alan] >> I believe there are some 300 functions in our public and private API, >> and not all of those will be affected by this (because there are no >> error conditions they generate or they need to propagate). So I would >> estimate "hundreds" of functions rather than "thousands" would be >> touched by this propagation effort. [Phil] > Note that some functions are called tens of times within plplot, and > with an error code return mechanism it is each call that needs a > check. That point is correct. So it is possible our C examples might never exercise a particular call chain that needs checking, or that call chain might be there to be checked, but it could be masked in tests because it is always called after a different call chain to the same function in everyone of our C examples. Nevertheless, our first line of defence is due diligence with grep which should easily find all instances where a particular routine is being called within our code base. So combining that with the test I described would likely catch the vast majority of errors in the error reporting. Therefore, I think this approach is already far better than "good enough". In any case this is a large change for users since before they could (largely) rely on PLplot to exit when there were errors and now suddenly the action to take concerning errors is on their shoulders instead. So we should write up this change extensively in the release notes including a section about how to report errors in the report system itself where there is an initial error message followed either by no non-zero return code or else some other PLplot crash before control is returned to the calling programme. But again, I think because of the due diligence noted above this would be quite a rare case for our users to contend with. > Also some functions return values; this would need > restructuring with the potential for introducing bugs - possibly very > hard to find bugs. I agree this is an issue that will need to be addressed _if_ there exists a PLplot function that currently returns a value and that function also has potential error conditions to report or propagate. However, this is likely a small problem (or may turn out to not be a problem at all) because the number of PLplot functions with return values now is quite small. >> Also, I think the editing task for propagating return codes would be a >> similar effort to the editing task for replacing the call to plexit by >> the generation of the equivalent message to stderr and an immediate >> return with non-zero return code. Note, the number of plexit calls is >> 220, > By contrast with an exception model we have less than 1 change to make > per plexit call. These can almost be done by find and replace. You may > wonder why this is less than. I imagine almost all of those are memory > allocation failures. If we create a plmalloc function which allocates > and tracks memory then this function can generate the exception if the > memory allocation fails. This removes all requirements to even check > whether a memory allocation has succeeded. Almost all the plexit calls > vanish by writing just one function, but even better all the cases > where we were lazy and didn't bother to check if our allocation > succeeded (I wonder how many of those exist in plplot right now?) get > automatically fixed. I agree that the vast majority of our current plexit calls are due to memory allocation errors and furthermore I agree with you we have been lazy about checking for such memory allocation errors so that fraction would become substantially larger if we started to check for such errors in all memory allocation cases. However, it should be noted that memory allocation errors (for debugged code that calls malloc et al. correctly) are essentially always a sign of an emergency condition where you have run out of memory due to some process leaking memory like crazy. And remember this is an extremely rare emergency condition in any case. For example, in my decades of experience with PLplot I have _never_ seen a plexit message concerning memory allocation failing. So my conclusion is an immediate exit is unlikely to irritate users in this one particular memory allocation error case. So let's simplify the problem by wrapping all our malloc calls with plmalloc (and calloc calls by plcalloc) as you suggest above. The difference with what you suggest above is I think those wrappers should rigourously check validity of arguments (i.e., sizes must be positive), automatically check for allocation errors, and if such an error is encountered print out a standard "likely out of memory" error message before doing an immediate exit. >From what you have said above, I think you will agree with me that single change would make the task of implementing a proper error report system for the remaining errors at least an order of magnitude easier. > I am a fairly recent convert to exceptions, but I now use them > throughout my C++ code, because using exceptions totally frees me from > having to worry about errors. I don't have to worry about checking > return values or anything like that, I am free to concentrate on the > interesting aspects of the code writing, rather than getting bogged > down in error checking and avoiding memory leaks. But more > importantly, if used in well styled code, exceptions are bomb proof, > they actually make it impossible to miss or fail to deal with an > error. Is this possibility only available in C++? If so, it does not help us with our core C library which does need an error reporting system now. We do not want to use the the possibility that we _might_ move to C++ in the future for our core library as an excuse for inaction on this important topic for our current core C library. [...] > Anyway, sorry if this sounds negative, it's not meant to be. As I said > I've found exceptions quite recently and they have revolutionised my > coding style, so I feel quite passionate about them. OK. By the way, I am not negative about C++, but I just don't have time right now to give it a high priority. Note that could change since even at this late date I am still learning and growing. :-) For example, I have many decades of Fortran 77 experience but only in the last couple of months have I finally learned modern Fortran as a result of my recent collaboration with Arjen on rewriting our Fortran binding. As a result I am now a strong advocate of modern Fortran (it has some high-level capabilities that are wonderfully useful in scientific computing) while before I was mainly indifferent to it. So as a result of this experience, I intend to convert both te_gen (a Fortran subproject of timeephem) and FreeEOS to modern Fortran. Live and learn.... Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Phil R. <p.d...@gm...> - 2016-02-26 17:34:54
|
We have discussed a few API breaking changes recently. Error reporting, thread safety, C++ API changes and the Fortran API changes are pretty close to complete by the seems of things. Is it time to bring up the possibility of PLPlot 6 again and how we intend to manage that? Alan there are some further points below > >> I am a fairly recent convert to exceptions, but I now use them >> throughout my C++ code, because using exceptions totally frees me from >> having to worry about errors. I don't have to worry about checking >> return values or anything like that, I am free to concentrate on the >> interesting aspects of the code writing, rather than getting bogged >> down in error checking and avoiding memory leaks. But more >> importantly, if used in well styled code, exceptions are bomb proof, >> they actually make it impossible to miss or fail to deal with an >> error. > > > Is this possibility only available in C++? If so, it does not help us > with our core C library which does need an error reporting system now. > We do not want to use the the possibility that we _might_ move to C++ > in the future for our core library as an excuse for inaction on this > important topic for our current core C library. > Regarding this point - C can do similar things using the setjmp()/longjmp() function and a resource pool/tracker. It is not as simple to implement as in C++, but I still think it would be much easier to implement and much easier to support than trying to propagate error codes through our internals. I still strongly recommend a read of that book chapter and if you ever feel like experimenting with C++ that book is excellent, although a bit out of date now we have C++11 and beyond. It just dawned on me that we could simply put longjmp(errcode); inside plexit() and then all we need to do is add the code to catch this in the API functions and the job is complete. Anyway please just have a read about exceptions before we make this call. >However, it should be noted that memory allocation errors (for >debugged code that calls malloc et al. correctly) are essentially >always a sign of an emergency condition where you have run out of >memory due to some process leaking memory like crazy. And remember >this is an extremely rare emergency condition in any case. For >example, in my decades of experience with PLplot I have _never_ seen a >plexit message concerning memory allocation failing. So my conclusion >is an immediate exit is unlikely to irritate users in this >one particular memory allocation error case. I would suggest that we should report back on memory allocation failures rather than exit. Even with a memory allocation failure, someone using plplot in an application (e.g. in GDL) may be able to offer their user a final chance to save their work, or may be able to free some memory. I'm not sure I have ever hit a memory allocation error within plplot, but I certainly regularly hit memory allocation errors during data analysis in general. Phil |
From: Alan W. I. <ir...@be...> - 2016-02-26 22:18:34
|
@Hazen: Please pay close attention to this ongoing discussion and participate in it further because I don't want you to start work on the return value approach for reporting errors like you have already volunteered to do just in case the (likely) decision is made that we go with error reporting based on C exception handling instead. More below. On 2016-02-26 17:34-0000 Phil Rosenberg wrote: [Phil] >>> I am a fairly recent convert to exceptions, but I now use them >>> throughout my C++ code, because using exceptions totally frees me from >>> having to worry about errors. I don't have to worry about checking >>> return values or anything like that, I am free to concentrate on the >>> interesting aspects of the code writing, rather than getting bogged >>> down in error checking and avoiding memory leaks. But more >>> importantly, if used in well styled code, exceptions are bomb proof, >>> they actually make it impossible to miss or fail to deal with an >>> error. >> >> [Alan] >> Is this possibility only available in C++? If so, it does not help us >> with our core C library which does need an error reporting system now. >> We do not want to use the the possibility that we _might_ move to C++ >> in the future for our core library as an excuse for inaction on this >> important topic for our current core C library. >> [Phil] > Regarding this point - C can do similar things using the > setjmp()/longjmp() function and a resource pool/tracker. It is not as > simple to implement as in C++, but I still think it would be much > easier to implement and much easier to support than trying to > propagate error codes through our internals. I still strongly > recommend a read of that book chapter and if you ever feel like > experimenting with C++ that book is excellent, although a bit out of > date now we have C++11 and beyond. > It just dawned on me that we could simply put longjmp(errcode); inside > plexit() and then all we need to do is add the code to catch this in > the API functions and the job is complete. > > Anyway please just have a read about exceptions before we make this call. @Phil: I promise to do that later today. But I have to admit you have already made a fairly convincing argument for error reporting via C exception handling, and if you look at the caller graphs I prepared there are some instances (even if we have immediate exit for memory emergencies) where the return value approach would be quite complicated. Furthermore, from my skimming of <https://en.wikipedia.org/wiki/Setjmp.h> it appears that your preferred setjmp()/longjmp() approach is usable for C exception handling just like you have claimed above. So the current status is both Hazen and I are keen to see a proper error report system for PLplot implemented soon. To the point where we were both willing to implement that with the return code method early in the next release cycle. But you have made some good arguments that primitive but easy to understand method should not be used, and instead C exception handling using setjmp()/longjmp() should be used instead as the basis for the error report system. I cannot speak for Hazen, but the major difficulty I have with the setjmp()/longjmp() method is I don't understand exactly how it should be implemented. So would you be willing to implement what you had in mind now on a private topic branch with the goal of merging it into master early in the next release cycle? (Or even in this release cycle if you can get something matured within the next few weeks.) Of course, if some part of that task was just routine editing, I would be willing to help with that, but it sounds like you might have something in mind that hardly requires much editing at all. Note, also that once you had a brand-new exception-based error report system matured and merged to master, I think you would pick up a lot of volunteers (I would be one of them) to go through our code with a fine-toothed comb looking for other errors we should be reporting (after every memory allocation, file manipulation, etc.) with that error report system. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Hazen B. <hba...@ma...> - 2016-02-27 03:45:42
|
On 02/26/2016 05:18 PM, Alan W. Irwin wrote: > @Hazen: > > Please pay close attention to this ongoing discussion and participate > in it further because I don't want you to start work on the return > value approach for reporting errors like you have already volunteered > to do just in case the (likely) decision is made that we go with error > reporting based on C exception handling instead. Don't worry, I promise not to start any time soon.. > On 2016-02-26 17:34-0000 Phil Rosenberg wrote: > > [Phil] >>>> I am a fairly recent convert to exceptions, but I now use them >>>> throughout my C++ code, because using exceptions totally frees me from >>>> having to worry about errors. I don't have to worry about checking >>>> return values or anything like that, I am free to concentrate on the >>>> interesting aspects of the code writing, rather than getting bogged >>>> down in error checking and avoiding memory leaks. But more >>>> importantly, if used in well styled code, exceptions are bomb proof, >>>> they actually make it impossible to miss or fail to deal with an >>>> error. >>> >>> > [Alan] >>> Is this possibility only available in C++? If so, it does not help us >>> with our core C library which does need an error reporting system now. >>> We do not want to use the the possibility that we _might_ move to C++ >>> in the future for our core library as an excuse for inaction on this >>> important topic for our current core C library. >>> > [Phil] >> Regarding this point - C can do similar things using the >> setjmp()/longjmp() function and a resource pool/tracker. It is not as >> simple to implement as in C++, but I still think it would be much >> easier to implement and much easier to support than trying to >> propagate error codes through our internals. I still strongly >> recommend a read of that book chapter and if you ever feel like >> experimenting with C++ that book is excellent, although a bit out of >> date now we have C++11 and beyond. >> It just dawned on me that we could simply put longjmp(errcode); inside >> plexit() and then all we need to do is add the code to catch this in >> the API functions and the job is complete. >> >> Anyway please just have a read about exceptions before we make this call. > > @Phil: > > I promise to do that later today. But I have to admit you have > already made a fairly convincing argument for error reporting via C > exception handling, and if you look at the caller graphs I prepared > there are some instances (even if we have immediate exit for memory > emergencies) where the return value approach would be quite > complicated. Such as? Furthermore, from my skimming of > <https://en.wikipedia.org/wiki/Setjmp.h> it appears that your > preferred setjmp()/longjmp() approach is usable for C exception > handling just like you have claimed above. I still favor the return value based approach. If one was to start again from scratch this is the way (I think) the library would be written, so I don't see why it would be done differently now. Yes it will probably be more work to implement, but I don't think it will be particularly hard to maintain as a compiler like gcc is more than happy to warn you if you are ignoring returned values. And yes the exit will not be clean, as there is likely to be memory that was allocated but not freed. However the possibility exists with this approach of going back in the future and cleaning all of that up, something which I don't think is possible with a setjmp() approach. Though I guess we could start with the setjmp() approach as a possibly quicker way to make the API change and then go back and clean up the internals at our leisure to use a return based approach. -Hazen |
From: Alan W. I. <ir...@be...> - 2016-02-27 06:26:13
|
On 2016-02-26 22:45-0500 Hazen Babcock wrote: > On 02/26/2016 05:18 PM, Alan W. Irwin wrote: [...] >> But I have to admit you [Phil] have >> already made a fairly convincing argument for error reporting via C >> exception handling, and if you look at the caller graphs I prepared >> there are some instances (even if we have immediate exit for memory >> emergencies) where the return value approach would be quite >> complicated. > > Such as? Hi Hazen: To give you an immediate fairly complex example look at the call graph and caller graph of grline in <http://plplot.sourceforge.net/doxygen/html/plcore_8c.html>. The caller graph includes some 50 functions while its call graph includes calls to plexit and one of those calls in plsave_set_locale is not anything to do with memory allocation so there is no question we would have to deal with it in our error report system. So that is already a pretty complicated example, but I just discovered that is just scratching the surface because the large call/caller graphs generated by doxygen are partitioned (as indicated by the red box around some references) so a given caller graph being presented on a single plot is in complex cases a subset of the complete result. For example, the caller graph for plsave_set_locale (in <http://plplot.sourceforge.net/doxygen/html/plctrl_8c.html> this time) does not include all the complications of the grline caller graph. Instead, it summarizes that 50-function call graph as a red box around grline with no further details. So I think to get the full picture for either a large call graph or caller graph you have to click on all the red boxes. Anyhow, please browse and click on these graphs including the red boxes to draw your own conclusions about the complexity of the call graphs that the return code approach for error reporting would have to deal with. I will stop there because I don't have the technical expertise to answer the concerns you expressed about using C exception handling (implemented with setjmp/longjmp) as the complete basis of the error reporting system for PLplot that Phil apparently has in mind. However, I assume Phil will answer your concerns. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |