This new patch is still for discussion only. There's some
attempt at portability.
The plugin directory provides an example and a small library
for platforms where the plugin can access symbols from the
gnuplot executable.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In preparation for a code freeze and the run-up to a release of
version 4.2, existing bugs and patchsets are being prioritized.
This patchset is not on my (sfeam) list for inclusion in 4.2 and
is therefore being marked as priority 2.
Note that this does not mean it is a bad patch, or that it won't
be incorporated into cvs after 4.2 is released. We can
re-evaluate priorities after 4.2 is out.
If you want to argue for immediate reconsideration - go right
ahead; but do so quickly!
Ethan Merritt
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This patchset has been closed because it is suspected to be obsolete. Possible reasons include
(1) the patch no longer applies against current cvs
(2) strong resistance was expressed during discussion
(3) an equivalent capability has been added since the patch was submitted
(4) we no longer remember or don't understand what the patch was intended to fix
If this is your patchset and you think that it is still relevant, please re-submit a revised version against the current CVS source.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Attached is an updated version of this patch. Since I can only test it on Windows (MVC2008), I removed the untestable changes to the configure scripts.
I think this new feature is very useful for computationally intensive functions used in fits, where the use of the "system" command as alternative is too slow.
Bumping the priority on this one so it doesn't get lost at the bottom of the pile.
I have not looked at it for a long time, but it looks like candidate for version 5.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Since this patch was originally created we have introduced string-valued functions as a standard feature. So the original syntax
f(x) = "foo:baz"
now defines a function that returns the string "foo:baz", which conflicts with what I think is the intended meaning "replace f(x) with foo::baz(x)".
I'm not at all sure what the best alternative syntax would be.
Also I don't understand what bit of magic is supposed to match the number of parameters in a gnuplot function call with the actual number of parameters implemented in the plugin function. Won't things die horribly if that goes wrong?
If someone is interested enough to whip up a linux equivalent, I'd be happy to test :-)
Last edit: Ethan Merritt 2014-02-18
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I updated the patch again and changed the calling convention to
f(x)=external("example:sinc")
to call the function "struct value sinc(int nargs, struct value arg, void priv)" in
the library "example.dll"/"example.so". This is surely not the best possible syntax,
but it does not collide with the string-valued functions anymore. Since one has to
open the shared library at the time that the function is defined, it was rather tricky to
implement.
The code (if I understand this part correct) counts the number of parameters defined for the userdefined function f (here only 1) and gives this number as parameter nargs to the external function. This function then has to take care, that only "nargs" of the arguments are accessed.
For the linux parts, I think that you only have to define the HAVE_EXTERNAL_FUNCTIONS constant for the compilation. The code in internal.h should define the correct functions for opening the ".so" files.
I just installed a virtual Ubuntu machine and added the necessary changes in the 'configure.in' and 'src/Makefile.am' to be able to build successfully with the attached patch (no changes in the rest of the source files). Using the
makefile in 'demo/plugin' one can build the example library used in
'plugin.dem'.
Thanks. I have been trying to give this a workout under linux. Here is a cleaned up version that
1) removes a redundant field that it added to udft_entry (I never did figure out why)
2) removed an unintentional reversion to old code in internal.c
3) cleaned up the configure file
4) added some missing #includes and initializations
5) added a 2-parameter function to the demo
It does work, but it's really fragile. Any deviation from correct syntax causes a segfault.
I don't think implementing external("foo:baz") as a builtin function is a good idea. It's too fragile. For example the following commands each cause an immediate segfault and it's not obvious how to fix it cleanly.
gnuplot> foo = external("baz")
gnuplot> foo = external("example:baz")
gnuplot> f(x) = 1. + external("example:sinc"); print f(1)
Another problem is that there is no way to return an error from the plugin function. I hacked in a NaN return but then realized that it's nontrivial to load NaN from the plugin. The issue is that parameter checking needs to be done inside the plugin, both for the number of parameters and their type. Note that since the patch was first written we've added non-numeric value types, so the assumption that everything is necessarily INTGR or CMPLX no longer holds.
I don't have a specific proposal for a more workable syntax.
Maybe elements of the following ideas could be explored?
gnuplot> import funcname(x,y,z) from "sharedobj" # even though I hate python
gnuplot> plugin "sharedobj" "funcname(x,y,z)" # no choice to rename the function
gnuplot> plugin f(x,y,z) "sharedobj" "entry" # maps f() to entry() with 3 params
In each case the number of parameters would ideally be checked at the time the function is loaded, but it's not obvious how to do this.
1) removes a redundant field that it added to udft_entry (I never did figure out why)
2) removed an unintentional reversion to old code in internal.c
I thought both the entry and the code in internal.c is used to estimate the
number of parameters of the function.
In my opinion your 3rd proposed syntax is most promising (reminds one to the fit
command). Maybe it makes sense to restrict the number of parameters, so
that it is not needed to pass an array to the external function.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
But the existing field (dummy_num) already contains the number of parameters. The new code in internal.c (not the reverted code) has the effect of counting from 0 to dummy_num-1 and storing the result in num_dummies. The only case in which they could differ is if more than MAX_NUM_VAR parameters were given, which is a separate error. So the new field is entirely redundant.
I don't like restricting the number of parameters. Why would you want to do that? The flexibility seems particularly useful in connection with the fit command, which allows you to pass up to MAX_NUM_VAR parameters.
As to passing an array to the external function, I can think of a very different alternative. Right now the external function (at least in the example) uses almost nothing of the gnuplot infrastructure. It could instead be modeled on the existing internal f_*() functions, using push() and pop() to obtain parameters and return the result. Using that model, instead of passing a parameter array you'd pass a callback to pop(). But I haven't tried this; it might turn out that you'd have to pass so much context to make it work that it would be horrible.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So
import foo(x) from "mylib"
import baz(x) from "mylib:foo"
both look for an exported symbol foo, but the second variant shows it under the name baz inside the gnuplot session.
I have added documentation and expanded the demo to include a 2-parameter function and a set of macros to enforce argument type and number.
The one remaining thing I think is needed is to have it search for the requested shared object using the search path in "set loadpath".
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Logged In: YES
user_id=27517
I'm uploading the patch (received from poster via mail...)
Updated patch
Logged In: YES
user_id=252223
This new patch is still for discussion only. There's some
attempt at portability.
The plugin directory provides an example and a small library
for platforms where the plugin can access symbols from the
gnuplot executable.
Logged In: YES
user_id=235620
In preparation for a code freeze and the run-up to a release of
version 4.2, existing bugs and patchsets are being prioritized.
This patchset is not on my (sfeam) list for inclusion in 4.2 and
is therefore being marked as priority 2.
Note that this does not mean it is a bad patch, or that it won't
be incorporated into cvs after 4.2 is released. We can
re-evaluate priorities after 4.2 is out.
If you want to argue for immediate reconsideration - go right
ahead; but do so quickly!
Ethan Merritt
This patchset has been closed because it is suspected to be obsolete. Possible reasons include
(1) the patch no longer applies against current cvs
(2) strong resistance was expressed during discussion
(3) an equivalent capability has been added since the patch was submitted
(4) we no longer remember or don't understand what the patch was intended to fix
If this is your patchset and you think that it is still relevant, please re-submit a revised version against the current CVS source.
Attached is an updated version of this patch. Since I can only test it on Windows (MVC2008), I removed the untestable changes to the configure scripts.
I think this new feature is very useful for computationally intensive functions used in fits, where the use of the "system" command as alternative is too slow.
Bumping the priority on this one so it doesn't get lost at the bottom of the pile.
I have not looked at it for a long time, but it looks like candidate for version 5.
Since this patch was originally created we have introduced string-valued functions as a standard feature. So the original syntax
f(x) = "foo:baz"
now defines a function that returns the string "foo:baz", which conflicts with what I think is the intended meaning "replace f(x) with foo::baz(x)".
I'm not at all sure what the best alternative syntax would be.
Also I don't understand what bit of magic is supposed to match the number of parameters in a gnuplot function call with the actual number of parameters implemented in the plugin function. Won't things die horribly if that goes wrong?
If someone is interested enough to whip up a linux equivalent, I'd be happy to test :-)
Last edit: Ethan Merritt 2014-02-18
I updated the patch again and changed the calling convention to
f(x)=external("example:sinc")
to call the function "struct value sinc(int nargs, struct value arg, void priv)" in
the library "example.dll"/"example.so". This is surely not the best possible syntax,
but it does not collide with the string-valued functions anymore. Since one has to
open the shared library at the time that the function is defined, it was rather tricky to
implement.
The code (if I understand this part correct) counts the number of parameters defined for the userdefined function f (here only 1) and gives this number as parameter nargs to the external function. This function then has to take care, that only "nargs" of the arguments are accessed.
For the linux parts, I think that you only have to define the HAVE_EXTERNAL_FUNCTIONS constant for the compilation. The code in internal.h should define the correct functions for opening the ".so" files.
I just installed a virtual Ubuntu machine and added the necessary changes in the 'configure.in' and 'src/Makefile.am' to be able to build successfully with the attached patch (no changes in the rest of the source files). Using the
makefile in 'demo/plugin' one can build the example library used in
'plugin.dem'.
Thanks. I have been trying to give this a workout under linux. Here is a cleaned up version that
1) removes a redundant field that it added to udft_entry (I never did figure out why)
2) removed an unintentional reversion to old code in internal.c
3) cleaned up the configure file
4) added some missing #includes and initializations
5) added a 2-parameter function to the demo
It does work, but it's really fragile. Any deviation from correct syntax causes a segfault.
I don't think implementing external("foo:baz") as a builtin function is a good idea. It's too fragile. For example the following commands each cause an immediate segfault and it's not obvious how to fix it cleanly.
gnuplot> foo = external("baz")
gnuplot> foo = external("example:baz")
gnuplot> f(x) = 1. + external("example:sinc"); print f(1)
Another problem is that there is no way to return an error from the plugin function. I hacked in a NaN return but then realized that it's nontrivial to load NaN from the plugin. The issue is that parameter checking needs to be done inside the plugin, both for the number of parameters and their type. Note that since the patch was first written we've added non-numeric value types, so the assumption that everything is necessarily INTGR or CMPLX no longer holds.
I don't have a specific proposal for a more workable syntax.
Maybe elements of the following ideas could be explored?
gnuplot> import funcname(x,y,z) from "sharedobj" # even though I hate python
gnuplot> plugin "sharedobj" "funcname(x,y,z)" # no choice to rename the function
gnuplot> plugin f(x,y,z) "sharedobj" "entry" # maps f() to entry() with 3 params
In each case the number of parameters would ideally be checked at the time the function is loaded, but it's not obvious how to do this.
I thought both the entry and the code in internal.c is used to estimate the
number of parameters of the function.
In my opinion your 3rd proposed syntax is most promising (reminds one to the fit
command). Maybe it makes sense to restrict the number of parameters, so
that it is not needed to pass an array to the external function.
But the existing field (dummy_num) already contains the number of parameters. The new code in internal.c (not the reverted code) has the effect of counting from 0 to dummy_num-1 and storing the result in num_dummies. The only case in which they could differ is if more than MAX_NUM_VAR parameters were given, which is a separate error. So the new field is entirely redundant.
I don't like restricting the number of parameters. Why would you want to do that? The flexibility seems particularly useful in connection with the fit command, which allows you to pass up to MAX_NUM_VAR parameters.
As to passing an array to the external function, I can think of a very different alternative. Right now the external function (at least in the example) uses almost nothing of the gnuplot infrastructure. It could instead be modeled on the existing internal f_*() functions, using push() and pop() to obtain parameters and return the result. Using that model, instead of passing a parameter array you'd pass a callback to pop(). But I haven't tried this; it might turn out that you'd have to pass so much context to make it work that it would be horrible.
I have tentatively decided to go with the syntax
import func(x,y,...) from "sharedobj[:funcname]"
So
import foo(x) from "mylib"
import baz(x) from "mylib:foo"
both look for an exported symbol foo, but the second variant shows it under the name baz inside the gnuplot session.
I have added documentation and expanded the demo to include a 2-parameter function and a set of macros to enforce argument type and number.
The one remaining thing I think is needed is to have it search for the requested shared object using the search path in "set loadpath".
Added to CVS