From: Tomas Z. <zel...@gm...> - 2005-09-29 06:14:42
|
Hello, while trying to make more from included readline, I encountered a problem that can be shown on this example: (use-package 'ffi) (def-call-out add-defun (:name "rl_add_defun") (:library :default) (:arguments (name c-string :in :malloc-free) =09 (callback (c-function (:language :stdc) =09=09=09 (:arguments (rep int) (char int))) =09 :in :malloc-free) =09 (key int)) (:language :stdc) (:return-type int)) (defun init () (labels ((edit-history (key rep) (print key) (print rep))) (add-defun "edit-history" #'edit-history -1))) (init) (push #'init *init-hooks*) (saveinitmem "foo") That is, a function with callback to linux is defined and then called. Now, this works fine when loaded (compiled or not), but when later loaded from image, it makes problem when compiled: $ clisp -norc -C problem.lisp $ clisp -q -M foo.mem *** - FFI::FOREIGN-CALL-OUT: #<INVALID FOREIGN-POINTER #x00000000> comes fr= om a previous Lisp session and is invalid The following restarts are available: SKIP :R1 skip (SAVEINITMEM foo) STOP :R2 stop loading file /home/tomas/src/misc/problem.lisp ABORT :R3 ABORT $ clisp -norc problem.lisp $ clisp -q -M foo.mem [1]> When read-history is defined with defun, it makes same problem even when not compiled. Am I doing something that should not be done? Can I achieve the effect otherwise?Even proper RTFM is welcome - I failed to find anything on topic. Tested with clisp 2.35, on various Linuxes. Best regards, Tomas Zellerin |
From: Sam S. <sd...@gn...> - 2005-09-29 13:19:20
|
Hi, > * Tomas Zellerin <mryyreva@tznvy.pbz> [2005-09-29 08:14:30 +0200]: > > while trying to make more from included readline, I encountered a > problem that can be shown on this example: > > (use-package 'ffi) > (def-call-out add-defun > (:name "rl_add_defun") > (:library :default) try (:library "libreadline.so") > *** - FFI::FOREIGN-CALL-OUT: #<INVALID FOREIGN-POINTER #x00000000> comes from > a previous Lisp session and is invalid foreign pointers are updated when it is known what library they come from. -- Sam Steingold (http://www.podval.org/~sds) running w2k <http://www.savegushkatif.org> <http://www.dhimmi.com/> <http://ffii.org/> <http://www.camera.org> <http://www.memri.org/> <http://pmw.org.il/> Marriage is the sole cause of divorce. |
From: Pascal B. <pj...@in...> - 2005-09-29 16:21:30
|
Tomas Zellerin writes: > Hello, > > while trying to make more from included readline, I encountered a > problem that can be shown on this example: > > (use-package 'ffi) > (def-call-out add-defun > (:name "rl_add_defun") > (:library :default) > (:arguments (name c-string :in :malloc-free) > (callback (c-function (:language :stdc) > (:arguments (rep int) (char int))) > :in :malloc-free) > (key int)) > (:language :stdc) > (:return-type int)) > (defun init () > (labels ((edit-history (key rep) > (print key) > (print rep))) > (add-defun "edit-history" #'edit-history -1))) > (init) > (push #'init *init-hooks*) > (saveinitmem "foo") > > That is, a function with callback to linux is defined and then called. > Now, this works fine when loaded (compiled or not), but when later > loaded from image, it makes problem when compiled: > > $ clisp -norc -C problem.lisp > $ clisp -q -M foo.mem > > *** - FFI::FOREIGN-CALL-OUT: #<INVALID FOREIGN-POINTER #x00000000> comes from > a previous Lisp session and is invalid > The following restarts are available: > SKIP :R1 skip (SAVEINITMEM foo) > STOP :R2 stop loading file /home/tomas/src/misc/problem.lisp > ABORT :R3 ABORT > $ clisp -norc problem.lisp > $ clisp -q -M foo.mem > [1]> > > When read-history is defined with defun, it makes same problem even > when not compiled. > > Am I doing something that should not be done? Can I achieve the effect > otherwise?Even proper RTFM is welcome - I failed to find anything on > topic. Don't you have any idea of why this happens? In the first process, you dynamically load and link a library. When you save the memory image, all files, including that library, are closed. When you start the second process, this dynamic library is still closed, so of course, you cannot call function that are stored in it! The solution should be obvious: use the :init-function argument to ext:saveinitmem and rerun the dynamic linkings in the new processes. -- "Specifications are for the weak and timid!" |
From: Sam S. <sd...@gn...> - 2005-09-29 22:22:52
|
> * Tomas Zellerin <mryyreva@tznvy.pbz> [2005-09-29 08:14:30 +0200]: > > while trying to make more from included readline, I encountered a > problem that can be shown on this example: > > (use-package 'ffi) > (def-call-out add-defun > (:name "rl_add_defun") > (:library :default) > (:arguments (name c-string :in :malloc-free) > (callback (c-function (:language :stdc) > (:arguments (rep int) (char int))) > :in :malloc-free) > (key int)) > (:language :stdc) > (:return-type int)) > (defun init () > (labels ((edit-history (key rep) > (print key) > (print rep))) > (add-defun "edit-history" #'edit-history -1))) > (init) > (push #'init *init-hooks*) > (saveinitmem "foo") > > That is, a function with callback to linux is defined and then called. > Now, this works fine when loaded (compiled or not), but when later > loaded from image, it makes problem when compiled: > > $ clisp -norc -C problem.lisp > $ clisp -q -M foo.mem > > *** - FFI::FOREIGN-CALL-OUT: #<INVALID FOREIGN-POINTER #x00000000> comes from > a previous Lisp session and is invalid > The following restarts are available: > SKIP :R1 skip (SAVEINITMEM foo) > STOP :R2 stop loading file /home/tomas/src/misc/problem.lisp > ABORT :R3 ABORT > $ clisp -norc problem.lisp > $ clisp -q -M foo.mem > [1]> > > When read-history is defined with defun, it makes same problem even > when not compiled. a callback is allocated on the heap (with malloc) and it cannot be saved into an image. the reason you are not getting the error under some conditions is that there is no attempt made to re-use the saved callback because the lisp function corresponding to the callback was not found in the callback hash table. PROPOSED SOLUTION: 1. exit_ffi() should release all callbacks and replace them in O(foreign_callin_vector) with NIL. this is not strictly necessary (you do not have to free all your mallocs before exit, but I think this is "good taste"). Bruno, Joerg, should we do this? 2. when convert_function_to_foreign() finds in invalid (or NIL) triple[1], it should just call alloc_callback(); see the appended patch. Tomas, could you please try this patch? it appears to fix the bug you reported. -- Sam Steingold (http://www.podval.org/~sds) running w2k <http://www.memri.org/> <http://truepeace.org> <http://www.honestreporting.com> <http://www.camera.org> <http://www.savegushkatif.org> <http://pmw.org.il/> If abortion is murder, then oral sex is cannibalism. --- foreign.d 21 Sep 2005 14:10:34 -0400 1.144 +++ foreign.d 29 Sep 2005 18:20:45 -0400 @@ -442,12 +442,23 @@ if (equal_fvd(resulttype,Car(acons)) && equal_argfvds(argtypes,Car(Cdr(acons))) && eq(flags,Car(Cdr(Cdr(acons))))) { - var gcv_object_t* triple = &TheSvector(TheIarray(O(foreign_callin_vector))->data)->data[3*posfixnum_to_V(Cdr(Cdr(Cdr(acons))))-2]; - triple[2] = fixnum_inc(triple[2],1); /* increment reference count */ + var uintV f_index = posfixnum_to_V(Cdr(Cdr(Cdr(acons)))); + var gcv_object_t* triple = &TheSvector(TheIarray(O(foreign_callin_vector))->data)->data[3*f_index-2]; var object ffun = triple[1]; ASSERT(equal_fvd(resulttype,TheFfunction(ffun)->ff_resulttype)); ASSERT(equal_argfvds(argtypes,TheFfunction(ffun)->ff_argtypes)); ASSERT(eq(flags,TheFfunction(ffun)->ff_flags)); + var object faddress = TheFfunction(ffun)->ff_address; + if (fp_validp(TheFpointer(TheFaddress(faddress)->fa_base))) { + triple[2] = fixnum_inc(triple[2],1); /* increment reference count */ + } else { /* callback from the previous session -- renew */ + /* leave the reference count alone */ + begin_system_call(); + TheFaddress(faddress)->fa_offset = + (uintP)alloc_callback(&callback,(void*)(uintP)f_index); + end_system_call(); + TheFaddress(faddress)->fa_base = O(fp_zero); + } return ffun; } } |
From: Tomas Z. <zel...@gm...> - 2005-10-02 23:21:09
|
2005/9/30, Sam Steingold <sd...@gn...>: > > * Tomas Zellerin <mryyreva@tznvy.pbz> [2005-09-29 08:14:30 +0200]: > > > > while trying to make more from included readline, I encountered a > > problem that can be shown on this example: > > > > (use-package 'ffi) > > (def-call-out add-defun > > (:name "rl_add_defun") > > (:library :default) > > (:arguments (name c-string :in :malloc-free) > > (callback (c-function (:language :stdc) > > (:arguments (rep int) (char int))) > > :in :malloc-free) > > (key int)) > > (:language :stdc) > > (:return-type int)) > > (defun init () > > (labels ((edit-history (key rep) > > (print key) > > (print rep))) > > (add-defun "edit-history" #'edit-history -1))) > > (init) > > (push #'init *init-hooks*) > > (saveinitmem "foo") > > > > That is, a function with callback to linux is defined and then called. > > Now, this works fine when loaded (compiled or not), but when later > > loaded from image, it makes problem when compiled: > > > > $ clisp -norc -C problem.lisp > > $ clisp -q -M foo.mem > > > > *** - FFI::FOREIGN-CALL-OUT: #<INVALID FOREIGN-POINTER #x00000000> come= s from > > a previous Lisp session and is invalid > > The following restarts are available: > > SKIP :R1 skip (SAVEINITMEM foo) > > STOP :R2 stop loading file /home/tomas/src/misc/problem.= lisp > > ABORT :R3 ABORT > > $ clisp -norc problem.lisp > > $ clisp -q -M foo.mem > > [1]> > > > > When read-history is defined with defun, it makes same problem even > > when not compiled. > > a callback is allocated on the heap (with malloc) and it cannot be saved > into an image. > the reason you are not getting the error under some conditions is that > there is no attempt made to re-use the saved callback because the lisp > function corresponding to the callback was not found in the callback > hash table. > > PROPOSED SOLUTION: > > 1. exit_ffi() should release all callbacks and replace them in > O(foreign_callin_vector) with NIL. > this is not strictly necessary (you do not have to free all your > mallocs before exit, but I think this is "good taste"). > Bruno, Joerg, should we do this? > > 2. when convert_function_to_foreign() finds in invalid (or NIL) > triple[1], it should just call alloc_callback(); > see the appended patch. > Tomas, could you please try this patch? > it appears to fix the bug you reported. The patch worked and helped - thanks. Tomas > > -- > Sam Steingold (http://www.podval.org/~sds) running w2k > <http://www.memri.org/> <http://truepeace.org> <http://www.honestreportin= g.com> > <http://www.camera.org> <http://www.savegushkatif.org> <http://pmw.org.il= /> > If abortion is murder, then oral sex is cannibalism. > > > --- foreign.d 21 Sep 2005 14:10:34 -0400 1.144 > +++ foreign.d 29 Sep 2005 18:20:45 -0400 > @@ -442,12 +442,23 @@ > if (equal_fvd(resulttype,Car(acons)) > && equal_argfvds(argtypes,Car(Cdr(acons))) > && eq(flags,Car(Cdr(Cdr(acons))))) { > - var gcv_object_t* triple =3D &TheSvector(TheIarray(O(foreign_c= allin_vector))->data)->data[3*posfixnum_to_V(Cdr(Cdr(Cdr(acons))))-2]; > - triple[2] =3D fixnum_inc(triple[2],1); /* increment reference = count */ > + var uintV f_index =3D posfixnum_to_V(Cdr(Cdr(Cdr(acons)))); > + var gcv_object_t* triple =3D &TheSvector(TheIarray(O(foreign_c= allin_vector))->data)->data[3*f_index-2]; > var object ffun =3D triple[1]; > ASSERT(equal_fvd(resulttype,TheFfunction(ffun)->ff_resulttype))= ; > ASSERT(equal_argfvds(argtypes,TheFfunction(ffun)->ff_argtypes))= ; > ASSERT(eq(flags,TheFfunction(ffun)->ff_flags)); > + var object faddress =3D TheFfunction(ffun)->ff_address; > + if (fp_validp(TheFpointer(TheFaddress(faddress)->fa_base))) { > + triple[2] =3D fixnum_inc(triple[2],1); /* increment referenc= e count */ > + } else { /* callback from the previous session -- renew */ > + /* leave the reference count alone */ > + begin_system_call(); > + TheFaddress(faddress)->fa_offset =3D > + (uintP)alloc_callback(&callback,(void*)(uintP)f_index); > + end_system_call(); > + TheFaddress(faddress)->fa_base =3D O(fp_zero); > + } > return ffun; > } > } > |
From: Tomas Z. <zel...@gm...> - 2005-09-30 06:36:12
|
Thanks, I'll try the patch and report. Looking at it for another time, I should probably try it again without calling (init) - my intention was to create the callback at *init-hooks* time anyway. I am still a bit confused - I thought the call to init before image saving would have no effect when loading image, as clisp has no way of knowing what was saved by another library (readline), and I do not understand why calling the function to get callback at *init-hooks* time should produce invalid reference. As I've said, I'll report about result of patch. BTW, replacing :default with library name string did not help. Thanks, Tomas Zellerin 2005/9/30, Sam Steingold <sd...@gn...>: > > * Tomas Zellerin <mryyreva@tznvy.pbz> [2005-09-29 08:14:30 +0200]: > > > > while trying to make more from included readline, I encountered a > > problem that can be shown on this example: > > > > (use-package 'ffi) > > (def-call-out add-defun > > (:name "rl_add_defun") > > (:library :default) > > (:arguments (name c-string :in :malloc-free) > > (callback (c-function (:language :stdc) > > (:arguments (rep int) (char int))) > > :in :malloc-free) > > (key int)) > > (:language :stdc) > > (:return-type int)) > > (defun init () > > (labels ((edit-history (key rep) > > (print key) > > (print rep))) > > (add-defun "edit-history" #'edit-history -1))) > > (init) > > (push #'init *init-hooks*) > > (saveinitmem "foo") > > > > That is, a function with callback to linux is defined and then called. > > Now, this works fine when loaded (compiled or not), but when later > > loaded from image, it makes problem when compiled: > > > > $ clisp -norc -C problem.lisp > > $ clisp -q -M foo.mem > > > > *** - FFI::FOREIGN-CALL-OUT: #<INVALID FOREIGN-POINTER #x00000000> come= s from > > a previous Lisp session and is invalid > > The following restarts are available: > > SKIP :R1 skip (SAVEINITMEM foo) > > STOP :R2 stop loading file /home/tomas/src/misc/problem.= lisp > > ABORT :R3 ABORT > > $ clisp -norc problem.lisp > > $ clisp -q -M foo.mem > > [1]> > > > > When read-history is defined with defun, it makes same problem even > > when not compiled. > > a callback is allocated on the heap (with malloc) and it cannot be saved > into an image. > the reason you are not getting the error under some conditions is that > there is no attempt made to re-use the saved callback because the lisp > function corresponding to the callback was not found in the callback > hash table. > > PROPOSED SOLUTION: > > 1. exit_ffi() should release all callbacks and replace them in > O(foreign_callin_vector) with NIL. > this is not strictly necessary (you do not have to free all your > mallocs before exit, but I think this is "good taste"). > Bruno, Joerg, should we do this? > > 2. when convert_function_to_foreign() finds in invalid (or NIL) > triple[1], it should just call alloc_callback(); > see the appended patch. > Tomas, could you please try this patch? > it appears to fix the bug you reported. > > -- > Sam Steingold (http://www.podval.org/~sds) running w2k > <http://www.memri.org/> <http://truepeace.org> <http://www.honestreportin= g.com> > <http://www.camera.org> <http://www.savegushkatif.org> <http://pmw.org.il= /> > If abortion is murder, then oral sex is cannibalism. > > > --- foreign.d 21 Sep 2005 14:10:34 -0400 1.144 > +++ foreign.d 29 Sep 2005 18:20:45 -0400 > @@ -442,12 +442,23 @@ > if (equal_fvd(resulttype,Car(acons)) > && equal_argfvds(argtypes,Car(Cdr(acons))) > && eq(flags,Car(Cdr(Cdr(acons))))) { > - var gcv_object_t* triple =3D &TheSvector(TheIarray(O(foreign_c= allin_vector))->data)->data[3*posfixnum_to_V(Cdr(Cdr(Cdr(acons))))-2]; > - triple[2] =3D fixnum_inc(triple[2],1); /* increment reference = count */ > + var uintV f_index =3D posfixnum_to_V(Cdr(Cdr(Cdr(acons)))); > + var gcv_object_t* triple =3D &TheSvector(TheIarray(O(foreign_c= allin_vector))->data)->data[3*f_index-2]; > var object ffun =3D triple[1]; > ASSERT(equal_fvd(resulttype,TheFfunction(ffun)->ff_resulttype))= ; > ASSERT(equal_argfvds(argtypes,TheFfunction(ffun)->ff_argtypes))= ; > ASSERT(eq(flags,TheFfunction(ffun)->ff_flags)); > + var object faddress =3D TheFfunction(ffun)->ff_address; > + if (fp_validp(TheFpointer(TheFaddress(faddress)->fa_base))) { > + triple[2] =3D fixnum_inc(triple[2],1); /* increment referenc= e count */ > + } else { /* callback from the previous session -- renew */ > + /* leave the reference count alone */ > + begin_system_call(); > + TheFaddress(faddress)->fa_offset =3D > + (uintP)alloc_callback(&callback,(void*)(uintP)f_index); > + end_system_call(); > + TheFaddress(faddress)->fa_base =3D O(fp_zero); > + } > return ffun; > } > } > |
From: Sam S. <sd...@gn...> - 2005-09-30 13:51:35
|
> * Tomas Zellerin <mryyreva@tznvy.pbz> [2005-09-30 08:36:04 +0200]: > > I am still a bit confused - I thought the call to init before image > saving would have no effect when loading image, as clisp has no way of > knowing what was saved by another library (readline), and I do not > understand why calling the function to get callback at *init-hooks* > time should produce invalid reference. the list of callbacks you create is kept in the lisp heap, so it is saved into the image and restored from it. alas, the restored list points to invalid C callback memory area. -- Sam Steingold (http://www.podval.org/~sds) running w2k <http://www.savegushkatif.org> <http://www.dhimmi.com/> <http://www.openvotingconsortium.org/> <http://truepeace.org> Software is like sex: it's better when it's free. |