From: Ville V. <vil...@gm...> - 2008-10-08 18:27:10
|
First the bad news: I have no solution yet. The undefined function seems to be (setf %m) That's what the repl says when evaluating the forms. The PUSH.5 form for example is (MACROLET ((%M (Z) Z)) (LET ((X NIL)) (VALUES (PUSH 1 (EXPAND-IN-CURRENT-ENV (%M X))) X))) (push 1..) of course ultimately expands to setf. Looks like somehow the macroexpand fails to expand. expand-in-current-env is defined in ansi-aux.lsp and is (defmacro expand-in-current-env (macro-form &environment env) (macroexpand macro-form env)) Erik and I talked about some macrolet form a while ago, and for some reason abcl does not macroexpand macros defined with macrolet, I just tried it with a simple macro and nothing whatsoever expands.. |
From: Ville V. <vil...@gm...> - 2008-10-08 19:17:27
|
On Wed, Oct 8, 2008 at 9:26 PM, Ville Voutilainen <vil...@gm...> wrote: > Erik and I talked about some macrolet form a while ago, and for some reason abcl > does not macroexpand macros defined with macrolet, I just tried it with a simple > macro and nothing whatsoever expands.. So. The expand-in-current-env invokes macroexpand. That is in Lisp.java, and calls macroexpand_1, again in Lisp.java. The snippet in Lisp.java macroexpand_1() function is if (car instanceof Symbol) { LispObject obj = env.lookupFunction(car); // this is null when we have a macrolet-defined macro!!!!! if (obj instanceof Autoload) { Autoload autoload = (Autoload) obj; autoload.load(); obj = car.getSymbolFunction(); } if (obj instanceof SpecialOperator) I don't know yet why the lookupFunction fails to find the %m in our case. Will continue investigations. Maybe the expander is not in the environment? Maybe the environment itself is somehow incorrect? -VJV- P.S. Erik, if you have any ideas.. ..I could use some help with this. :) |
From: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - 2008-10-08 19:27:12
|
On Wed, Oct 8, 2008 at 9:17 PM, Ville Voutilainen <vil...@gm...> wrote: > On Wed, Oct 8, 2008 at 9:26 PM, Ville Voutilainen > <vil...@gm...> wrote: >> Erik and I talked about some macrolet form a while ago, and for some reason abcl >> does not macroexpand macros defined with macrolet, I just tried it with a simple >> macro and nothing whatsoever expands.. > > So. The expand-in-current-env invokes macroexpand. That is in Lisp.java, and > calls macroexpand_1, again in Lisp.java. > > The snippet in Lisp.java macroexpand_1() function is > > if (car instanceof Symbol) > { > LispObject obj = env.lookupFunction(car); // this is null > when we have a macrolet-defined macro!!!!! > if (obj instanceof Autoload) > { > Autoload autoload = (Autoload) obj; > autoload.load(); > obj = car.getSymbolFunction(); > } > if (obj instanceof SpecialOperator) > > I don't know yet why the lookupFunction fails to find the %m in our > case. Will continue investigations. > Maybe the expander is not in the environment? Maybe the environment > itself is somehow incorrect? Which environment gets passed to the expand-in-current-env macro? One in which the expander function is defined? > -VJV- > > P.S. Erik, if you have any ideas.. ..I could use some help with this. :) > Bye, Erik. |
From: Ville V. <vil...@gm...> - 2008-10-08 19:49:10
|
On Wed, Oct 8, 2008 at 10:26 PM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: > Which environment gets passed to the expand-in-current-env macro? One > in which the expander function is defined? Does not look like it, the environment passed to expand-in-current-env contains no function bindings and no block. I gather the expander is thus not in the environment either. |
From: Ville V. <vil...@gm...> - 2008-10-08 19:58:03
|
> On Wed, Oct 8, 2008 at 10:26 PM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: >> Which environment gets passed to the expand-in-current-env macro? One > Does not look like it, the environment passed to expand-in-current-env > contains no function bindings and no block. I gather the expander is thus > not in the environment either. It looks like pushnew is macroexpanded with a NIL environment. I'd think this means that there's no chance that anything within the pushnew would have the expander for %m, including the expand-in-current-env itself. Is this analysis correct? Now we just have to find why the proper environment is not passed all the way. precompiler seems to be involved, according to the call traces I see in debugger.. |
From: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - 2008-10-08 20:33:03
|
On Wed, Oct 8, 2008 at 9:49 PM, Ville Voutilainen <vil...@gm...> wrote: > On Wed, Oct 8, 2008 at 10:26 PM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: >> Which environment gets passed to the expand-in-current-env macro? One >> in which the expander function is defined? > > Does not look like it, the environment passed to expand-in-current-env > contains no function bindings and no block. I gather the expander is thus > not in the environment either. Yup. That would be a reason, yes. I see in your other mail you have some stack traces. Could you put those here or in a pastebin somewhere? If the macroexpanders don't pass the environment down to the lowest level, that would explain the problem, I'd say. Bye, Erik. |
From: Ville V. <vil...@gm...> - 2008-10-08 20:39:05
|
On Wed, Oct 8, 2008 at 11:32 PM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: > I see in your other mail you have some stack traces. Could you put > those here or in a pastebin somewhere? If the macroexpanders don't > pass the environment down to the lowest level, that would explain the > problem, I'd say. I can take a trace from the eclipse debugger and I can also get the value of the environment. It just requires some manual work. You can actually see this yourself if you take the expand-in-current-env macro and one of the test cases in subject and run them with abcl and debug with eclipse or netbeans. If someone has ideas how I could get the stack trace with the environment value from eclipse easily, I'm all ears. -VJV- |
From: Ville V. <vil...@gm...> - 2008-10-08 20:53:32
Attachments:
pushnew-call-trace
|
The call trace is easy to get. Attached for the pushnew.21 case. macroexpand of %m is Primitives$112.execute(LispObject, LispObject) line: 2961, very near the top. This block is where environment is nil: Primitives$113.execute(LispObject, LispObject) line: 2980 Symbol.execute(LispObject, LispObject) line: 721 LispThread.execute(LispObject, LispObject, LispObject) line: 647 setf_2.execute(LispObject, LispObject) line: 37 Symbol.execute(LispObject, LispObject) line: 721 LispThread.execute(LispObject, LispObject, LispObject) line: 647 setf_3.execute(LispObject, LispObject) line: 47 Symbol.execute(LispObject, LispObject) line: 721 setf_3.execute(LispObject) line: 47 Symbol.execute(LispObject) line: 708 LispThread.execute(LispObject, LispObject) line: 626 macros_14.execute(LispObject, LispObject) line: 79 The Primitives$113 is macroexpand of expand-in-current-env. The macros_14 is pushnew. The macroexpand creates an empty environment if null is given to it, as happens here. Thus after this the environment is non-null, but contains nothing of use. HTH, -VJV- |
From: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - 2008-10-08 21:01:28
|
Well, looking at macros.lisp, pushnew macro, I don't see it taking an environment argument nor passing that onto the macroexpand call. Nor does it try to pass the current environment to get-setf-expansion, which also seems to be able to work with that argument when required. Both seem not adapted to the situation expand-in-current-env seems to try to create. Does that analysis help? Bye, Erik. On Wed, Oct 8, 2008 at 10:53 PM, Ville Voutilainen <vil...@gm...> wrote: > The call trace is easy to get. Attached for the pushnew.21 case. > macroexpand of %m is > Primitives$112.execute(LispObject, LispObject) line: 2961, very near the top. > > This block is where environment is nil: > > Primitives$113.execute(LispObject, LispObject) line: 2980 > Symbol.execute(LispObject, LispObject) line: 721 > LispThread.execute(LispObject, LispObject, LispObject) line: 647 > setf_2.execute(LispObject, LispObject) line: 37 > Symbol.execute(LispObject, LispObject) line: 721 > LispThread.execute(LispObject, LispObject, LispObject) line: 647 > setf_3.execute(LispObject, LispObject) line: 47 > Symbol.execute(LispObject, LispObject) line: 721 > setf_3.execute(LispObject) line: 47 > Symbol.execute(LispObject) line: 708 > LispThread.execute(LispObject, LispObject) line: 626 > macros_14.execute(LispObject, LispObject) line: 79 > > The Primitives$113 is macroexpand of expand-in-current-env. The > macros_14 is pushnew. The macroexpand > creates an empty environment if null is given to it, as happens here. > Thus after this the environment is non-null, > but contains nothing of use. > > HTH, > -VJV- > |
From: Ville V. <vil...@gm...> - 2008-10-09 19:10:05
|
On Thu, Oct 9, 2008 at 12:01 AM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: > Well, looking at macros.lisp, pushnew macro, I don't see it taking an > environment argument nor passing that onto the macroexpand call. Nor IMHO this does not matter, PUSHNEW.20 works and it's not that different. > does it try to pass the current environment to get-setf-expansion, > which also seems to be able to work with that argument when required. Now I think we're getting closer. I took a look at the setf-expander in this case: CL-USER(4): (GET-SETF-EXPANSION '(EXPAND-IN-CURRENT-ENV (%M X))) (#:G48) (X) (#:G47) (FUNCALL (FUNCTION (SETF %M)) #:G47 #:G48) (%M #:G48) (function (setf..))? Huh? Quoth the hyperspec from http://www.lispworks.com/documentation/HyperSpec/Body/s_fn.htm "Specifically, it is an error to use function on a symbol that denotes a macro or special form." AFAIK setf is just that, a macro or a special form. That's why the test says such a function is undefined, because it isn't a function. I think the next step would be to find out why get-setf-expansion returns such an expander. |
From: Ville V. <vil...@gm...> - 2008-10-09 19:41:08
|
On Thu, Oct 9, 2008 at 10:09 PM, Ville Voutilainen <vil...@gm...> wrote: > (FUNCALL (FUNCTION (SETF %M)) #:G47 #:G48) This probably comes from (in setf.lisp where get-setf-expansion lives) (defun expand-or-get-setf-inverse (form environment) (multiple-value-bind (expansion expanded) (macroexpand-1 form environment) (if expanded (get-setf-expansion expansion environment) (get-setf-method-inverse form `(funcall #'(setf ,(car form))) t)))) Apparently fails to expand, so expanded is nil, so returns the ill form that we get. So it's an expansion problem, probably because our macroexpand doesn't get the proper environment, as suggested. Back to square one, I suppose. |
From: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - 2008-10-09 20:37:47
|
On Thu, Oct 9, 2008 at 9:40 PM, Ville Voutilainen <vil...@gm...> wrote: > On Thu, Oct 9, 2008 at 10:09 PM, Ville Voutilainen > <vil...@gm...> wrote: >> (FUNCALL (FUNCTION (SETF %M)) #:G47 #:G48) > > This probably comes from (in setf.lisp where get-setf-expansion lives) > (defun expand-or-get-setf-inverse (form environment) > (multiple-value-bind (expansion expanded) > (macroexpand-1 form environment) > (if expanded > (get-setf-expansion expansion environment) > (get-setf-method-inverse form `(funcall #'(setf ,(car form))) > t)))) > > Apparently fails to expand, so expanded is nil, so returns the ill > form that we get. So it's an expansion > problem, probably because our macroexpand doesn't get the proper environment, as > suggested. Back to square one, I suppose. Not really square 1; we know it's a problem with expansion and environments now. The only problem is adding an environment to pushnew or push: somehow it makes the build crash with a stack overflow... Bye, Erik. |
From: Ville V. <vil...@gm...> - 2008-10-09 21:22:07
|
On Thu, Oct 9, 2008 at 11:37 PM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: >> (if expanded >> (get-setf-expansion expansion environment) >> (get-setf-method-inverse form `(funcall #'(setf ,(car form))) >> t)))) > Not really square 1; we know it's a problem with expansion and > environments now. The only problem is adding an environment to pushnew > or push: somehow it makes the build crash with a stack overflow... I still find the else-form of the if above quite odd, are there supposedly situations where something like that would work? setf is still not a function and I fail to see how function with setf could ever work. That's a separate problem, though. |
From: Ville V. <vil...@gm...> - 2008-10-09 22:50:47
|
On Thu, Oct 9, 2008 at 11:37 PM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: > Not really square 1; we know it's a problem with expansion and > environments now. The only problem is adding an environment to pushnew > or push: somehow it makes the build crash with a stack overflow... Would this have anything to do with the problem?: private static final Primitive MAKE_EXPANDER_FOR_MACROLET = ... LispObject envArg = gensym("ENVIRONMENT-", thread); // Ignored. LispObject expander = list3(Symbol.LAMBDA, list2(formArg, envArg), list3(Symbol.APPLY, toBeApplied, list2(Symbol.CDR, formArg))); Ignored? The environment is not used? |
From: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - 2008-10-09 22:40:38
|
On Thu, Oct 9, 2008 at 11:41 PM, Ville Voutilainen <vil...@gm...> wrote: > On Thu, Oct 9, 2008 at 11:37 PM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: >> Not really square 1; we know it's a problem with expansion and >> environments now. The only problem is adding an environment to pushnew >> or push: somehow it makes the build crash with a stack overflow... > > Would this have anything to do with the problem?: > private static final Primitive MAKE_EXPANDER_FOR_MACROLET = > ... > LispObject envArg = gensym("ENVIRONMENT-", thread); // Ignored. > LispObject expander = > list3(Symbol.LAMBDA, list2(formArg, envArg), > list3(Symbol.APPLY, toBeApplied, > list2(Symbol.CDR, formArg))); > > Ignored? The environment is not used? > It would be nice to understand how that behaves together with: LispObject expander = make_expander_for_macrolet.execute(def); Closure expansionFunction = new Closure(expander, env); MacroObject macroObject = new MacroObject(symbol, expansionFunction); ext.addFunctionBinding(symbol, macroObject); (from macrolet in primitives.java). Bye, Erik. |
From: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX - 2008-10-09 22:39:43
|
On Thu, Oct 9, 2008 at 10:37 PM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: > On Thu, Oct 9, 2008 at 9:40 PM, Ville Voutilainen > <vil...@gm...> wrote: >> On Thu, Oct 9, 2008 at 10:09 PM, Ville Voutilainen >> <vil...@gm...> wrote: >>> (FUNCALL (FUNCTION (SETF %M)) #:G47 #:G48) >> >> This probably comes from (in setf.lisp where get-setf-expansion lives) >> (defun expand-or-get-setf-inverse (form environment) >> (multiple-value-bind (expansion expanded) >> (macroexpand-1 form environment) >> (if expanded >> (get-setf-expansion expansion environment) >> (get-setf-method-inverse form `(funcall #'(setf ,(car form))) >> t)))) >> >> Apparently fails to expand, so expanded is nil, so returns the ill >> form that we get. So it's an expansion >> problem, probably because our macroexpand doesn't get the proper environment, as >> suggested. Back to square one, I suppose. > > Not really square 1; we know it's a problem with expansion and > environments now. The only problem is adding an environment to pushnew > or push: somehow it makes the build crash with a stack overflow... Do you see this too? If so, from the debugger, can you find the source of the endless loop of stackcalls? Bye, Erik. |
From: Ville V. <vil...@gm...> - 2008-10-09 23:49:35
|
On Fri, Oct 10, 2008 at 12:30 AM, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX wrote: >> environments now. The only problem is adding an environment to pushnew >> or push: somehow it makes the build crash with a stack overflow... > Do you see this too? If so, from the debugger, can you find the source > of the endless loop of stackcalls? Yes, I see it too. I haven't debugged it much, but I've seen the exception stacktrace of it. I don't have it at hand, I can dig it up tomorrow. |