From: Erik H. <eh...@gm...> - 2008-12-30 10:59:15
|
On IRC (where there's a lot of activity as of lately; see irc://irc.freenode.net/#abcl), I've been talking about the plan to want to clone() ClosureTemplateFunction. This is part of my plan to be able to fix DEFUN.* and LABELS.* ansi tests. So, why are these tests failing? They are failing for several reasons, but they all come down to the same solution. Some fail because of special variable scoping. Others fail because they depend on FLET/LABELS defined functions for initforms of &OPTIONAL or &KEY arguments. How come? Compiled functions are created with an empty environment (null value). The compiled code compensates for this fact because it hard-codes special variable references and calls to locally defined functions. However, this information is not available at initform evaluation-time, because the initforms don't get compiled: They are evaluated in processArgs() or fastProcessArgs() [methods of the Closure class]. processArgs() and fastProcessArgs() depend on the Closure 'environment' field to represent the environment in which the function/closure was created. The compiler isn't providing this context. As I wrote before, for the compiled code itself , this isn't a problem. However, for code which tries to evaluate initforms, this *is* an issue, which is shown by the 2 procedures mentioned. Then how to resolve this? It seems all too logical to provide an environment to the closure/function so the processArgs() and fastProcessArgs() functions have the right context to do their work (i.e. evaluate their initforms, if required). The current system doesn't allow that though: compiled closures have a code-skeleton which is implemented as a derivative from ClosureTemplateFunction (CTF). This CTF is used for every closure of the same type created. This means it's not possible to store closure-specific data in the CTF. The compiler works around this issue: to create an instance of a closure, it wraps the CTF in a CompiledClosure object (CC). This CC stores a 'context' record: an array of LispObjects which describe the environment in which the closure was created. Now, this information should become available to processArgs() too. The only way to do that would be to create an Environment object and store that in the CTF. The CTF however can't contain closure-instance specific data. If we would be able to have an instance of the CTF per created closure, it would be possible to store closure-instance data (like an Environment). So, this is where my attempts to clone() CTFs come in: once cloned, they can hold specific data. Well, sorry for the long story, but this explains why I *really* want cloning to work for CTFs. Anybody with questions? Better ideas? Comments? I'd love to hear your views. Bye, Erik. |
From: Ville V. <vil...@gm...> - 2008-12-30 11:09:11
|
On Tue, Dec 30, 2008 at 12:59 PM, Erik Huelsmann <eh...@gm...> wrote: > If we would be able to have an instance of the CTF per created > closure, it would be possible to store closure-instance data (like an > Environment). So, this is where my attempts to clone() CTFs come in: > once cloned, they can hold specific data. > > Well, sorry for the long story, but this explains why I *really* want > cloning to work for CTFs. Anybody with questions? Better ideas? > Comments? The analysis seems sound to me - the closures need per-closure environment information, so that's not in question at all. clone() seems to be somehow hard to get working, maybe a copy-constructor for CTF or a custom clone function would do. We have been trying to use dup, which is supposed to call super.clone(), but AFAIK all attempts so far have failed? So maybe it's time to override clone in CTF? |
From: Erik H. <eh...@gm...> - 2008-12-30 12:14:23
|
On Tue, Dec 30, 2008 at 12:09 PM, Ville Voutilainen <vil...@gm...> wrote: > On Tue, Dec 30, 2008 at 12:59 PM, Erik Huelsmann <eh...@gm...> wrote: >> If we would be able to have an instance of the CTF per created >> closure, it would be possible to store closure-instance data (like an >> Environment). So, this is where my attempts to clone() CTFs come in: >> once cloned, they can hold specific data. >> >> Well, sorry for the long story, but this explains why I *really* want >> cloning to work for CTFs. Anybody with questions? Better ideas? >> Comments? > > The analysis seems sound to me - the closures need per-closure environment > information, so that's not in question at all. > > clone() seems to be somehow hard to get working, maybe a copy-constructor > for CTF or a custom clone function would do. We have been trying to use dup, > which is supposed to call super.clone(), but AFAIK all attempts so far have > failed? So maybe it's time to override clone in CTF? Until now, I've been avoiding constructors, because they cause the argument list to be reparsed, presumably. However, at least for now, it would be a way to show that our scheme can work. The other issue is that I *think* we're using anonymous classes, which override only 1 of the execute() methods from the CTF or Closure classes. Having a copy constructor for an anonymous class doesn't look viable. Instead of using a copy-constructor, we could force a reload from class-bytes though. For all of the above, I have *no* idea about the speed consequences. We'll need to investigate those as soon as we found a working scheme. Bye, Erik. |
From: Ville V. <vil...@gm...> - 2008-12-30 12:26:29
|
On Tue, Dec 30, 2008 at 1:50 PM, Erik Huelsmann <eh...@gm...> wrote: > The other issue is that I *think* we're using anonymous classes, which > override only 1 of the execute() methods from the CTF or Closure > classes. Having a copy constructor for an anonymous class doesn't look > viable. True. But have we ever had an overridden clone()? Let's at least eliminate that reason for cloning not working. |
From: Erik H. <eh...@gm...> - 2008-12-30 12:29:14
|
On Tue, Dec 30, 2008 at 12:54 PM, Ville Voutilainen <vil...@gm...> wrote: > On Tue, Dec 30, 2008 at 1:50 PM, Erik Huelsmann <eh...@gm...> wrote: >> The other issue is that I *think* we're using anonymous classes, which >> override only 1 of the execute() methods from the CTF or Closure >> classes. Having a copy constructor for an anonymous class doesn't look >> viable. > > True. But have we ever had an overridden clone()? Let's at least eliminate that > reason for cloning not working. No, we haven't. But maybe you can specify what you'd expect that overriden clone method to do? I mean, I started out with clone() locally, instead of dup(), but I couldn't get it to work (possibly for some other reason,it took me quite some iterations before the dup() was working in the end). I just wanted a bit-wise copy of the original, which is why I'm relying on the default clone() implementation: it's documented to do exactly that. Bye, Erik. |
From: Ville V. <vil...@gm...> - 2008-12-30 12:11:32
|
On Tue, Dec 30, 2008 at 2:00 PM, Erik Huelsmann <eh...@gm...> wrote: >> True. But have we ever had an overridden clone()? Let's at least eliminate that >> reason for cloning not working. > No, we haven't. But maybe you can specify what you'd expect that > overriden clone method to do? I mean, I started out with clone() For starters, just invoke super.clone(). Object.clone() is not accessible so we cannot invoke that outside the class hierarchy. At the same time, Cloneable must be implemented. If you've already done that, then you can ignore this idea. It's actually surprising to me that dup would invoke clone(), because http://java.sun.com/docs/books/jvms/second_edition/html/Instructions2.doc3.html doesn't seem to say that at all. |
From: Erik H. <eh...@gm...> - 2008-12-30 12:18:43
|
On Tue, Dec 30, 2008 at 1:11 PM, Ville Voutilainen <vil...@gm...> wrote: > On Tue, Dec 30, 2008 at 2:00 PM, Erik Huelsmann <eh...@gm...> wrote: >>> True. But have we ever had an overridden clone()? Let's at least eliminate that >>> reason for cloning not working. >> No, we haven't. But maybe you can specify what you'd expect that >> overriden clone method to do? I mean, I started out with clone() > > For starters, just invoke super.clone(). Object.clone() is not accessible so > we cannot invoke that outside the class hierarchy. At the same time, Cloneable > must be implemented. If you've already done that, then you can ignore this > idea. > > It's actually surprising to me that dup would invoke clone(), because > http://java.sun.com/docs/books/jvms/second_edition/html/Instructions2.doc3.html > doesn't seem to say that at all. Oh, but I'm not emitting a <dup> instruction: I'm creating a method by the name of "dup" and using invokevirtual on it. So, possibly we're not coming from the same planet yet? :-) bye, Erik. |
From: Ville V. <vil...@gm...> - 2008-12-30 12:21:39
|
On Tue, Dec 30, 2008 at 2:18 PM, Erik Huelsmann <eh...@gm...> wrote: > Oh, but I'm not emitting a <dup> instruction: I'm creating a method by > the name of "dup" and using invokevirtual on it. > So, possibly we're not coming from the same planet yet? :-) Right. How does that dup() method copy/clone the base classes? If you have that working so far, it should be doable to override clone() and invokevirtual that instead. |
From: Erik H. <eh...@gm...> - 2008-12-30 12:28:12
|
On Tue, Dec 30, 2008 at 1:21 PM, Ville Voutilainen <vil...@gm...> wrote: > On Tue, Dec 30, 2008 at 2:18 PM, Erik Huelsmann <eh...@gm...> wrote: >> Oh, but I'm not emitting a <dup> instruction: I'm creating a method by >> the name of "dup" and using invokevirtual on it. >> So, possibly we're not coming from the same planet yet? :-) > > Right. How does that dup() method copy/clone the base classes? > If you have that working so far, it should be doable to override clone() and > invokevirtual that instead. It does that by calling out to super.clone(). The only reason I'm not using the name clone() itself is because I seemed to be running into issues with the compiler where the class loader would complain about the level of protection of the clone() method. Bye, Erik. |
From: <log...@gm...> - 2008-12-30 12:28:01
|
I am about to read the this thread but I wanted to get my 2 cents in. At I am going to suggest for sanity test.. leave the compiler-pass[1|2].lisp as it is the trunk this will make it continue to call Lisp.makeCompileClosure(....) then change the code in the Lisp.java file to try out your cloneing ideas.. once you have it nailed down.. then go ahead and write the bytecode to do what works.. but confirm it does the right thing first from the .java file Current CODE: public static final LispObject makeCompiledClosure(LispObject template, LispObject[] context) throws ConditionThrowable { ClosureTemplateFunction ctf = (ClosureTemplateFunction) template; CompiledClosure result = new CompiledClosure(ctf, context); LispObject classBytes = getf(ctf.getPropertyList(), Symbol.CLASS_BYTES, NIL); if (classBytes != NIL) result.setPropertyList(list2(Symbol.CLASS_BYTES, classBytes)); return result; } I tried this myself but I could not wrap my head arround how we switch gears from CompiledClosure to CompiledTemplateFunction. public static final LispObject makeCompiledClosure(LispObject template, LispObject[] context) throws ConditionThrowable { ClosureTemplateFunction result = ((ClosureTemplateFunction) template).dup(); // alias to clone() result .setContext(context); return result; } Cloning the CompiledTemplateFunction and then setContext was still not sufficent to passing the tests. ----- Original Message ----- From: "Ville Voutilainen" <vil...@gm...> To: "Erik Huelsmann" <eh...@gm...> Cc: <arm...@li...> Sent: Tuesday, December 30, 2008 4:21 AM Subject: Re: [j-devel] The plan to fix DEFUN.* and LABELS.* (compiled) ansitests > On Tue, Dec 30, 2008 at 2:18 PM, Erik Huelsmann <eh...@gm...> wrote: >> Oh, but I'm not emitting a <dup> instruction: I'm creating a method by >> the name of "dup" and using invokevirtual on it. >> So, possibly we're not coming from the same planet yet? :-) > > Right. How does that dup() method copy/clone the base classes? > If you have that working so far, it should be doable to override clone() and > invokevirtual that instead. > > ------------------------------------------------------------------------------ > _______________________________________________ > armedbear-j-devel mailing list > arm...@li... > https://lists.sourceforge.net/lists/listinfo/armedbear-j-devel |
From: Erik H. <eh...@gm...> - 2008-12-30 12:43:26
|
On Tue, Dec 30, 2008 at 1:28 PM, <log...@gm...> wrote: > I am about to read the this thread but I wanted to get my 2 cents in. At > > I am going to suggest for sanity test.. leave the compiler-pass[1|2].lisp as > it is the trunk this will make it continue to call > Lisp.makeCompileClosure(....) then change the code in the Lisp.java file to > try out your cloneing ideas.. once you have it nailed down.. then go ahead > and write the bytecode to do what works.. but confirm it does the right > thing first from the .java file > > > Current CODE: > > public static final LispObject makeCompiledClosure(LispObject template, > LispObject[] context) > throws ConditionThrowable > { > ClosureTemplateFunction ctf = (ClosureTemplateFunction) template; > CompiledClosure result = new CompiledClosure(ctf, context); > LispObject classBytes = > getf(ctf.getPropertyList(), Symbol.CLASS_BYTES, NIL); > if (classBytes != NIL) > result.setPropertyList(list2(Symbol.CLASS_BYTES, classBytes)); > return result; > } > > > I tried this myself but I could not wrap my head arround how we switch > gears from CompiledClosure to CompiledTemplateFunction. > > > public static final LispObject makeCompiledClosure(LispObject template, > LispObject[] context) > throws ConditionThrowable > { > ClosureTemplateFunction result = ((ClosureTemplateFunction) > template).dup(); // alias to clone() > result .setContext(context); > return result; > } > > Cloning the CompiledTemplateFunction and then setContext was still not > sufficent to passing the tests. Which is basically my question: both CompiledClosure and ClosureTemplateFunction inherit from Function (the latter through Closure). So, Since CompiledClosure only overrides methods, they specify the same interface. So, from the perspective of the class hierarchy, I'm not expecting surprises. Bye, Erik. |
From: <log...@gm...> - 2008-12-30 12:38:37
|
You could have just as easily rename dup() to clone(); This: final public ClosureTemplateFunction dup() throws CloneNotSupportedException { return (ClosureTemplateFunction)super.clone();} renamed to final public ClosureTemplateFunction clone() throws CloneNotSupportedException { return (ClosureTemplateFunction)super.clone();} And then the method is publiclly available But for now you can keep it named dup() and that is just fine. > The only reason I'm not using the name clone() itself is because I > seemed to be running into issues with the compiler where the class > loader would complain about the level of protection of the clone() > method. > > Bye, > > Erik. The reasons the tests are failing is because these to peices of code ar not compatable: public static final LispObject makeCompiledClosure(LispObject template, LispObject[] context) throws ConditionThrowable { ClosureTemplateFunction ctf = (ClosureTemplateFunction) template; CompiledClosure result = new CompiledClosure(ctf, context); LispObject classBytes = getf(ctf.getPropertyList(), Symbol.CLASS_BYTES, NIL); if (classBytes != NIL) result.setPropertyList(list2(Symbol.CLASS_BYTES, classBytes)); return result; } AND public static final LispObject makeCompiledClosure(LispObject template, LispObject[] context) throws ConditionThrowable { ClosureTemplateFunction result = ((ClosureTemplateFunction) template).dup(); // alias to clone() result .setContext(context); return result; } > > ------------------------------------------------------------------------------ > _______________________________________________ > armedbear-j-devel mailing list > arm...@li... > https://lists.sourceforge.net/lists/listinfo/armedbear-j-devel |
From: Erik H. <eh...@gm...> - 2008-12-30 12:41:53
|
On Tue, Dec 30, 2008 at 1:38 PM, <log...@gm...> wrote: > You could have just as easily rename dup() to clone(); > > This: > > final public ClosureTemplateFunction dup() throws > CloneNotSupportedException > { return (ClosureTemplateFunction)super.clone();} > > renamed to > final public ClosureTemplateFunction clone() throws > CloneNotSupportedException > { return (ClosureTemplateFunction)super.clone();} > > > > And then the method is publiclly available > But for now you can keep it named dup() and that is just fine. > >> The only reason I'm not using the name clone() itself is because I >> seemed to be running into issues with the compiler where the class >> loader would complain about the level of protection of the clone() >> method. >> >> Bye, >> >> Erik. > > The reasons the tests are failing is because these to peices of code ar not > compatable: > > > > public static final LispObject makeCompiledClosure(LispObject template, > LispObject[] context) > throws ConditionThrowable > { > ClosureTemplateFunction ctf = (ClosureTemplateFunction) template; > CompiledClosure result = new CompiledClosure(ctf, context); > LispObject classBytes = > getf(ctf.getPropertyList(), Symbol.CLASS_BYTES, NIL); > if (classBytes != NIL) > result.setPropertyList(list2(Symbol.CLASS_BYTES, classBytes)); > return result; > } > > AND > > public static final LispObject makeCompiledClosure(LispObject template, > LispObject[] context) > throws ConditionThrowable > { > ClosureTemplateFunction result = ((ClosureTemplateFunction) > template).dup(); // alias to clone() > result .setContext(context); > return result; > } Ok. I accept that as a given, but I don't understand WHY they don't have the same effect. If someone could explain that to me, then I could probably come up with a solution to what I want... The thing is that I could keep instantiating off the class bytes every time, but for now, it would look like .clone() is supposed to be more efficient... Could you test by reloading off the class bytes every time a new one is constructed? Bye, Erik. |
From: <log...@gm...> - 2008-12-31 02:08:48
|
Sorry I missed everyone on irc today! I might be not really here tommorow even though my irc nick will be there. We'll see though.. We all have strange timezones that make us somehow intersect. When I get back in a couple days I think I see how to get CompileTemplateFunctions (CTFs) to do the correct thing and act more like CompliledClosures (CCs). Usually CCs just wrap a CTF and do nothing usefull on their own. Except they allow its own context field to override the CTF it wraps. So anyway pretty much am sure know how to fix the clone() thing work and pass the tests.. Just the new years holiday is holding me hostage. :) ----- Original Message ----- From: "Erik Huelsmann" <eh...@gm...> To: <dm...@us...> Cc: "Ville Voutilainen" <vil...@gm...>; <arm...@li...> Sent: Tuesday, December 30, 2008 4:43 AM Subject: Re: [j-devel] The plan to fix DEFUN.* and LABELS.* (compiled) ansitests > On Tue, Dec 30, 2008 at 1:28 PM, <log...@gm...> wrote: >> I am about to read the this thread but I wanted to get my 2 cents in. At >> >> I am going to suggest for sanity test.. leave the compiler-pass[1|2].lisp as >> it is the trunk this will make it continue to call >> Lisp.makeCompileClosure(....) then change the code in the Lisp.java file to >> try out your cloneing ideas.. once you have it nailed down.. then go ahead >> and write the bytecode to do what works.. but confirm it does the right >> thing first from the .java file >> >> >> Current CODE: >> >> public static final LispObject makeCompiledClosure(LispObject template, >> LispObject[] context) >> throws ConditionThrowable >> { >> ClosureTemplateFunction ctf = (ClosureTemplateFunction) template; >> CompiledClosure result = new CompiledClosure(ctf, context); >> LispObject classBytes = >> getf(ctf.getPropertyList(), Symbol.CLASS_BYTES, NIL); >> if (classBytes != NIL) >> result.setPropertyList(list2(Symbol.CLASS_BYTES, classBytes)); >> return result; >> } >> >> >> I tried this myself but I could not wrap my head arround how we switch >> gears from CompiledClosure to CompiledTemplateFunction. >> >> >> public static final LispObject makeCompiledClosure(LispObject template, >> LispObject[] context) >> throws ConditionThrowable >> { >> ClosureTemplateFunction result = ((ClosureTemplateFunction) >> template).dup(); // alias to clone() >> result .setContext(context); >> return result; >> } >> >> Cloning the CompiledTemplateFunction and then setContext was still not >> sufficent to passing the tests. > > Which is basically my question: both CompiledClosure and > ClosureTemplateFunction inherit from Function (the latter through > Closure). So, Since CompiledClosure only overrides methods, they > specify the same interface. > > So, from the perspective of the class hierarchy, I'm not expecting surprises. > > Bye, > > Erik. |