Re: gsoc: New compiler (almost) ANSI conforming, progress report.

[Charles Z. <ka...@be...>]

Hello Bruno,

> Oh, I would have thought that dead code elimination, copy propagation,
> and constant propagation would be built-in in cleavir? Can some of
> this code be propagated back into SICL?

SICL is very new, so it is lacking in many optimizations. The existing
code in SICL is enough to get a correct compiler working. There is
also a local inlining pass. So, I've needed to write almost all the
optimizations passes myself. I've spent much of this period sending
pull requests upstream to make local inlining faster and contributing
some components I've written such as:

- Basic blocks
- Flow graph loop code
- Much faster local inlining and compilation times with CSTs.

I am working on upstreaming more components so that there is less
maintenance work on CLISP for things that are independent of the
Cleavir/CLISP interface.

Charles

On Wed, Jul 18, 2018 at 8:46 PM, Bruno Haible <br...@cl...> wrote:
> Hi Charles,
>
> Replying to your mail from June 3:
>> See some issues I opened on GitLab. ...
>> So far, I have handled macroexpansion by just calling macroexpanders
>> with the null macroexpansion environment #(NIL NIL). Since I am trying
>> to leverage CLISP's macro definitions as much as possible, I'd like to
>> know for what type of macros this will break down for CLISP. I'm not
>> quite sure which information I really need to make all macros work.
>
> If a macro expander ignores its macroexpansion environment (like you
> discovered is the case for MULTIPLE-VALUE-SETQ), it is a bug.
>
>> About the compiler, the code produced currently looks like it is
>> promising in some ways, and bad in others. For example, since I am
>> translating an SSA flow graph to byte code directly, the code looks
>> like it is trying to use the stack as a register machine, and
>> therefore at times LOADs and and PUSHs a lot. On the other hand, flow
>> sensitive information is readily apparent and it should be easy to
>> write previously impossible optimizations. To demonstrate ...
>
> Very nice!
>
>> the Cleavir code does not mention the constant 'e at all, because it's
>> obvious from the SSA flow graph that 'e is never used
>
> Marvellous! This is really what the old clisp compiler is not good at.
>
>> The passes that I've already written myself
>> are the SSA conversion, dead code elimination, and copy propogation
>> passes.
>
> Oh, I would have thought that dead code elimination, copy propagation,
> and constant propagation would be built-in in cleavir? Can some of
> this code be propagated back into SICL?
>
>> For now, I am still focusing on correctness, before jumping to more
>> sophisticated optimizations like conditional constant propogation or
>> fixing some things like special casing for the various branch
>> instructions. I wonder how I should integrate this new compiler to
>> toplevel functions like COMPILE-FILE/EVAL. Would it make sense to have
>> a dynamic variable like *use-cleavir* that switches on the new
>> compiler, or a keyword parameter to COMPILE/COMPILE-FILE? When I tried
>> using a dynamic variable, I ran into the issue where CLOS methods
>> would try and randomly recompile using Cleavir when I had only meant
>> to compile one form with Cleavir, since it's heavily CLOS based.
>
> When you start using a dynamic variable like *use-cleavir*, immediately
> the question comes up how to limit it to a particular scope. Then one
> starts to write
>   (defun foo ...
>     (compiler-let ((*use-cleavir* t))
>       ...))
> and it gets ugly. Source code should not contain operational instructions
> for the compiler, only declarations.
>
> At some point we could have a logic that evaluates the OPTIMIZE declarations.
> But it is also good to have a completely unambiguous way to say
> "this code is to be compiled by cleavir".
>
> clisp already has a non-standardized declaration (COMPILE)
> https://clisp.sourceforge.io/impnotes/declarations.html#compile-decl
> It says "this code is to be compiled".
>
> Maybe we could have a declaration (COMPILER [basic|cleavir]) that specifies
> which compiler to use, when compilation is requested. Like this:
>
> Or maybe add an argument to the COMPILE declaration:
>   (locally (declare (compile) (compiler basic)) ...)   ; uses the original compiler
>   (locally (declare (compile) (compiler cleavir)) ...) ; uses the cleavir compiler
>
> For COMPILE-FILE, one needs a global setting. This global setting
> would be available through DECLAIM:
>   (declaim (compiler cleavir))
>
> With this proposal,
>   - The compiler can be chosen locally or globally.
>   - The mechanism integrates with the Common Lisp declaration system.
>   - Interpreted code can continue to be interpreted.
>   - The logic that evaluates the OPTIMIZE declarations can be decided upon
>     independently.
>
> Bruno
>