#25 JIT via lightning

closed-accepted
Sam Steingold
None
5
2013-11-28
2008-01-27
Yann N. Dauphin
No

Still incomplete JIT compiler for CLISP.

The jit compilation occurs at every function call(no reuse), so it runs slower for now.

Lightning is used because:
- It's portable. Can reportedly be ported under a day.
- It's simple. If the GNU lightning project was to stop, it wouldn't be a problem. Maintaining it is easy. Not so for LLVM.
- It's lean and mean. Code generation is straightforward, so very fast.

Tested on x86 with:

- Linux
- Mac OS X: Must 'export DYLD_BIND_AT_LAUNCH=' for now because the dynamic linker messes with jit function calls

The patch is too big for SF so:
http://www.step.polymtl.ca/~polyrad/polyrad/jit-clisp-2.43.patch

Discussion

<< < 1 2 (Page 2 of 2)
  • Sam Steingold
    Sam Steingold
    2008-03-16

    Logged In: YES
    user_id=5735
    Originator: NO

    I get this:
    In file included from ../src/eval.d:8265:
    lightning.c: In function 'jit_compile_':
    lightning.c:5448: error: invalid operands to binary -
    ideas?

     
  • Logged In: YES
    user_id=1993164
    Originator: YES

    I don't get an error, but I guess the following should work:
    Change line 5448 of lightning.c to:
    pushSTACK(fixnum(byteptr - (const uintB*)codeptr->data - byteptr_min));

    On a separate matter, change line 170 of spvw_gcmark. to:
    if (cclosurep(curr) && fpointerp(cclosure_jitc(curr))) gc_mark_jitc_object(TheFpointer(cclosure_jitc(curr))->fp_pointer)

    Tell me how it works out

     
  • Sam Steingold
    Sam Steingold
    2008-03-17

    Logged In: YES
    user_id=5735
    Originator: NO

    the cast does not help (as expected - the same expression appears in eval.d without a cast).

    on amd64 we use a different memory model (TYPECODES vs HEAPCODES on i386) which causes a different set of issues.
    I find quite confusing:
    lightning.c: In function 'jit_compile_':
    lightning.c:1300: warning: implicit declaration of function 'LEAQmr'
    lightning.c:1312: warning: implicit declaration of function 'jit_muli_ul'
    lightning.c:1346: warning: implicit declaration of function '_s32P'
    lightning.c:1760: warning: implicit declaration of function 'jitc_getsize_framer'
    lightning.c:2010: warning: implicit declaration of function 'jit_nei_l'

     
  • Sam Steingold
    Sam Steingold
    2008-03-17

    Logged In: YES
    user_id=5735
    Originator: NO

    jitc_finish_framer is defined incorrectly: its definition is HEAPCODES-specific and does not work for TYPECODES. it has to be redefined to use framebottomword instead of makebottomword.
    same for cons_bias and tfl - they have no place in lightning.c

     
  • Logged In: YES
    user_id=1993164
    Originator: YES

    Typecodes and Heapcodes:

    I failed to mention it, but right now only Heapcodes is supported.

    I chose not to support both because that would require a lot of code duplication(it's assembly). That wouldn't be a good idea for an initial release. It would make everything harder.
    I chose Heapcodes because it's supported on every platform, it's as fast(said in imp notes), it keeps the code simple, and because I cannot use Typecodes(I have a 32 bit system).

    Of course, when jitc has been tested and is functionnal with Heapcodes, we can support typecodes.

    Ciao

     
  • Sam Steingold
    Sam Steingold
    2008-03-18

    Logged In: YES
    user_id=5735
    Originator: NO

    USE_JITC requires HEAPCODES -- OK, I am now on HEAPCODES.
    still on 64-bits I get plenty of "cast from pointer to integer of different size".
    if you need access to a 64-bit system: http://www.testdrive.hp.com/current.shtml
    this looks bad:
    lightning.c:1714:24: error: macro "jit_qop_" requires 5 arguments, but only 4 given
    lightning.c:1714: error: 'jit_qop_' undeclared (first use in this function)
    lightning.c:1714: error: (Each undeclared identifier is reported only once
    lightning.c:1714: error: for each function it appears in.)

    also, how about re-using S_operand_ignore et al from eval.d?

     
  • Logged In: YES
    user_id=1993164
    Originator: YES

    Thanks a lot.
    I subscribed right away.
    Unfortunately, I cannot compile clisp on those servers. A lot of important programs are missing(aclocal for one).
    Do you know any other service like this one?

     
  • Sam Steingold
    Sam Steingold
    2008-03-19

    Logged In: YES
    user_id=5735
    Originator: NO

    jitc_get_fieldr causes "warning: cast from pointer to integer of different size" on amd64 with SAFETY=3
    also, how about re-using S_operand_ignore et al from eval.d?

    what I meant by the previous comment is that you do not need aclocal et al - just pass --disable-maintainer-mode to the top-level configure script.

    I now get to
    ...
    ;; Loading file /homedata/ssteingold/src/clisp/current/src/defs2.lisp ...
    Program received signal SIGSEGV, Segmentation fault.
    0x00000000008f0db3 in ?? ()
    (gdb) where
    #0 0x00000000008f0db3 in ?? ()
    #1 0x00000000008f10c5 in ?? ()
    #2 0x000000099a35e6d9 in ?? ()
    #3 0x00000000008ec7b0 in ?? ()
    #4 0x000000099a35e6d9 in ?? ()
    #5 0x00000000008f2f49 in ?? ()
    #6 0x00000002008c8421 in ?? ()
    #7 0x000000099a2f19b3 in ?? ()
    #8 0xffffff6f008c84a1 in ?? ()
    #9 0x00000000008f2f4e in ?? ()
    #10 0x00000000008f10c5 in ?? ()
    #11 0x000000099a35e6f0 in ?? ()
    #12 0x00007fff18a96a40 in ?? ()
    #13 0x0000000000411390 in allocate_fpointer (foreign=0x7fff18a96a40)
    at ../src/spvw_typealloc.d:456
    #14 0x00007fff18ae1880 in ?? ()
    #15 0x0000000000000000 in ?? ()
    (gdb)

     
  • Sam Steingold
    Sam Steingold
    2008-03-19

    Logged In: YES
    user_id=5735
    Originator: NO

    the aforementioned crash is in the first call to bc_func in jitc_run

     
  • Sam Steingold
    Sam Steingold
    2008-03-19

    Logged In: YES
    user_id=5735
    Originator: NO

    what is TheSbvector(as_object(0))?
    as_object(0)==nullobj
    why are you casting it to TheSbvector?
    this is confusing...

     
  • Logged In: YES
    user_id=1993164
    Originator: YES

    on it

     
  • Logged In: YES
    user_id=1993164
    Originator: YES

    > TheSbvector(as_object(0))
    This is used when calling interpret_bytecode_ to add the 'bias' to the address of codevec

     
  • Sam Steingold
    Sam Steingold
    2008-03-19

    Logged In: YES
    user_id=5735
    Originator: NO

    >> TheSbvector(as_object(0))
    >This is used when calling interpret_bytecode_ to add the 'bias' to the address of codevec
    where? I don't understand, sorry.

     
  • Logged In: YES
    user_id=1993164
    Originator: YES

    Well, one example is in '#define jitc_funcallc' on line 711.
    I need to call interpret_bytecode_, but it requires TheSbvector(vec) as second argument, not just vec. And I can't just use the predefined macro because vec is not available at compile time.

    As said in lispbibl.c:3385, all 'The##Type' does in HEAPCODES is substract a 'bias' to the argument.
    So I do that at runtime.
    Example(lightning.c:736):
    // JIT_R0 contains address of a closure
    // Add (the offset of field 'clos_codevec' in Cclosure minus the bias for TheCclosure) to JIT_R0, load the content at this address in JIT_R1
    jit_ldxi_p(JIT_R1, JIT_R0, TheCclosure(as_object(jit_ptr_field(Cclosure, clos_codevec))));
    // JIT_R1 contains the address of the closure's codevec
    // Add it to TheSbvector(as_object(0)), which is equal to -varobject_bias
    jit_addi_p(JIT_R1,JIT_R1,TheSbvector(as_object(0)));

    Why not just use '-varobject_bias'? I thought using the macro was better because '-varobject_bias' is obscure(it's never used, and people might not know that it's just the def of TheSbvector).

     
  • Sam Steingold
    Sam Steingold
    2008-03-20

    Logged In: YES
    user_id=5735
    Originator: NO

    why are you calling interpret_bytecode_? (that was a question I wanted to ask for a long time!)
    I think you want to call its jitc counterpart interpret_bytecode defined around eval.d:318 (should be turned into a function for your purposes, I can do that for you)
    so you are saying that TheSbvector(as_object(0)) is just a trick to extract byteptr_in from codeptr at run time? that should be fixed by the aforementioned function, right?

     
  • Sam Steingold
    Sam Steingold
    2008-05-25

    Logged In: YES
    user_id=5735
    Originator: NO

    please open a new patch item for further patches.
    this is already long enough...

     
  • Sam Steingold
    Sam Steingold
    2008-05-25

    • assigned_to: haible --> sds
    • status: open --> closed-accepted
     
<< < 1 2 (Page 2 of 2)