Menu
▾
▴
[ clisp-Patches-1880809 ] JIT via lightning
|
From: SourceForge.net <no...@so...> - 2008-02-13 23:48:45
|
Patches item #1880809, was opened at 2008-01-27 13:25 Message generated for change (Comment added) made by sds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=301355&aid=1880809&group_id=1355 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Yann Nicolas Dauphin (y-d) Assigned to: Bruno Haible (haible) Summary: JIT via lightning Initial Comment: Still incomplete JIT compiler for CLISP. The jit compilation occurs at every function call(no reuse), so it runs slower for now. Lightning is used because: - It's portable. Can reportedly be ported under a day. - It's simple. If the GNU lightning project was to stop, it wouldn't be a problem. Maintaining it is easy. Not so for LLVM. - It's lean and mean. Code generation is straightforward, so very fast. Tested on x86 with: - Linux - Mac OS X: Must 'export DYLD_BIND_AT_LAUNCH=' for now because the dynamic linker messes with jit function calls The patch is too big for SF so: http://www.step.polymtl.ca/~polyrad/polyrad/jit-clisp-2.43.patch ---------------------------------------------------------------------- >Comment By: Sam Steingold (sds) Date: 2008-02-13 18:48 Message: Logged In: YES user_id=5735 Originator: NO OK, so the first stab at keeping JITC code with the closure is in the CVS head. Sorry about taking so long... Alas, I cannot really test it because lightning does not compile for me. what I observe now is when I compile lightning.c I see lots of errors and warnings, e.g., lightning.c: In function 'jit_compile_': lightning.c:1304: warning: implicit declaration of function 'LEAQmr' lightning.c:1304: warning: value computed is not used lightning.c:1310: warning: value computed is not used lightning.c:1310: warning: value computed is not used lightning.c:1310: warning: value computed is not used lightning.c:1310: warning: value computed is not used lightning.c:1310: warning: value computed is not used lightning.c:1310: warning: value computed is not used lightning.c:1310: warning: value computed is not used lightning.c:1310: warning: value computed is not used lightning.c:1311: warning: value computed is not used lightning.c:1311: warning: value computed is not used lightning.c:1311: warning: value computed is not used lightning.c:1316: warning: implicit declaration of function 'jit_muli_ul' lightning.c:1316: warning: value computed is not used lightning.c:1324: error: 'cod_labels' undeclared (first use in this function) lightning.c:1324: error: (Each undeclared identifier is reported only once lightning.c:1324: error: for each function it appears in.) lightning.c:1324: error: expected ';' before ':' token lightning.c:1338: warning: value computed is not used lightning.c:1338: warning: value computed is not used lightning.c:1338: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1342: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: cast from pointer to integer of different size lightning.c:1350: warning: implicit declaration of function '_s32P' lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: cast from pointer to integer of different size lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: value computed is not used lightning.c:1350: warning: cast from pointer to integer of different size lightning.c:1350: warning: value computed is not used lightning.c:1357: warning: value computed is not used also, you assume that object is a pointer type which it is not when SAFETY is 3 and OBJECT_STRUCT is defined. so, the ball is in your court is now: removing warnings supporting OBJECT_STRUCT testing the GC hooks ---------------------------------------------------------------------- Comment By: Sam Steingold (sds) Date: 2008-02-04 19:03 Message: Logged In: YES user_id=5735 Originator: NO oops - forgot that fpointers are usually immediate. we can have a chain of jit functions and mark them at gc_mark and then sweep them. this is a variation on the finaliser idea. ---------------------------------------------------------------------- Comment By: Sam Steingold (sds) Date: 2008-02-04 00:23 Message: Logged In: YES user_id=5735 Originator: NO then we are in trouble wrt re-using the compiled form. we need to store the compiled closure (codeBuffer + bcIndex &c) in the Cclosure as a Fpointer to a C structure. how do we release it when the Cclosure object is garbage collected? (a) the most direct way is to use finalize, but that would make GC very expensive. (b) we can add a 3rd phase between mark and copy - scan memory for dead cclosures and free the jit code (also expensive) (c) we can allocate Fpointers in a separate memory area and mark those who should be free()d with a special bit, i.e., special case finalizing Fpointers with free(), then the additional phase as in (b) has to be done only on that separate fpointer memory area. note that this feature may also be useful in other areas, e.g., FFI. Bruno, what do you think? ---------------------------------------------------------------------- Comment By: Yann Nicolas Dauphin (y-d) Date: 2008-02-03 22:44 Message: Logged In: YES user_id=1993164 Originator: YES codeBuffer and bcIndex have to be kept in constant memory because the operands of jump instructions in the JITed code are absolute positions. As a side-note, I do plan to remove the need to permanently store bcIndex. But that's not high priority. Thanks ---------------------------------------------------------------------- Comment By: Sam Steingold (sds) Date: 2008-02-03 21:00 Message: Logged In: YES user_id=5735 Originator: NO thanks - I committed your patch. do codeBuffer & bcIndex have to be in constant memory (as in malloc) as opposed to lisp heap (movable by GC)? ---------------------------------------------------------------------- Comment By: Yann Nicolas Dauphin (y-d) Date: 2008-02-03 11:22 Message: Logged In: YES user_id=1993164 Originator: YES Lightning vs Libjit Technical: Libjit is well written and looks lovely, but Lightning too. The main differences are that Libjit is more high-level (Auto register allocation, memory allocation, ...) and that it optimizes the code a bit. This is why I first considered using libjit. But [1] shows that an optimizing(as in AOT compilers) compiler working at a low-level is not dramatically faster that a template based JIT compiler(like lightning). For significant performance improvements, using trace based JIT compilation with Trace SSA code analysis as described in [2] is the best alternative. This approach requires control over register allocation and optimizes the code. This makes much of libjit features redundant. Social: Shortly: Lightning is more active. GNU Smalltalk, MzScheme, OcamlJIT, Rubinius(in FFI module) use it. MzScheme achieves good performance with it[3]. No active project uses libjit. Which is a shame, since it's a good lib. The current version of libjit only supports 1 architecture while Lightning supports 4. And it seems that number will keep on growing. It's risky to use Libjit. [1] http://research.sun.com/techrep/2000/abstract-87.html Also note the speed up over a simple switch based interpreter for number crunshing. [2] http://www.usenix.org/events/vee06/full_papers/p144-gal.pdf This has the merit of optimizing 0 (CONST&PUSH 0) ; 2 1 (CONST 1) ; (4) 2 (CONS) So that 0 and 1 don't push anything on the stack and get directly passed in registers (if possible). [3] http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=all ---------------------------------------------------------------------- Comment By: Yann Nicolas Dauphin (y-d) Date: 2008-01-30 01:53 Message: Logged In: YES user_id=1993164 Originator: YES jit-clisp-cvs-20080131.patch.gz: This new patch includes the clean ups from Reini and will work against a clean checkout from CVS. Please install the latest version of GNU lightning(at least 1.2b): http://git.savannah.gnu.org/gitweb/?p=lightning.git;a=snapshot;h=master It includes: - Modifying make file to add jit.d with jit.h as one of it's depedencies - Assuming GNU Lightning is installed - lispbibl sets USE_JIT if I80386 is set and registers are saved if !USE_JIT - Replacing all // comments with /**/ and tabs with spaces - Removed vestigial c source and jit_begin/end - Random cleanups News tests have been added to 'make check'. CLISP runs out of memory on one of the last ones, 'ctak'. This is because using longjmp can cause the memory of codeBuffer to leak. This is of course resolved by reusing the jit function. libjit vs lightning coming up. File Added: jit-clisp-cvs-20080131.patch.gz ---------------------------------------------------------------------- Comment By: Sam Steingold (sds) Date: 2008-01-28 18:25 Message: Logged In: YES user_id=5735 Originator: NO clean-up? http://permalink.gmane.org/gmane.lisp.clisp.devel/17523 http://rurban.xarch.at/cygr/clisp/jit-extra.patch.gz ---------------------------------------------------------------------- Comment By: Sam Steingold (sds) Date: 2008-01-28 18:22 Message: Logged In: YES user_id=5735 Originator: NO lightning vs libjit http://permalink.gmane.org/gmane.lisp.clisp.general/12114 ---------------------------------------------------------------------- Comment By: Sam Steingold (sds) Date: 2008-01-28 17:00 Message: Logged In: YES user_id=5735 Originator: NO it would be nice if you could elaborate on advantages of lightning over libjit http://freshmeat.net/projects/libjit/ ---------------------------------------------------------------------- Comment By: Sam Steingold (sds) Date: 2008-01-27 21:14 Message: Logged In: YES user_id=5735 Originator: NO http://article.gmane.org/gmane.lisp.clisp.devel:17497 http://article.gmane.org/gmane.lisp.clisp.devel:17509 http://article.gmane.org/gmane.lisp.clisp.devel:17515 ---------------------------------------------------------------------- Comment By: Sam Steingold (sds) Date: 2008-01-27 19:52 Message: Logged In: YES user_id=5735 Originator: NO the patch is too big because it includes some binary files (.DS_Store) and the full sources for lightning. please assume that lightning is installed on the developer's machine. please include only the eval.d and jit.h, NOT lightning sources please do NOT use "//" comments ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=301355&aid=1880809&group_id=1355 |