You can subscribe to this list here.
2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(2) |
Oct
|
Nov
|
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2003 |
Jan
|
Feb
|
Mar
|
Apr
(8) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(8) |
Nov
(1) |
Dec
|
2004 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2006 |
Jan
(2) |
Feb
(24) |
Mar
(263) |
Apr
(115) |
May
(88) |
Jun
(215) |
Jul
(408) |
Aug
(247) |
Sep
(334) |
Oct
(174) |
Nov
(131) |
Dec
(112) |
2007 |
Jan
(121) |
Feb
(62) |
Mar
(122) |
Apr
(286) |
May
(209) |
Jun
(241) |
Jul
(203) |
Aug
(156) |
Sep
(236) |
Oct
(145) |
Nov
(375) |
Dec
(165) |
2008 |
Jan
(11) |
Feb
(18) |
Mar
(58) |
Apr
(83) |
May
(26) |
Jun
(4) |
Jul
(7) |
Aug
(102) |
Sep
(30) |
Oct
(65) |
Nov
(4) |
Dec
(6) |
2009 |
Jan
(103) |
Feb
(107) |
Mar
(150) |
Apr
(402) |
May
(141) |
Jun
(194) |
Jul
(30) |
Aug
(4) |
Sep
(14) |
Oct
(59) |
Nov
(34) |
Dec
(10) |
2010 |
Jan
(24) |
Feb
(12) |
Mar
(14) |
Apr
(1) |
May
(9) |
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
(15) |
Nov
|
Dec
|
2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(2) |
Oct
(17) |
Nov
(1) |
Dec
|
From: john s. <sk...@us...> - 2011-11-01 01:37:41
|
On 31/10/2011, at 2:32 PM, john skaller wrote: > > On 31/10/2011, at 3:03 PM, john skaller wrote: > >> >> On 31/10/2011, at 1:44 PM, John Tromp wrote: >> >>> hi John, >>> >>>> Hmm .. what am I doing wrong here? This is a simple C program... >>> >>> May be a problem with your Judy installation. >> >> Yes. That seems likely (since we use our own build script written in Python). >> > > Yep .. its bugged: this code: > > macros = [dstname.upper()] > if types.voidp.size == 8: > macros.append('JU_64BIT') > else: > macros.append('JU_32BIT') > > sets macros when building Judy1/JudyL but isn't invoked when building JudySL and JudyHS. > > So JudySL and HS are built with default which I guess is 32 bits. Fixed! -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-31 21:34:34
|
On 01/11/2011, at 4:30 AM, James Dennett wrote: > > You could build/install your own GCC 4.6.x on Mac OS X, but it won't > be an Apple-supported build. Actually I tried to install a pre-built binary from a HPC crowd, the compiler worked but the library didn't. -- john skaller sk...@us... |
From: James D. <jam...@gm...> - 2011-10-31 18:30:16
|
On Sun, Oct 30, 2011 at 9:03 PM, john skaller <sk...@us...> wrote: > > BTW: > > ~/felix>gcc --version > i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3) > > > I'd sure love to get C++1x for OSX... need LLVM compiler for that though? AFAIK Apple doesn't play to ship a recent GCC version, so yes, you'd need Clang (the C/C++/... compiler built on the LLVM framework) in order to get more C++11 support, and while Clang's making good progress there it's still somewhat behind GCC in my experience. You could build/install your own GCC 4.6.x on Mac OS X, but it won't be an Apple-supported build. -- James |
From: john s. <sk...@us...> - 2011-10-31 15:37:21
|
On 01/11/2011, at 1:13 AM, Doug Baskins wrote: > John: > > My latest version of Judy checks to see if it is being compiled in a > 64 bit environment and sets the JU_64BIT macro. Sorry I did not > do that in the released version. I'm not sure that's a good idea. How are you going to get this right? Note that just because the host machine is 64 bit doesn't mean the target is also 64 bit: for example compiling for iPhone (ARM processor!) on modern Mac (Intel, 64 bit). I wish you'd get rid of the table gen stuff. There are only two sizes in reality, 32 and 64 bit. Why not just commit both to the repository, already generated? The problem is the generator must NOT use the environment size as that of the generated tables. The generator runs on the host machine, the tables may be used on Judy running on a completely different machine. The same applies to endian-ness. > I wish the compilers had a portable > way of testing for a 32/64 bit compile in the code. What? Expect a Standards committee to do something useful? > Good catch John Tromp. I would of never found that one. I suspected it, because I knew Erick changed the Felix build process for Judy from my original one which used modified files, to one which used the authentic originals from sourceforge -- thereby allowing any upgrades to be imported. Also the corruption symptoms suggested it, since I know Judy is reliable if built properly. Thanks everyone for your attention! It remains to get Erick to fix the build script (since I can't figure out how to do it). -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-31 04:33:53
|
On 31/10/2011, at 3:03 PM, john skaller wrote: > > On 31/10/2011, at 1:44 PM, John Tromp wrote: > >> hi John, >> >>> Hmm .. what am I doing wrong here? This is a simple C program... >> >> May be a problem with your Judy installation. > > Yes. That seems likely (since we use our own build script written in Python). > Yep .. its bugged: this code: macros = [dstname.upper()] if types.voidp.size == 8: macros.append('JU_64BIT') else: macros.append('JU_32BIT') sets macros when building Judy1/JudyL but isn't invoked when building JudySL and JudyHS. So JudySL and HS are built with default which I guess is 32 bits. -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-31 04:04:37
|
On 31/10/2011, at 1:44 PM, John Tromp wrote: > hi John, > >> Hmm .. what am I doing wrong here? This is a simple C program... > > May be a problem with your Judy installation. Yes. That seems likely (since we use our own build script written in Python). > I runs fine on my brand new MacBook Air with > gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) > and judy-1.0.5 in 64 bit mode. > Last 3 lines of output: > found XX 0997 abcdefg -> 997 > found XX 0998 abcdefg -> 998 > found XX 0999 abcdefg -> 999 I have played around with it, and using memset to clear buffer, changing the sprintf to use %016d, %d, or various things makes a huge mess of the keys. Indeed if all the strings are the same length it seems to work. I even got this: found XX 1 -> 0 found XX 2 -> 81 found XX 3 -> 780 found XX 4 -> 117 found XX 5 -> 466 found XX 6 -> 419 found XX 7 -> 444 found XX 8 -> 785 found XX 9 -> 346 (exit here) when memset buffer to all 0 (and no error ..). BTW: ~/felix>gcc --version i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3) I'd sure love to get C++1x for OSX... need LLVM compiler for that though? -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-31 02:06:56
|
On 30/10/2011, at 9:40 AM, john skaller wrote: > Hi, I'm having some problems with JudySL .. not sure what's going on. Hmm .. what am I doing wrong here? This is a simple C program... #include "Judy.h" #include <stdio.h> int main() { printf("Judy test\n"); JError_t je; void *ja = NULL; Word_t *pv; char buffer[1000]; int i; for (i=0; i<1000; ++i) { sprintf(buffer,"XX %04d abcdefg",i); pv = (Word_t*)JudySLIns(&ja, buffer, &je); *pv = i; printf("Done %s -> %d\n",buffer, i); } buffer[0]=0; pv = (Word_t*)JudySLFirst(ja,buffer,&je); while(pv) { i = (int)*pv; printf("found %s -> %d\n",buffer, i); pv = (Word_t*)JudySLNext(ja,buffer,&je); } } Output: Done XX 0998 abcdefg -> 998 Done XX 0999 abcdefg -> 999 Segmentation fault -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-30 05:31:17
|
On 30/10/2011, at 9:40 AM, john skaller wrote: > Hi, I'm having some problems with JudySL .. not sure what's going on. Well ... isn't this impossible? // program open Judy; x := JSLArray(); var je: JError_t; var pv : &word; var buffer: &char = C_hack::cast[&char] (C_hack::malloc(1000)); for var i in 0 upto 1000 do var j = Cstdlib::rand(); println$ "Rand = " + str j; var s = j.str; println$ "Rand = " + s; var cp = s.cstr; println$ "C string="+ str cp; strncpy(buffer,cp,800); print$ str i + " --> " + str j + " = " + buffer; JudySLIns(x,buffer,&je,&pv); assert not (isNULL pv); print$ " Done " + str i; if C_hack::cast[long] (*pv) != 0l do println "DUPLICATE"; done *pv = C_hack::cast[word] i; println$ " Assigned " + str i; done // tail of output 812 --> 1608776735 = 1608776735 Done 812 Assigned 812 Rand = 1891469415 Rand = 1891469415 C string=1891469415 813 --> 1891469415 = 1891469415 Done 813 Assigned 813 Rand = 726031364 Rand = 726031364 C string=726031364 814 --> 726031364 = 726031364 Done 814 Assigned 814 Rand = 407052494 Rand = 407052494 C string=407052494 815 --> 407052494 = 407052494 Done 815 Assigned 815 Rand = 1595850963 Rand = 1595850963 C string=1595850963 816 --> 1595850963 = 1595850963 Done 816 Assigned 816 Rand = 1543867758 Rand = 1543867758 C string=1543867758 817 --> 1543867758 = 1543867758 Done 817 Assigned 817 Rand = 1887985652 Rand = 1887985652 C string=1887985652 Segmentation fault /// So .. strncpy has valid data but it is segfaulting. The only way this could possibly happen is if the pointer variable buffer got corrupted. -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-29 22:40:57
|
Hi, I'm having some problems with JudySL .. not sure what's going on. First a test program written in Felix: ///// open Judy; x := JSLArray(); var je: JError_t; var pv : &word; var buffer: &char = C_hack::cast[&char] (C_hack::malloc(1000)); for var i in 0 upto 100 do var j = Cstdlib::rand(); strcpy(buffer, j.str.cstr); println$ str i + " --> " + str j + " = " + buffer; JudySLIns(x,buffer,&je,&pv); if C_hack::cast[long] (*pv) != 0l do println "DUPLICATE"; done *pv = C_hack::cast[word] i; done println "Listing ds=d->s"; *buffer=char 0; JudySLFirst(x,buffer,&je,&pv); var k= 0; whilst not isNULL pv do j := C_hack::cast[int] (*pv); println$ "Recorded copy " + k.str + ": " + string buffer + " => " + str j; ++k; JudySLNext(x,buffer,&je,&pv); done ///// I hope this is readable even to a C programmer! What happens: as written this stores only 45 entries. When I change the count to 1000 it segfaults during the storage phase. OK so here is the Felix binding: proc JudySLIns: JSLArray * &char * &JError_t * &&word = """ if (::std::strlen($2) >= JUDY_SL_MAXLEN) throw "JudySLIns strlen>10000"; *(Word_t**)$4=(Word_t*)JudySLIns($1,(unsigned char*)$2,$3); """; There's one more thing to look at: Felix now allows JudySL arrays to contain Felix pointers as values. The GC uses the follow routine to scan a JudySL array for such values: void *JudySL_scanner(collector_t *collector, gc_shape_t *shape, void *pp, unsigned long dyncount, int reclimit) { void *p = *(void**)pp; //printf("Scanning judyL array %p->%p\n", pp, p); JError_t je; unsigned char *key = (unsigned char*)::std::malloc(10000); // HACK *key = 0; Word_t *pval = 0; pval = (Word_t*)JudySLFirst(p, key, &je); while(pval) { //printf("JudyL scanning p=%s\n",key); collector->register_pointer((void*)*pval,reclimit); pval = (Word_t*)JudySLNext(p, key, &je); } ::std::free(key); return 0; } The collector->register_pointer() function simply checks if the value is a Felix pointer, if it is, the pointer is "marked" by the GC as reachable and not to be collected. It is unlikely this code is invoked! I uncommented the diagostic and it isn't being printed but I include this code for completeness. The output of the Felix program with loop counter 1000 shows: 779 --> 1728536152 = 1728536152 780 --> 348330048 = 348330048 DUPLICATE 781 --> 342695014 = 342695014 .... 816 --> 1595850963 = 1595850963 817 --> 1543867758 = 1543867758 818 --> 1887985652 = 1887985652 Segmentation fault A duplicate is not likely (pseudo random number generators do not duplicate). The program "flx_cp" is a sane replacement for unix "cp". This program stores the source and destination of a wildcarded filecopy command in two judy SL arrays (src->dst and dst->src) so it can check for overlap before copying. The index is a null terminated string as required by JudySL, the value is a pointer to a C++ string. In that program the index values are scrambled exactly as if the keys are overwritting. It is possible that Erick has screwed up the Judy build. Judy used to be built by the standard Felix build code, but because of code generation in Judy Makefile and huge ugly macros to reuse code, I reorganised the code to eliminate the code generation (by pre-generating the required tables) and copied some files about to eliminate the need for complex include directives. The generated code is actually a special indexing table for either 32 or 64 bit Judy. In fact there's no reason to generate this code since there are exactly two options.. and there even no reason not to build Judy for BOTH 32 and 64 bit (it can work with BOTH sizes on both 32 and 64 bit machines with a bit of hacking about to decouple the binary size used for the data structure from the C types used to manipulate them). So it is possible the build is screwed up. However I think Judy1 and JudyL work since they're used extensively by the GC (although .. if they don't it would explain why some programs crash in the GC .. such as the webserver .. but it seems more likely to be a bug in the GC). Bottom line: it looks like a bug in JudySL but Judy has proven to be bug free if built correctly, just hard to use properly. So it is more likely a bug in my code .. I just can't see it. On 30/10/2011, at 5:33 AM, Jason E. Aten wrote: > did you get the JudySLIns figured out? > > Here's how I wrap the JLSI macro... which is just FYI: as you can see by my code I do not use the macros. The confuse values with lvalues. The function calls make the distinction explicit. Unfortunately it is the macros that are documented so I have to backtrack the documentation from the function calls (an extra cognitive layer) and then also consider the Felix binding .. as well as checking proper use in my application. > > void insertkey(const char* key, V value) > { > > PWord_t PValue = 0; > uint8_t Index[BUFSIZ]; // string to sort. > strncpy((char*)Index,key,BUFSIZ); > > JSLI( > PValue, > (_judyS), > Index); // store string into array > if (0==PValue) { > assert(0); > } > if ((long)PValue == -1) { > assert(0); > } > > // was here to diagnose duplicates t_typ's > // but we never want to overwrite old string without free-ing > them first, > // so leave this insert in. > if (*PValue != 0) { > assert(0); > } > > ++_size; > (*((V*)PValue)) = value; > } > > > > On Sat, Oct 29, 2011 at 9:21 AM, john skaller > <sk...@us...> wrote: >> flx_cp isn't working .. it seems I;m not using JudySLIns correctly: >> >> tools/flx_cp web "(.*)" 'xxx\1' --test >> >> //////// >> >> Consider file download.html >> Listing ds=d->s >> Listing sd=s->d >> subs 1 -> download.html >> [pre-Ins] Src = web/download.html Dst= xxxdownload.html >> [post-Ins sd] Src = web/download.html Dst= xxxdownload.html >> [post-Ins ds] Src = web/download.html Dst= xxxdownload.html >> Copy web/download.html -> xxxdownload.html >> Consider file Easy_to_Deploy.fdoc >> Listing ds=d->s >> Recorded copy xxxdownload.html <- web/download.html >> Listing sd=s->d >> Recorded copy web/download.html -> xxxdownload.html >> subs 1 -> Easy_to_Deploy.fdoc >> [pre-Ins] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc >> [post-Ins sd] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc >> [post-Ins ds] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc >> Copy web/Easy_to_Deploy.fdoc -> xxxEasy_to_Deploy.fdoc >> Consider file Easy_to_Read_and_Write.fdoc >> Listing ds=d->s >> Recorded copy xxxEasy_to_Deploy.fdoc <- web/Easy_to_Deploy.fdoc >> Recorded copy xxxdasy_oad.html <- web/download.html >> Listing sd=s->d >> Recorded copy web/asy__to_htmloy.fdoc -> xxxEasy_to_Deploy.fdoc >> Recorded copy web/asy_loadhtmll -> xxxdownload.html >> subs 1 -> Easy_to_Read_and_Write.fdoc >> [pre-Ins] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc >> [post-Ins sd] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc >> [post-Ins ds] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc >> Copy web/Easy_to_Read_and_Write.fdoc -> xxxEasy_to_Read_and_Write.fdoc >> Consider file Easy_to_Read_and_Write_01.cpp >> Listing ds=d->s >> Recorded copy xxxEasy_to_Dead_y.fdoc <- web/Easy_to_Deploy.fdoc >> Recorded copy xxxEasy_to_Read_and_Write.fdoc <- web/Easy_to_Read_and_Write.fdoc >> Recorded copy xxxdasy_oad.html <- web/download.html >> Listing sd=s->d >> Recorded copy web/asy__to_html_andWritte.fdoc -> xxxEasy_to_Read_and_Write.fdoc >> Segmentation fault >> /////////// >> >> clearly some overwiting happening .... >> >> >> -- >> john skaller >> sk...@us... >> >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups "Felix Language" group. >> To post to this group, send email to fel...@go.... >> To unsubscribe from this group, send email to fel...@go.... >> For more options, visit this group at http://groups.google.com/group/felix-language?hl=en. >> >> -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-29 14:22:16
|
flx_cp isn't working .. it seems I;m not using JudySLIns correctly: tools/flx_cp web "(.*)" 'xxx\1' --test //////// Consider file download.html Listing ds=d->s Listing sd=s->d subs 1 -> download.html [pre-Ins] Src = web/download.html Dst= xxxdownload.html [post-Ins sd] Src = web/download.html Dst= xxxdownload.html [post-Ins ds] Src = web/download.html Dst= xxxdownload.html Copy web/download.html -> xxxdownload.html Consider file Easy_to_Deploy.fdoc Listing ds=d->s Recorded copy xxxdownload.html <- web/download.html Listing sd=s->d Recorded copy web/download.html -> xxxdownload.html subs 1 -> Easy_to_Deploy.fdoc [pre-Ins] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc [post-Ins sd] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc [post-Ins ds] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc Copy web/Easy_to_Deploy.fdoc -> xxxEasy_to_Deploy.fdoc Consider file Easy_to_Read_and_Write.fdoc Listing ds=d->s Recorded copy xxxEasy_to_Deploy.fdoc <- web/Easy_to_Deploy.fdoc Recorded copy xxxdasy_oad.html <- web/download.html Listing sd=s->d Recorded copy web/asy__to_htmloy.fdoc -> xxxEasy_to_Deploy.fdoc Recorded copy web/asy_loadhtmll -> xxxdownload.html subs 1 -> Easy_to_Read_and_Write.fdoc [pre-Ins] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc [post-Ins sd] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc [post-Ins ds] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc Copy web/Easy_to_Read_and_Write.fdoc -> xxxEasy_to_Read_and_Write.fdoc Consider file Easy_to_Read_and_Write_01.cpp Listing ds=d->s Recorded copy xxxEasy_to_Dead_y.fdoc <- web/Easy_to_Deploy.fdoc Recorded copy xxxEasy_to_Read_and_Write.fdoc <- web/Easy_to_Read_and_Write.fdoc Recorded copy xxxdasy_oad.html <- web/download.html Listing sd=s->d Recorded copy web/asy__to_html_andWritte.fdoc -> xxxEasy_to_Read_and_Write.fdoc Segmentation fault /////////// clearly some overwiting happening .... -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-29 14:22:16
|
flx_cp isn't working .. it seems I;m not using JudySLIns correctly: tools/flx_cp web "(.*)" 'xxx\1' --test //////// Consider file download.html Listing ds=d->s Listing sd=s->d subs 1 -> download.html [pre-Ins] Src = web/download.html Dst= xxxdownload.html [post-Ins sd] Src = web/download.html Dst= xxxdownload.html [post-Ins ds] Src = web/download.html Dst= xxxdownload.html Copy web/download.html -> xxxdownload.html Consider file Easy_to_Deploy.fdoc Listing ds=d->s Recorded copy xxxdownload.html <- web/download.html Listing sd=s->d Recorded copy web/download.html -> xxxdownload.html subs 1 -> Easy_to_Deploy.fdoc [pre-Ins] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc [post-Ins sd] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc [post-Ins ds] Src = web/Easy_to_Deploy.fdoc Dst= xxxEasy_to_Deploy.fdoc Copy web/Easy_to_Deploy.fdoc -> xxxEasy_to_Deploy.fdoc Consider file Easy_to_Read_and_Write.fdoc Listing ds=d->s Recorded copy xxxEasy_to_Deploy.fdoc <- web/Easy_to_Deploy.fdoc Recorded copy xxxdasy_oad.html <- web/download.html Listing sd=s->d Recorded copy web/asy__to_htmloy.fdoc -> xxxEasy_to_Deploy.fdoc Recorded copy web/asy_loadhtmll -> xxxdownload.html subs 1 -> Easy_to_Read_and_Write.fdoc [pre-Ins] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc [post-Ins sd] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc [post-Ins ds] Src = web/Easy_to_Read_and_Write.fdoc Dst= xxxEasy_to_Read_and_Write.fdoc Copy web/Easy_to_Read_and_Write.fdoc -> xxxEasy_to_Read_and_Write.fdoc Consider file Easy_to_Read_and_Write_01.cpp Listing ds=d->s Recorded copy xxxEasy_to_Dead_y.fdoc <- web/Easy_to_Deploy.fdoc Recorded copy xxxEasy_to_Read_and_Write.fdoc <- web/Easy_to_Read_and_Write.fdoc Recorded copy xxxdasy_oad.html <- web/download.html Listing sd=s->d Recorded copy web/asy__to_html_andWritte.fdoc -> xxxEasy_to_Read_and_Write.fdoc Segmentation fault /////////// clearly some overwiting happening .... -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-29 00:50:28
|
On 28/09/2011, at 8:28 PM, Rhythmic Fistman wrote: > I just tracked down a CPU burning bug to felix's waitable_bool class: > > // a waitable boolean. > class PTHREAD_EXTERN waitable_bool { > flx::pthread::flx_mutex_t cv_lock; // to work with the condition var > flx::pthread::flx_condv_t finished_cond; > bool finished; // might seem redundant, but that's how CVs work. > public: > waitable_bool(); > > void wait_until_true(); > void signal_true(); > }; > > Felix uses it to politely ask threads to quit, which works fine. > > What I'd forgotten (or never knew) is that this is a one shot object - > once set to true, it's always true and all further calls to > wait_until_true correctly return immediately, hence my CPU burn. > > I'd like to modify it by adding a set_to_false method (not signal, > because there's no wait_until_false!). > I'm writing to this list to see if that raises any alarm bells (what > if there are multiple waiters? in felix there aren't, nor are there in > my use) and it would be nice to check such a change in, even though my > branch of the demux/pthread stuff is fossilised at svn commit #1695 > because after that the lightweight pthread wrapper pulled in a bunch > of garbage collection stuff. Well feel free to check any change in. But more importantly I think: we need to reorganise Felix so you can use the most recent version. What's the problem with the GC code being pulled in? A stub GC could be made (that throws an exception if any serious attempt is made to use the gc). More "seriously" we could look at making the GC optional. More "seriously" than that, we could look at a long term goal: to generate pure C/C++ code (requiring no flx_run). Many Felix programs don't actually use any p or f threading and so don't actually need flx_run. Decoupling stuff isn't easy. Code in flx_run will call the collector entry points in various places because if the collector IS being used it's necessary. WRT my other mail on work schedule: I have big plans for making Felix more dynamic (this was always intended!). Recently a way to incorporate custom data structures into the gc was added, primarily this was to support Judy arrays. However it could also be used to support C++ Vectors containing Felix pointers. In fact the hard bit is distinguising between such vectors requiring scanning by the gc and those that don't (eg vectors of ints). I mention this because this is the first step towards a more dynamic type system. The gc previously used an array of offsets to find pointers in an object, now it will take an arbitrary scanning function. The ability to create, copy, move, and delete dynamically defined types can also be added eventually. Oh, and backstepping a bit: custom event loops such as the kind RF is using need to be supported too. In fact it would be nice to be able to embed the whole Felix system as callback of a third party event loops (of course that makes using threads for async I/O a little tricky .. :) -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-28 13:47:06
|
On 28/10/2011, at 11:43 PM, john skaller wrote: > > Compare: > > http://localhost:1234/web/Easy_to_Read_and_Write.fdoc > > where it says " "variable length array y" (y in typewritter font) with > > http://felix-lang.org/Easy_to_Read_and_Write.fdoc > > where it says "variable length array {y}" > > The latter simply can't be the most recent version of Felix webserver. Ah .. I think I fixed it! The command to start the port 1116 server failed to specify a root. Since user felixweb is actually using MY home directory some stuff must have come from the wrong place. Upstart .. seems quite nice now I bothered to actually read the docs :) -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-28 12:44:51
|
Erick .. why is the web server still running? What I mean is: there is STILL the bug that {xxx} is not correctly typeset online. Compare: http://localhost:1234/web/Easy_to_Read_and_Write.fdoc where it says " "variable length array y" (y in typewritter font) with http://felix-lang.org/Easy_to_Read_and_Write.fdoc where it says "variable length array {y}" The latter simply can't be the most recent version of Felix webserver. Ps shows: 11908 ? Ss 0:00 /bin/sh -e -c privbind -u felixweb /usr/local/bin/webserver --port=80 --root=/usr/local/lib/felix/felix-latest/w 11910 ? Sl 3:02 /usr/local/bin/webserver --port=80 --root=/usr/local/lib/felix/felix-latest/web/ 11912 ? S 0:00 privbind -u felixweb /usr/local/bin/webserver --port=80 --root=/usr/local/lib/felix/felix-latest/web/ 17686 ? Ss 0:00 /bin/sh -e -c privbind -u felixweb /usr/local/bin/webserver --port=1116 /bin/sh 17688 ? Sl 1:05 /usr/local/bin/webserver --port=1116 17690 ? S 0:00 privbind -u felixweb /usr/local/bin/webserver --port=1116 huh? Well, all those commands are running /usr/local/bin/webserver ... Now, if I do this: skaller@felix:~$ webserver --root=/usr/local/lib/felix/felix-latest/web/ and look at http://felix-lang.org:1234/Easy_to_Read_and_Write.fdoc It all works right (with the "y" in typewritter font). I am baffled! OOoo .. and now the webserver on port 1116 has crashed :) Grrr :) -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-27 22:26:13
|
On 28/10/2011, at 3:42 AM, Erick Tryzelaar wrote: > That's great. That sounds like an awesome trip. How long was the sail? Six weeks all up. Well, doing resume now. Hey, Felix-lang.org is still running! Well .. now I need to sort out repository and build system ;( And fix the {bug} that {word} are still not formatted correctly in .fdoc files online even though the current software on my box does it. -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-27 16:13:02
|
On 27/10/2011, at 11:51 PM, Jeff Schultz wrote: > On Thu, Oct 27, 2011 at 10:10:17AM +0200, Rhythmic Fistman wrote: >>> FYI: I've just returned to Cairns from Papua New Guinea! >>> I may start some Felix work again soon. >> >> cool! I was starting to get worried > > Warm! Cairns: Fine. Min 21 Max 30 UV 13 > > Humidity 90% at 0500. Stinky .. PNG was hotter but one could jump in the sea and go snorkelling anytime. Best snorkelling (and diving) in the world. -- john skaller sk...@us... |
From: john s. <sk...@us...> - 2011-10-27 08:49:12
|
On 28/09/2011, at 8:28 PM, Rhythmic Fistman wrote: FYI: I've just returned to Cairns from Papua New Guinea! I may start some Felix work again soon. -- john skaller sk...@us... |
From: Rhythmic F. <rfi...@gm...> - 2011-10-27 08:10:24
|
On 27 October 2011 08:41, john skaller <sk...@us...> wrote: > > On 28/09/2011, at 8:28 PM, Rhythmic Fistman wrote: > > FYI: I've just returned to Cairns from Papua New Guinea! > I may start some Felix work again soon. cool! I was starting to get worried |
From: Rhythmic F. <rfi...@gm...> - 2011-09-28 10:45:38
|
I added a reset_and_wait_until_true method. Any dire predictions? RF On 28 September 2011 12:28, Rhythmic Fistman <rfi...@gm...> wrote: > I just tracked down a CPU burning bug to felix's waitable_bool class: > > // a waitable boolean. > class PTHREAD_EXTERN waitable_bool { > flx::pthread::flx_mutex_t cv_lock; // to work with the condition var > flx::pthread::flx_condv_t finished_cond; > bool finished; // might seem redundant, but that's how CVs work. > public: > waitable_bool(); > > void wait_until_true(); > void signal_true(); > }; > > Felix uses it to politely ask threads to quit, which works fine. > > What I'd forgotten (or never knew) is that this is a one shot object - > once set to true, it's always true and all further calls to > wait_until_true correctly return immediately, hence my CPU burn. > > I'd like to modify it by adding a set_to_false method (not signal, > because there's no wait_until_false!). > I'm writing to this list to see if that raises any alarm bells (what > if there are multiple waiters? in felix there aren't, nor are there in > my use) and it would be nice to check such a change in, even though my > branch of the demux/pthread stuff is fossilised at svn commit #1695 > because after that the lightweight pthread wrapper pulled in a bunch > of garbage collection stuff. > > RF > |
From: Rhythmic F. <rfi...@gm...> - 2011-09-28 10:29:02
|
I just tracked down a CPU burning bug to felix's waitable_bool class: // a waitable boolean. class PTHREAD_EXTERN waitable_bool { flx::pthread::flx_mutex_t cv_lock; // to work with the condition var flx::pthread::flx_condv_t finished_cond; bool finished; // might seem redundant, but that's how CVs work. public: waitable_bool(); void wait_until_true(); void signal_true(); }; Felix uses it to politely ask threads to quit, which works fine. What I'd forgotten (or never knew) is that this is a one shot object - once set to true, it's always true and all further calls to wait_until_true correctly return immediately, hence my CPU burn. I'd like to modify it by adding a set_to_false method (not signal, because there's no wait_until_false!). I'm writing to this list to see if that raises any alarm bells (what if there are multiple waiters? in felix there aren't, nor are there in my use) and it would be nice to check such a change in, even though my branch of the demux/pthread stuff is fossilised at svn commit #1695 because after that the lightweight pthread wrapper pulled in a bunch of garbage collection stuff. RF |
From: <no...@fe...> - 2010-10-21 15:06:21
|
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "An advanced programming language". The branch, master has been updated via f0fdfe2d60b6419e7f83e18326f47c90f825b24b (commit) from f7b6c2bc40d70920bba87539a5e068be995d34b6 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit f0fdfe2d60b6419e7f83e18326f47c90f825b24b Author: Erick Tryzelaar <ida...@us...> Date: Thu Oct 21 08:05:47 2010 -0700 Remove flx_parse.mli. diff --git a/src/compiler/flx_parse/flx_parse.mli b/src/compiler/flx_parse/flx_parse.mli deleted file mode 100644 index e5ecdba..0000000 --- a/src/compiler/flx_parse/flx_parse.mli +++ /dev/null @@ -1,41 +0,0 @@ -type 'a parser_state_t = - (Flx_ast.statement_t -> 'a -> 'a) * - 'a * - Flx_token.local_data_t - -(** Create a new parser state. *) -val make_parser_state: - (Flx_ast.statement_t -> 'a -> 'a) -> (** A function that folds statements. *) - 'a -> (** Initial data to fold. *) - 'a parser_state_t - -(** Extract the current compliation unit from the parser state. *) -val parser_data: 'a parser_state_t -> 'a - -(** Parse a channel and return the new parser state. *) -val parse_channel: - ?name:string -> (** The optional name for this channel. *) - 'a parser_state_t -> (** The state for the felix parser. *) - in_channel -> (** The channel to parse. *) - 'a parser_state_t - -(** Parse a file and return the new parser state. *) -val parse_file: - ?include_dirs: string list -> - 'a parser_state_t -> (** The state for the felix parser. *) - string -> (** The filename to parse. *) - 'a parser_state_t - -(** Parse a string and return the new parser state. *) -val parse_string: - ?name:string -> (** The optional name for this string. *) - 'a parser_state_t -> (** The state for the felix parser. *) - string -> (** The string to parse. *) - 'a parser_state_t - -(** Parse a function and return the new parser state. *) -val parse_function: - ?name:string -> (** The optional name for this string. *) - 'a parser_state_t -> (** The state for the felix parser. *) - (string -> int -> int) -> (** The lexer function generator. *) - 'a parser_state_t ----------------------------------------------------------------------- Summary of changes: src/compiler/flx_parse/flx_parse.mli | 41 ---------------------------------- 1 files changed, 0 insertions(+), 41 deletions(-) delete mode 100644 src/compiler/flx_parse/flx_parse.mli hooks/post-receive -- An advanced programming language |
From: <no...@fe...> - 2010-10-21 15:05:09
|
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "An advanced programming language". The branch, master has been updated via f7b6c2bc40d70920bba87539a5e068be995d34b6 (commit) via 9c417e009c004a24f4709874fffe76b454b06bb0 (commit) via ace430cfa9a6260375550e4cc23274173266de52 (commit) via 997860f361af8d15362ca6bb2fc68736dcc41316 (commit) via 1db6b034cbd260191845098648fdb65d597167f3 (commit) via 5f8a60e0040585d9812c2824f4854eb6dbf91647 (commit) via 0afe2ccec025ee6b3411767c6b003cf5ec1c7617 (commit) via 2960d0fcaa8810ebbdb48fd4975ab26e0ceb0e21 (commit) via 2dc112fbe00f65851115609be12549335a60cff5 (commit) via 721748e6fd188340e07207082445b461ef856440 (commit) via ddf70a63ee273efa59c2a3f9b6bc3bec9a83cc6a (commit) via 2b7679f462a2f271271b84688a964537115a806e (commit) via 753016f13e52956ea3f34e6c391c290bdf694cf5 (commit) via 4e78d45f996adcc1083a2516c10c8df7bd4add54 (commit) from fc4f483d20272dbf308e40cb352558b82fc2ca0c (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit f7b6c2bc40d70920bba87539a5e068be995d34b6 Author: Erick Tryzelaar <ida...@us...> Date: Thu Oct 21 07:53:06 2010 -0700 Migrate to latest fbuild. diff --git a/buildsystem/iscr.py b/buildsystem/iscr.py index b0fff82..4c22a07 100644 --- a/buildsystem/iscr.py +++ b/buildsystem/iscr.py @@ -69,7 +69,7 @@ class Iscr(fbuild.db.PersistentObject): if m: dsts.append(Path(m.group(1))) - fbuild.db.add_external_dependencies_to_call(self.ctx, srcs=srcs) + self.ctx.db.add_external_dependencies_to_call(srcs=srcs) return dsts diff --git a/buildsystem/speed.py b/buildsystem/speed.py index 1278e0b..4b4aff6 100644 --- a/buildsystem/speed.py +++ b/buildsystem/speed.py @@ -149,7 +149,7 @@ def run_test(ctx, path, dst, srcs:fbuild.db.SRCS, expect:fbuild.db.OPTIONAL_SRC, # If we failed to compile, just move on return None, None - fbuild.db.add_external_dependencies_to_call(ctx, dsts=[exe]) + ctx.db.add_external_dependencies_to_call(dsts=[exe]) # Run the executable and measure the wall clock time ctx.logger.check('running ' + exe) diff --git a/fbuild b/fbuild index 3fa955e..1f75235 160000 --- a/fbuild +++ b/fbuild @@ -1 +1 @@ -Subproject commit 3fa955e5d8a238125d170923fbbe1c188ad1150a +Subproject commit 1f752353d445f0cd251a5e5191014e55306ee216 commit 9c417e009c004a24f4709874fffe76b454b06bb0 Merge: fc4f483 ace430c Author: Erick Tryzelaar <ida...@us...> Date: Thu Oct 21 07:36:12 2010 -0700 Merge branch 'master' of github.com:erickt/felix commit ace430cfa9a6260375550e4cc23274173266de52 Author: skaller <Max...@gm...> Date: Thu Oct 21 01:55:57 2010 +1100 Add syntax colouring for C/C++ files, and hyperlinking of #include files. As yet you can't jump from a felix header '#include "file.hpp"'; to C++. diff --git a/tools/webserver.flx b/tools/webserver.flx index 5e4e1bc..4b39015 100644 --- a/tools/webserver.flx +++ b/tools/webserver.flx @@ -206,11 +206,238 @@ span.big_keyword {color:#FF1010; } span.small_keyword {color:#802040; } span.qualifier {color:#A02020; } span.hack {color:#00FF00; } +span.preproc {color:#00FF00; } </style> </head> <body> """; +fun inarray[N](s:string, a:array[string,N])=> + mem (fun (x:string) => s == x) a +; + +// C++ and C +val cpp_big_keywords = + "class", + "struct", + "union", + "namespace", + "typedef", + "enum" +; + +val cpp_small_keywords = + "if", "while", "until","do","for","return","goto","std" +; + +val cpp_qualifiers = + "virtual", "inline", "static", + "int","long","unsigned","float","double","char","short","signed","void","size_t", + "const","volatile" +; + +val cpp_preproc = + "define","if","endif","else","include","ifdef","ifndef" +; + +proc write_cpp(k:socket_t, t:string, eof_flag: &bool) +{ + proc write_string(k:socket_t, t:string) + { + if not *eof_flag do Flx_stream::write_string(k,t,eof_flag); + else goto giveup; + done + } + + union state_t = + | sot // start of token + | id // processing identifier + | num // in a number + | sq // processing single quote string + | dq // processing double quote string + | ccomment // a C style comment + | cppcomment // a C++ style comment + ; + fun str(s:state_t) => match s with + | sot => "sot" + | id => "id" + | num => "num" + | sq => "sq" + | dq => "dq" + | ccomment=> "ccomment" + | cppcomment => "cppcomment" + endmatch; + + var i = 0; var s:state_t; + var ch = t.[i]; + proc next() { ch = t.[i]; ++i; } + fun ahead (j:int)=> t.[i+j-1]; + + var b = ""; + var last_id = ""; + var last_op = ""; + proc cp() { b += ch; } + proc ws() { + if last_id == "include" do // hackery + var n = b; + whilst n.[0] == char "'" or n.[0] == char '"' do n = n.[1 to]; done + whilst n.[-1] == char "'" or n.[-1] == char '"' do n = n.[to -1]; done + write_string(k,'<a href="/rtl/'+n+'" >' + b + '</a>') ; + else + write_string(k,'<span class=fstring>'+b+"</span>"); + done + } + proc w() { + //println$ "Token["+str s+"]="+b; + match s with + | dq => { ws; } + | sq => { ws; } + | ccomment=> { write_string(k,'<span class=comment>'+b+"</span>"); } + | cppcomment=> { write_string(k,'<span class=comment>'+b+"</span>"); } + | id => + { + last_id = b; + if inarray(b,cpp_big_keywords) do write_string(k,'<span class=big_keyword>'+b+"</span>"); + elif inarray(b,cpp_small_keywords) do write_string(k,'<span class=small_keyword>'+b+"</span>"); + elif inarray(b,cpp_qualifiers) do write_string(k,'<span class=qualifier>'+b+"</span>"); + elif last_op == "#" and inarray(b,cpp_preproc) do write_string(k,'<span class=preproc>'+b+"</span>"); last_op=""; + else write_string(k,b); done + } + | _ => + { + last_op=b; + if b == "<" do b = "<"; + elif b == ">" do b = ">"; + elif b == "&" do b = "&"; + done; + write_string(k,b); + } + endmatch; + b = ""; + } + + + goto nextt; + +contin:> // copy char and continue + cp(); + goto nextch; + +overrun:> // one past last char of token + w(); + s = sot; + goto thisch; + +lastch:> // last char of token + cp(); + w(); + +nextt:> // new token on next char + s = sot; + +nextch:> // next char + next(); + +thisch:> // same char, reconsider it + //println$ "Considering char " + str(ord(ch)); + if isnull ch goto fin; // out of data + match s with + | sot => + { + if isidstart ch do s = id; goto contin; + elif isdigit ch do s = num; goto contin; + elif issq ch do s = sq; goto contin; + elif isdq ch do s = dq; goto contin; + elif ch == char "/" do + if ahead(1) == char "/" do cp; next; s = cppcomment; goto contin + elif ahead(1) == char "*" do cp; next; s = ccomment; goto contin + else goto lastch + done + else cp; w; goto nextt; + done + } + + | id => + { + if isalphanum ch do goto contin; + else goto overrun; + done + } + | num => + { + if isnumeric ch do goto contin; + else goto overrun; + done + } + // single quoted strings + | sq => + { + if issq ch do goto lastch; + elif ch== char "<" do b+="<"; goto nextch; + elif ch== char ">" do b+=">"; goto nextch; + elif ch== char "&" do b+="&"; goto nextch; + else goto contin; + done + } + | dq => + { + if isdq ch do goto lastch; + elif ch== char "<" do b+="<"; goto nextch; + elif ch== char ">" do b+=">"; goto nextch; + elif ch== char "&" do b+="&"; goto nextch; + else goto contin; + done + } + // comments + | cppcomment => + { + if iseol ch do goto lastch; + else goto contin; + done + } + | ccomment => // doesn't handle nested comments yet + { + if ch == char "*" and ahead(1) == char "/" do + cp; + goto lastch; + else goto contin; + done + } + endmatch + ; // execute selected function + println$ "Unexpected drop thru"; + +fin:> + println "outof data"; + w(); // whatever is left over gets written +giveup:> +} + +proc serve_cpp(s:socket_t, fname:string) +{ + var eof_flag = false; + proc write_string(k:socket_t, t:string) + { + if not eof_flag do Flx_stream::write_string(k,t,&eof_flag); + else goto giveup; + done + } + + var flx = Text_file::load(fname); + if flx == "" do flx = Text_file::load(LIBROOT+"/"+fname); done + if flx == "" do flx = "NO FILE "+fname+" FOUND IN " + LIBROOT; done + println$ "Loaded C/C++file " + fname; + //println$ "Contents=" + flx; + write_string(s, html_header); + write_string(s, flx_head); + write_string(s,"<pre>"); + write_cpp(s, flx, &eof_flag); + write_string(s,"</pre>\n"); + write_string(s,"</body></html>\n"); +giveup:> +} + +// Felix val big_keywords = "module", "fun", @@ -237,7 +464,7 @@ val big_keywords = val small_keywords = "if", "then", "else", "elif", "endif", "do", "done", "in", "forall", "while", "whilst","to", "upto","downto", - "match","endmatch","with","requires" + "match","endmatch","with","requires","return","goto" ; val qualifiers = @@ -245,10 +472,7 @@ val qualifiers = ; val hack = "C_hack","C_hack"; // to make it an array we need 2 components -fun inarray[N](s:string, a:array[string,N])=> - mem (fun (x:string) => s == x) a -; - + proc write_felix(k:socket_t, t:string, eof_flag: &bool) { proc write_string(k:socket_t, t:string) @@ -328,7 +552,15 @@ proc write_felix(k:socket_t, t:string, eof_flag: &bool) elif inarray(b,hack) do write_string(k,'<span class=hack>'+b+"</span>"); else write_string(k,b); done } - | _ => { write_string(k,b); } + | _ => + { + last_op=b; + if b == "<" do b = "<"; + elif b == ">" do b = ">"; + elif b == "&" do b = "&"; + done; + write_string(k,b); + } endmatch; b = ""; } @@ -501,6 +733,9 @@ proc serve_file(infname: string, s: socket_t) print "suffix is "; print suffix; endl; if suffix == "flx" do serve_felix(s:socket_t, fname:string); + elif suffix == "cpp" or suffix == "hpp" + or suffix == "cxx" or suffix == "h" or suffix == "c" + do serve_cpp(s:socket_t, fname:string); else if WIN32 do win32_send_file(s,fname, suffix); commit 997860f361af8d15362ca6bb2fc68736dcc41316 Author: skaller <Max...@gm...> Date: Wed Oct 20 03:17:26 2010 +1100 Finally fix the webserver bug, which turned out to be a gc problem. Now it runs, and the gc is running too. diff --git a/lpsrc/flx_maker.pak b/lpsrc/flx_maker.pak index 39944f2..92430ba 100644 --- a/lpsrc/flx_maker.pak +++ b/lpsrc/flx_maker.pak @@ -733,8 +733,10 @@ if FELIX == 1 do write_include_file(filebase); else dbug?? println "Felix compilation skipped by switch"; + calpackages(); // need to do this here to set linker switches done + if STATIC == 0 do dbug?? println "Dynamic linkage"; CCMD=List::cat ' ' (List::list ( diff --git a/src/compiler/flx_cpp_backend/flx_ogen.ml b/src/compiler/flx_cpp_backend/flx_ogen.ml index acaea58..73928d0 100644 --- a/src/compiler/flx_cpp_backend/flx_ogen.ml +++ b/src/compiler/flx_cpp_backend/flx_ogen.ml @@ -522,7 +522,6 @@ let gen_offset_tables syms bsym_table module_name = if not is_pod then begin bcat s ("static void " ^ name ^ "_finaliser(collector_t *, void *p){\n"); bcat s (" (("^ tname ^ "*)p)->~" ^ tname ^ "();\n"); - bcat s (" p = (void*)((char*)p + sizeof("^tname^"));\n"); bcat s ("}\n") end ; @@ -530,7 +529,7 @@ let gen_offset_tables syms bsym_table module_name = bcat s (" " ^ old_ptr_map ^ ",\n"); bcat s (" \"" ^ name ^ "\",\n"); bcat s (" " ^ si k ^ ",\n"); - bcat s (" sizeof("^name^"),\n"); + bcat s (" sizeof("^tname^"),\n"); (* NOTE: size of ONE element!! *) bcat s ( if not is_pod then (" "^name^"_finaliser,\n") diff --git a/src/faio/faio_posixio.cpp b/src/faio/faio_posixio.cpp index aff1012..17923cb 100644 --- a/src/faio/faio_posixio.cpp +++ b/src/faio/faio_posixio.cpp @@ -94,7 +94,7 @@ socketio_wakeup::wakeup(posix_demuxer& demux) else if(wakeup_flags & PDEMUX_EOF) { connection_closed = true; - fprintf(stderr,"posix faio wakeup PDEMUX_EOF, connection closed = %d\n", connection_closed); + //fprintf(stderr,"posix faio wakeup PDEMUX_EOF, connection closed = %d\n", connection_closed); //pb.bytes_written=0; } @@ -112,8 +112,8 @@ socketio_wakeup::wakeup(posix_demuxer& demux) assert(wakeup_flags == PDEMUX_WRITE); //fprintf(stderr,"posix faio wakeup PDEMUX_WRITE, writing..\n"); connection_closed = posix_demuxer::socket_send(s, &pb); - if(connection_closed) - fprintf(stderr,"posix faio wakeup PDEMUX_WRITE, connection closed = %d\n", connection_closed); + //if(connection_closed) + // fprintf(stderr,"posix faio wakeup PDEMUX_WRITE, connection closed = %d\n", connection_closed); } // fprintf(stderr,"posthandle wakeup, this: %p, read: %i, len: %i, done %i\n", diff --git a/src/faio/flx_stream.flx b/src/faio/flx_stream.flx index 9681adb..cf4e078 100644 --- a/src/faio/flx_stream.flx +++ b/src/faio/flx_stream.flx @@ -263,7 +263,7 @@ module Flx_stream { var st: string=""; var finished = false; - fprintln$ cerr,"Get line from IByteStream "+id; + //fprintln$ cerr,"Get line from IByteStream "+id; whilst not finished do var len = 1; var eof: bool; diff --git a/src/gc/flx_collector.cpp b/src/gc/flx_collector.cpp index 0f0cc64..018873a 100644 --- a/src/gc/flx_collector.cpp +++ b/src/gc/flx_collector.cpp @@ -223,13 +223,17 @@ void *flx_collector_t::create_empty_array( void flx_collector_t::impl_finalise(void *fp) { assert(fp!=NULL); - gc_shape_t *shape = get_shape(fp); + //fprintf(stderr, "Finaliser for %p\n", fp); + gc_shape_t *shape = get_shape(fp); // inefficient, since we already know the shape! + //fprintf(stderr, "Got shape %p=%s\n", shape,shape->cname); void (*finaliser)(collector_t*, void*) = shape->finaliser; + //fprintf(stderr, "Got finaliser %p\n", finaliser); if (finaliser) { unsigned char *cp = (unsigned char*)fp; unsigned long n_used = get_count(fp) * shape->count; unsigned long eltsize = shape->amt; + //fprintf(stderr, "Finalising at %p for type %s %ld objects each size %ld\n", cp, shape->cname, n_used, eltsize); for(unsigned long j = 0; j<n_used; ++j) { (*finaliser)(this,(void*)cp); @@ -244,6 +248,7 @@ void flx_collector_t::unlink(void *fp) assert(fp!=NULL); // call the finaliser if there is one + //fprintf(stderr,"Calling finaliser\n"); impl_finalise(fp); allocation_count--; @@ -251,12 +256,15 @@ void flx_collector_t::unlink(void *fp) unsigned long n_objects = get_count(fp); unsigned long nobj = shape -> count * n_objects; std::size_t size = shape->amt * nobj; + //fprintf(stderr, "Uncounting %ld bytes\n", long(size)); allocation_amt -= size; // unlink the frame from the collectors list + //fprintf(stderr,"Removing address from Judy lists\n"); JudyLDel(&j_shape, (Word_t)fp, &je); JudyLDel(&j_nused, (Word_t)fp, &je); JudyLDel(&j_nalloc, (Word_t)fp, &je); + //fprintf(stderr,"Finished unlinking\n"); } void flx_collector_t::post_delete(void *fp) @@ -350,14 +358,19 @@ unsigned long flx_collector_t::sweep() if(debug) fprintf(stderr,"Garbage %p=%s\n",current,((gc_shape_t*)(*pshape & ~1UL))->cname); ++ sweeped; + //fprintf(stderr,"Unlinking ..\n"); unlink(current); + //fprintf(stderr,"Posting delete ..\n"); post_delete(current); + //fprintf(stderr,"Reaping done\n"); } else if(debug) fprintf(stderr,"Reachable %p=%s\n",current,((gc_shape_t*)(*pshape & ~1UL))->cname); + //fprintf(stderr,"Calling Judy for next object\n"); pshape = (Word_t*)JudyLNext(j_shape,(Word_t*)(void*)¤t,&je); + //fprintf(stderr,"Judy got next object %p\n",pshape); } parity = !parity; diff --git a/src/gc/flx_gc.cpp b/src/gc/flx_gc.cpp index 31c3838..7bdff9f 100644 --- a/src/gc/flx_gc.cpp +++ b/src/gc/flx_gc.cpp @@ -67,7 +67,7 @@ unsigned long gc_profile_t::actually_collect() { void *gc_profile_t::allocate( flx::gc::generic::gc_shape_t *shape, - unsigned long amt, + unsigned long count, bool allow_gc ) { @@ -78,20 +78,27 @@ void *gc_profile_t::allocate( { maybe_collect(); try { - return collector -> allocate(shape,amt); + return collector -> allocate(shape,count); } catch (flx::rtl::flx_out_of_memory_t&) { actually_collect(); - return collector -> allocate(shape,amt); + return collector -> allocate(shape,count); } } else - return collector -> allocate(shape,amt); + return collector -> allocate(shape,count); } }}} // end namespaces // in global namespace now .. +// +// NOTE: Felix arrays are two dimensional. The shape.amt field is the size of +// one element. The shape.count field is the number of elements for a static +// array type. The dynamic length is for varrays, it is stored in a judy array +// associated with the array address. If there is nothing in the judy array, +// the dynamic length is one. C++ operator new allocates arrays of dynamic length 1. +// void *operator new( std::size_t amt, flx::gc::generic::gc_profile_t &gcp, @@ -99,13 +106,13 @@ void *operator new( bool allow_gc ) { - if (amt != shape.amt) + if (amt != shape.amt * shape.count) { fprintf(stderr,"Shape size error: allocator size = %ld\n",amt); - fprintf(stderr,"Shape %s size = %ld\n",shape.cname,shape.amt); + fprintf(stderr,"Shape %s element size = %ld, element count = %;d\n",shape.cname,shape.amt,shape.count); abort(); } - void *p = gcp.allocate(&shape,1,allow_gc); + void *p = gcp.allocate(&shape,1,allow_gc); // dynamic array count = 1 return p; } commit 1db6b034cbd260191845098648fdb65d597167f3 Author: skaller <Max...@gm...> Date: Tue Oct 19 13:04:33 2010 +1100 Fix the webserver so it doesn't write on closed connections. Add some more colours and keywords. diff --git a/lpsrc/flx_maker.pak b/lpsrc/flx_maker.pak index de407f9..39944f2 100644 --- a/lpsrc/flx_maker.pak +++ b/lpsrc/flx_maker.pak @@ -433,7 +433,7 @@ dbug?? println$ "cpps="+cpps; dbug?? println$ "cppos="+cppos; var USER_ARGS = ""; whilst argno < System::argc do USER_ARGS+=" " + System::argv argno; ++argno; done -println$ "USER_ARGS=" + USER_ARGS; +//println$ "USER_ARGS=" + USER_ARGS; if NOOPTIMISE == 0 do dbug?? println "Set C++ compiler optimisation switches"; diff --git a/src/compiler/flx_cpp_backend/flx_gen.ml b/src/compiler/flx_cpp_backend/flx_gen.ml index de282eb..bcf9a23 100644 --- a/src/compiler/flx_cpp_backend/flx_gen.ml +++ b/src/compiler/flx_cpp_backend/flx_gen.ml @@ -996,7 +996,8 @@ let gen_exe filename | Function -> begin match label_kind with - | `Far -> assert false + | `Far -> failwith ("[gen_exe] In function " ^ Flx_bsym.id bsym ^ + ": Non-local going to label " ^s) | `Near -> " " ^ cid_of_flxid s ^ ":;\n" | `Unused -> "" diff --git a/src/demux/demux_demuxer.hpp b/src/demux/demux_demuxer.hpp index 98c65fc..a8df878 100644 --- a/src/demux/demux_demuxer.hpp +++ b/src/demux/demux_demuxer.hpp @@ -6,10 +6,22 @@ namespace flx { namespace demux { struct sel_param { char* buffer; // set on input - long buffer_size; // set on input - long bytes_written; // set on input and output + long buffer_size; // set on input + long bytes_written; // set on input and output + bool eof_detected; // NEW: a proper eof flag + // maybe implemented in Posix, + // not done in Windows yet (since I don't have a Windows development box) - bool finished() { return bytes_written == buffer_size; } + bool error_detected; // NEW: a proper error flag + // maybe implemented on Posix + // not done in Windows yet (since I don't have a Windows development box) + // NOTE: there's no indication of what the error is + // because that is platform dependent + // A normal EOF condition is NOT an error + // However most errors will be associated with an eof! + + sel_param() : buffer(0), buffer_size(0), bytes_written(0), eof_detected(false), error_detected(false) {} + bool finished() { return error_detected || eof_detected || (bytes_written == buffer_size); } }; // rename ... diff --git a/src/demux/posix/demux_posix_demuxer.cpp b/src/demux/posix/demux_posix_demuxer.cpp index 8e155ad..f3b7da4 100644 --- a/src/demux/posix/demux_posix_demuxer.cpp +++ b/src/demux/posix/demux_posix_demuxer.cpp @@ -37,15 +37,18 @@ posix_demuxer::socket_recv(int s, sel_param* pb) s,pb, int(pb->bytes_written), int(pb->buffer_size - pb->bytes_written), int(nbytes) ); */ - if(nbytes <= 0) + if(nbytes <= 0) // so what happens if the client wants to send 0 bytes? { if(nbytes == 0) { + pb->eof_detected = true; return true; // connection closed } else { perror("recv"); // can get reset connection here + pb->eof_detected = true; + pb->error_detected = true; return true; // so say closed, yeah? } } @@ -82,11 +85,22 @@ posix_demuxer::socket_send(int s, sel_param* pb) // what's the story with zero? Is that allowed or does it signal // that the connection closed? + // JS: According to the man page, it can happen on an async socket + // but for us this could be if the notification event lied + // OR a socket connection got closed in between the notification + // and this call: we can tell by looking at errno but that utterly + // sucks .. oh well, unix is a pretty bad OS in some ways + // OR .. it could happen if the client decided to send 0 bytes! + if(-1 == nbytes) { fprintf(stderr,"posix_demuxer: socket send failed, connection closed by client?\n"); perror("send"); fprintf(stderr,"Should have printed the error code above ..\n"); + pb->eof_detected = true; + pb->error_detected = true; // really, trying to write on a connection merely closed + // by the client is NOT an error, since there's no other way + // to tell than try to do a write return true; // I guess the connection closed } else diff --git a/src/demux/win/demux_overlapped.cpp b/src/demux/win/demux_overlapped.cpp index 8ab3dbd..2eee5f1 100644 --- a/src/demux/win/demux_overlapped.cpp +++ b/src/demux/win/demux_overlapped.cpp @@ -347,6 +347,7 @@ wsasocketio_control_block::start_overlapped() } fprintf(stderr,"WSARecv/Send returned SOCKET_ERR: %li\n", err); + ppb->eof_detected=true; return true; // assume it's bad and we won't get a wakeup } break; diff --git a/src/faio/faio_asyncio.cpp b/src/faio/faio_asyncio.cpp index bd7febd..529e74b 100644 --- a/src/faio/faio_asyncio.cpp +++ b/src/faio/faio_asyncio.cpp @@ -21,7 +21,7 @@ void flx_driver_request_base:: start_async_op(finote_t *fn_a) void flx_driver_request_base:: notify_finished() { - //fprintf(stderr, "faio_req=%p, Notify %p\n", this,fn); + //fprintf(stderr, "faio_req=%p, Notify finished %p\n", this,fn); assert(fn!=0); finote_t *fin = fn; fn=0; diff --git a/src/faio/faio_posixio.cpp b/src/faio/faio_posixio.cpp index 2913375..aff1012 100644 --- a/src/faio/faio_posixio.cpp +++ b/src/faio/faio_posixio.cpp @@ -88,11 +88,13 @@ socketio_wakeup::wakeup(posix_demuxer& demux) { connection_closed = true; //pb.bytes_written=0; + fprintf(stderr,"posix faio wakeup PDEMUX_ERROR, connection closed = %d\n", connection_closed); } else if(wakeup_flags & PDEMUX_EOF) { connection_closed = true; + fprintf(stderr,"posix faio wakeup PDEMUX_EOF, connection closed = %d\n", connection_closed); //pb.bytes_written=0; } @@ -110,7 +112,8 @@ socketio_wakeup::wakeup(posix_demuxer& demux) assert(wakeup_flags == PDEMUX_WRITE); //fprintf(stderr,"posix faio wakeup PDEMUX_WRITE, writing..\n"); connection_closed = posix_demuxer::socket_send(s, &pb); - //fprintf(stderr,"posix faio wakeup PDEMUX_WRITE, connection closed = %d\n", connection_closed); + if(connection_closed) + fprintf(stderr,"posix faio wakeup PDEMUX_WRITE, connection closed = %d\n", connection_closed); } // fprintf(stderr,"posthandle wakeup, this: %p, read: %i, len: %i, done %i\n", @@ -121,6 +124,10 @@ socketio_wakeup::wakeup(posix_demuxer& demux) if(connection_closed || pb.bytes_written == pb.buffer_size) { // fprintf(stderr,"schedding %p, drv: %p, f: %p\n", this, drv, f); + // if the connection closed, this notify should tell the caller + // not to keep trying to write, but it doesn't .. why not? + // who called it anyhow? + // I think the writing code ignores error returns .. request->notify_finished(); return; } diff --git a/src/faio/flx_faio.flx b/src/faio/flx_faio.flx index 01e7f33..3f5c827 100644 --- a/src/faio/flx_faio.flx +++ b/src/faio/flx_faio.flx @@ -26,9 +26,14 @@ module Faio { proc calc_eof(pb: sel_param_ptr, len: &int, eof: &bool) { + //println "Calc_eof .."; var bytes_done = pb.bytes_done; + //println$ "Bytes done = "+ str bytes_done; + //println$ "Req len= "+ str (*len); *eof = (bytes_done != *len); + //println$ "Eof = " + str (*eof); *len = bytes_done; + //println$ "Reset len to bytes done .."; } type sleep_request = 'flx::faio::sleep_request' requires faio_timer_h; diff --git a/src/faio/flx_faio_posix.flx b/src/faio/flx_faio_posix.flx index 3d51b26..d61f6e2 100644 --- a/src/faio/flx_faio_posix.flx +++ b/src/faio/flx_faio_posix.flx @@ -103,9 +103,13 @@ fun get_pb: socketio_request -> sel_param_ptr = '&$1.sv.pb'; // read & write differ only by a flag proc async_rw(fd: socket_t, len: &int, buf: address, eof: &bool, read_flag: bool) { + //println$ "faio/flx_faoi_posix.flx: async_rw (s,"+str (*len)+",buf,"+str(*eof)+", "+str read_flag+") calling mk_socketio_req .."; var asyncb = mk_socketio_request(sys_demux,fd, buf, *len, read_flag); faio_req$ &asyncb; + //println$ "faio/flx_faoi_posix.flx: async_rw ("+ str fd+", "+str (*len)+",buf,"+str(*eof)+", "+str read_flag+") calculating eof .."; + calc_eof(asyncb.pb, len, eof); + //println$ "faio/flx_faoi_posix.flx: async_rw (s,"+str (*len)+",buf,"+str(*eof)+", "+str read_flag+") called mk_socketio_req .."; } proc async_read(fd: socket_t, len: &int, buf: address, @@ -116,7 +120,9 @@ proc async_read(fd: socket_t, len: &int, buf: address, proc async_write(fd: socket_t, len: &int, buf: address, eof: &bool) { + //println$ "faio/flx_faoi_posix.flx: async_write(s,"+str (*len)+",buf,"+str(*eof)+" calling async_rw .."; async_rw(fd, len, buf, eof, false); // write + //println$ "faio/flx_faoi_posix.flx: async_write(s,"+str (*len)+",buf,"+str(*eof)+" call async_rw .."; } type flxfileio_request = "flx::faio::flxfileio_request"; diff --git a/src/faio/flx_socket.flx b/src/faio/flx_socket.flx index 9463593..e2f1845 100644 --- a/src/faio/flx_socket.flx +++ b/src/faio/flx_socket.flx @@ -77,7 +77,11 @@ module Flx_socket { { if POSIX do proc write(s: socket_t, len: &int, buf: address, eof: &bool) - { Faio_posix::async_write(s, len, buf, eof); } + { + //println$ "faio/flx_socket.flx: Flx_stream::OByteStream[socket_t]: write(s,"+str (*len)+",buf,"+str(*eof)+") calling async_write .."; + Faio_posix::async_write(s, len, buf, eof); + //println$ "faio/flx_socket.flx: Flx_stream::OByteStream[socket_t]: write(s,"+str (*len)+",buf,"+str(*eof)+") called async_write .."; + } else proc write(s: socket_t, len: &int, buf: address, eof: &bool) { Faio_win32::WSASend(s, len, buf, eof); } diff --git a/src/faio/flx_stream.flx b/src/faio/flx_stream.flx index a6a9279..9681adb 100644 --- a/src/faio/flx_stream.flx +++ b/src/faio/flx_stream.flx @@ -302,11 +302,10 @@ module Flx_stream { proc write_string[ostr with OByteStream[ostr]] - (sk: ostr, var s: string) + (sk: ostr, var s: string, eof: &bool) { var slen = len s; var a = C_hack::cast[address]$ cstr s; - var eof: bool; - write(sk, &slen, a, &eof); + write(sk, &slen, a, eof); } } // module Flx_stream diff --git a/src/pthread/pthread_condv.cpp b/src/pthread/pthread_condv.cpp index d6f190c..b4200b8 100644 --- a/src/pthread/pthread_condv.cpp +++ b/src/pthread/pthread_condv.cpp @@ -4,15 +4,59 @@ namespace flx { namespace pthread { -flx_condv_t::flx_condv_t() { pthread_cond_init(&cv, NULL); } -flx_condv_t::~flx_condv_t() { pthread_cond_destroy(&cv); } -void flx_condv_t::wait(flx_mutex_t *m) { pthread_cond_wait(&cv,&(m->m)); } -void flx_condv_t::signal() { pthread_cond_signal(&cv);} -void flx_condv_t::broadcast() { pthread_cond_broadcast(&cv); } +flx_condv_t::flx_condv_t() { + int res = pthread_cond_init(&cv, NULL); + #if !FLX_WIN32 + if(res==EINVAL) { + // I suspect this is an error .. perhaps something got deleted + fprintf(stderr,"pthred_cond_init returned EINVAL!"); + } + #endif +} +flx_condv_t::~flx_condv_t() { + int res = pthread_cond_destroy(&cv); + #if !FLX_WIN32 + if(res==EINVAL) { + // I suspect this is an error .. perhaps something got deleted + fprintf(stderr,"pthred_cond_destroy returned EINVAL!"); + } + #endif +} +void flx_condv_t::wait(flx_mutex_t *m) { + int res = pthread_cond_wait(&cv,&(m->m)); + #if !FLX_WIN32 + if(res==EINVAL) { + // I suspect this is an error .. perhaps something got deleted + fprintf(stderr,"pthred_cond_wait returned EINVAL!"); + } + #endif +} +void flx_condv_t::signal() { + int res = pthread_cond_signal(&cv); + #if !FLX_WIN32 + if(res==EINVAL) { + // I suspect this is an error .. perhaps something got deleted + fprintf(stderr,"pthred_cond_signal returned EINVAL!"); + } + #endif +} +void flx_condv_t::broadcast() { + int res = pthread_cond_broadcast(&cv); + #if !FLX_WIN32 + if(res==EINVAL) { + // I suspect this is an error .. perhaps something got deleted + fprintf(stderr,"pthred_cond_broadcast returned EINVAL!"); + } + #endif +} int flx_condv_t::timedwait(flx_mutex_t *m, timespec *t) { int res = pthread_cond_timedwait(&cv,&(m->m),t); #if !FLX_WIN32 - if(res==EINVAL) return 0; // this is NOT an error! + if(res==EINVAL) { + // I suspect this is an error .. perhaps something got deleted + fprintf(stderr,"pthred_cond_timedwait returned EINVAL!"); + return 0; // this is NOT an error! + } #endif return res; } diff --git a/test/faio/faio-1.01.01-0.flx b/test/faio/faio-1.01.01-0.flx index 20ef86e..a585338 100644 --- a/test/faio/faio-1.01.01-0.flx +++ b/test/faio/faio-1.01.01-0.flx @@ -23,6 +23,7 @@ print "spawning connector\n"; spawn_fthread { { + var eof = false; // print "Connector dude\n"; // get rid of, hard to test var c: socket_t; connect(&c, c"127.0.0.1", port); // connect to localhost @@ -31,7 +32,7 @@ spawn_fthread get_line(c, &st,"fthread"); print "connector got "; print st; endl; - write_string(c, "thanks\n"); // newline important + write_string(c, "thanks\n", &eof); // newline important ioclose(c); // finished with this //println$ "fthread closed " + str c; @@ -43,8 +44,9 @@ accept(listener, &s); //println$ "Mainline accepted connection on socket " + str s; ioclose(listener); // not needed anymore +var eof = false; print "got connection\n"; -write_string(s, "server says hi\n"); // newline important here +write_string(s, "server says hi\n", &eof); // newline important here var st: string; get_line(s, &st,"mainline"); diff --git a/tools/webserver.flx b/tools/webserver.flx index 58d78dd..5e4e1bc 100644 --- a/tools/webserver.flx +++ b/tools/webserver.flx @@ -113,12 +113,12 @@ Content-Type: text/html\r PAGE NOT FOUND: """; -proc write_http_header(s:socket_t, suffix:string) +proc write_http_header(s:socket_t, suffix:string, eof: &bool) { // mime type mapping from suffix. make better here. - if("gif" == suffix) then { write_string(s, gif_header); } - elif("css" == suffix) then { write_string(s, css_header); } - else { write_string(s, html_header); } endif; + if("gif" == suffix) then { write_string(s, gif_header, eof); } + elif("css" == suffix) then { write_string(s, css_header, eof); } + else { write_string(s, html_header, eof); } endif; } if WIN32 do @@ -143,7 +143,7 @@ if WIN32 do done } elif POSIX do - proc posix_send_file(s:socket_t,fname:string,suffix:string) + proc posix_send_file(s:socket_t,fname:string,suffix:string, eof: &bool) { // this fn sets the O_NONBLOCK flag which is completely unnecessary // as read goes via the preading worker fifo. don't know if @@ -153,14 +153,14 @@ elif POSIX do if Faio_posix::invalid fd do { print ("BUGGER, posix open of "+fname+" failed\n"); - write_string(s, notfound_header); - write_string(s, fname+"\r\n\n"); + write_string(s, notfound_header, eof); + write_string(s, fname+"\r\n\n", eof); } else { print ("got fd "+str fd +" for read file of "+fname+"\n"); - write_http_header(s,suffix); + write_http_header(s,suffix, eof); var from_strm: Faio_posix::fd_t = fd; var to_strm: socket_t = s; Flx_stream::cat(from_strm, to_strm); @@ -202,8 +202,10 @@ flx_head := """ <style type="text/css"> span.fstring {color:darkblue; font-style:italic; } span.comment {font-family:arial; color:blue; font-style:italic; } -span.big_keyword {color:red; } -span.small_keyword {color:darkred; } +span.big_keyword {color:#FF1010; } +span.small_keyword {color:#802040; } +span.qualifier {color:#A02020; } +span.hack {color:#00FF00; } </style> </head> <body> @@ -212,8 +214,11 @@ span.small_keyword {color:darkred; } val big_keywords = "module", "fun", + "gen", "proc", "type", + "union", + "struct", "typedef", "var", "val", @@ -225,7 +230,9 @@ val big_keywords = "include", "open", "spawn_fthread", - "spawn_pthread" + "spawn_pthread", + "reduce", "axiom", + "open", "inherit" ; val small_keywords = "if", "then", "else", "elif", "endif", "do", "done", @@ -233,12 +240,24 @@ val small_keywords = "match","endmatch","with","requires" ; +val qualifiers = + "virtual", "inline", "noinline", "private", "incomplete" +; +val hack = "C_hack","C_hack"; // to make it an array we need 2 components + fun inarray[N](s:string, a:array[string,N])=> mem (fun (x:string) => s == x) a ; -proc write_felix(k:socket_t, t:string) +proc write_felix(k:socket_t, t:string, eof_flag: &bool) { + proc write_string(k:socket_t, t:string) + { + if not *eof_flag do Flx_stream::write_string(k,t,eof_flag); + else goto giveup; + done + } + union state_t = | sot // start of token | id // processing identifier @@ -305,6 +324,8 @@ proc write_felix(k:socket_t, t:string) last_id = b; if inarray(b,big_keywords) do write_string(k,'<span class=big_keyword>'+b+"</span>"); elif inarray(b,small_keywords) do write_string(k,'<span class=small_keyword>'+b+"</span>"); + elif inarray(b,qualifiers) do write_string(k,'<span class=qualifier>'+b+"</span>"); + elif inarray(b,hack) do write_string(k,'<span class=hack>'+b+"</span>"); else write_string(k,b); done } | _ => { write_string(k,b); } @@ -427,25 +448,36 @@ thisch:> // same char, reconsider it fin:> println "outof data"; w(); // whatever is left over gets written +giveup:> } proc serve_felix(s:socket_t, fname:string) { - var flx = Text_file::load(fname); - if flx == "" do flx = Text_file::load(LIBROOT+"/"+fname); done - if flx == "" do flx = "NO FILE "+fname+" FOUND IN " + LIBROOT; done - println$ "Loaded felix file " + fname; - //println$ "Contents=" + flx; - write_string(s, html_header); - write_string(s, flx_head); - write_string(s,"<pre>"); - write_felix(s, flx); - write_string(s,"</pre>\n"); - write_string(s,"</body></html>\n"); + var eof_flag = false; + proc write_string(k:socket_t, t:string) + { + if not eof_flag do Flx_stream::write_string(k,t,&eof_flag); + else goto giveup; + done + } + + var flx = Text_file::load(fname); + if flx == "" do flx = Text_file::load(LIBROOT+"/"+fname); done + if flx == "" do flx = "NO FILE "+fname+" FOUND IN " + LIBROOT; done + println$ "Loaded felix file " + fname; + //println$ "Contents=" + flx; + write_string(s, html_header); + write_string(s, flx_head); + write_string(s,"<pre>"); + write_felix(s, flx, &eof_flag); + write_string(s,"</pre>\n"); + write_string(s,"</body></html>\n"); +giveup:> } proc serve_file(infname: string, s: socket_t) { + var eof_flag = false; // if empty string, serve index.html // not quite right - needs to handle directories too, so // not only foo.com/ -> index.html, but foo.com/images/ -> images/index.html @@ -472,7 +504,7 @@ proc serve_file(infname: string, s: socket_t) else if WIN32 do win32_send_file(s,fname, suffix); - elif POSIX do posix_send_file(s,fname, suffix); + elif POSIX do posix_send_file(s,fname, suffix,&eof_flag); done done } @@ -524,7 +556,7 @@ noinline proc handler (var k:socket_t) () //cat(s, DEVNULL); fprint$ cerr,"fthread closing socket "+str k+"\n"; - Faio::sleep(clock,1.0); // give OS time to empty its buffers + Faio::sleep(clock,0.1); // give OS time to empty its buffers ioclose(k); fprint$ cerr,"fthread "+str k+" terminating!\n"; }; commit 5f8a60e0040585d9812c2824f4854eb6dbf91647 Author: skaller <Max...@gm...> Date: Mon Oct 18 04:15:42 2010 +1100 Hyperlinks in webserver, use --root=FLX_INSTALL_DIR to set the root for include statements to do their lookup. diff --git a/lpsrc/flx_maker.pak b/lpsrc/flx_maker.pak index 5b82247..de407f9 100644 --- a/lpsrc/flx_maker.pak +++ b/lpsrc/flx_maker.pak @@ -432,6 +432,9 @@ dbug?? println$ "path="+path+": dir="+dir+",base="+base", ext="+ext; dbug?? println$ "cpps="+cpps; dbug?? println$ "cppos="+cppos; +var USER_ARGS = ""; whilst argno < System::argc do USER_ARGS+=" " + System::argv argno; ++argno; done +println$ "USER_ARGS=" + USER_ARGS; + if NOOPTIMISE == 0 do dbug?? println "Set C++ compiler optimisation switches"; CCFLAGS=CCFLAGS+" " + OPTIMISE; @@ -607,17 +610,7 @@ proc write_include_file(path:string) { Text_file::fclose f; } - -// grab program arguments -grab=1; -fun pop (x:List::list[string], n:int) => - if n == 0 then x - else match x with | Cons(_,?t) => pop(t,n-1) | Empty[string] => List::Empty[string] endmatch - endif -; - -var tail = pop (System::args(), argno); -var args= List::cat " " tail; +val args = USER_ARGS; dbug?? println$ "Target program args = "+args; var INCLUDE_DIR="-I"+Filename::join(FLX_INSTALL_DIR,"lib","rtl") + " -I"+Filename::join(FLX_INSTALL_DIR,"config","target"); diff --git a/tools/webserver.flx b/tools/webserver.flx index 492d105..58d78dd 100644 --- a/tools/webserver.flx +++ b/tools/webserver.flx @@ -24,6 +24,22 @@ macro fun dbg(x) = { fprint (cerr,x); }; // read new sockets off it .. open TerminalIByteStream[socket_t]; +var arg = ""; +var argno = 1; +fun prefix(arg:string,key:string)=> + arg.[to len key]==key +; +var LIBROOT = ""; +whilst argno<System::argc do + arg = System::argv argno; + println$ "ARG=" + arg; + if prefix(arg,"--root=") do + LIBROOT=arg.[7 to]+"/lib"; + done + ++argno; +done +println$ "LIBROOT="+LIBROOT; + fun getline_to_url (get:string) => if not startswith get "GET " then "" @@ -262,18 +278,31 @@ proc write_felix(k:socket_t, t:string) ; var b = ""; + var last_id = ""; proc cp() { b += ch; } + proc ws() { + if last_id == "include" do // hackery + var n = b; + whilst n.[0] == char "'" or n.[0] == char '"' do n = n.[1 to]; done + whilst n.[-1] == char "'" or n.[-1] == char '"' do n = n.[to -1]; done + if n.[-4 to] != ".flx" do n+= ".flx"; done // hack, fixme + write_string(k,'<a href="/'+n+'" >' + b + '</a>') ; + else + write_string(k,'<span class=fstring>'+b+"</span>"); + done + } proc w() { //println$ "Token["+str s+"]="+b; match s with - | dq => { write_string(k,'<span class=fstring>'+b+"</span>"); } - | sq => { write_string(k,'<span class=fstring>'+b+"</span>"); } - | sq3 => { write_string(k,'<span class=fstring>'+b+"</span>"); } - | dq3 => { write_string(k,'<span class=fstring>'+b+"</span>"); } + | dq => { ws; } + | sq => { ws; } + | sq3 => { ws; } + | dq3 => { ws; } | ccomment=> { write_string(k,'<span class=comment>'+b+"</span>"); } | cppcomment=> { write_string(k,'<span class=comment>'+b+"</span>"); } | id => { + last_id = b; if inarray(b,big_keywords) do write_string(k,'<span class=big_keyword>'+b+"</span>"); elif inarray(b,small_keywords) do write_string(k,'<span class=small_keyword>'+b+"</span>"); else write_string(k,b); done @@ -318,11 +347,11 @@ thisch:> // same char, reconsider it elif issq ch do s = sq; goto contin; elif isdq ch do s = dq; goto contin; elif ch == char "/" do - if ahead(1) == char "/" do cp(); next(); s = cppcomment; goto contin - elif ahead(1) == char "*" do cp(); next(); s = ccomment; goto contin + if ahead(1) == char "/" do cp; next; s = cppcomment; goto contin + elif ahead(1) == char "*" do cp; next; s = ccomment; goto contin else goto lastch done - else cp(); w(); goto nextt; + else cp; w; goto nextt; done } @@ -360,7 +389,7 @@ thisch:> // same char, reconsider it // triple quoted strings | sq3 => { - if issq3() do cp(); next(); cp(); next(); cp(); w(); goto nextt; + if issq3() do cp; next; cp; next; cp; w; goto nextt; elif ch== char "<" do b+="<"; goto nextch; elif ch== char ">" do b+=">"; goto nextch; elif ch== char "&" do b+="&"; goto nextch; @@ -369,7 +398,7 @@ thisch:> // same char, reconsider it } | dq3 => { - if isdq3() do cp(); next(); cp(); next(); cp(); w(); goto nextt; + if isdq3() do cp; next; cp; next; cp; w; goto nextt; elif ch== char "<" do b+="<"; goto nextch; elif ch== char ">" do b+=">"; goto nextch; elif ch== char "&" do b+="&"; goto nextch; @@ -386,7 +415,7 @@ thisch:> // same char, reconsider it | ccomment => // doesn't handle nested comments yet { if ch == char "*" and ahead(1) == char "/" do - cp(); + cp; goto lastch; else goto contin; done @@ -402,15 +431,17 @@ fin:> proc serve_felix(s:socket_t, fname:string) { - val flx = Text_file::load(fname); + var flx = Text_file::load(fname); + if flx == "" do flx = Text_file::load(LIBROOT+"/"+fname); done + if flx == "" do flx = "NO FILE "+fname+" FOUND IN " + LIBROOT; done println$ "Loaded felix file " + fname; //println$ "Contents=" + flx; write_string(s, html_header); write_string(s, flx_head); write_string(s,"<pre>"); write_felix(s, flx); - write_string(s,"</pre>"); - write_string(s,"</body></html>"); + write_string(s,"</pre>\n"); + write_string(s,"</body></html>\n"); } proc serve_file(infname: string, s: socket_t) @@ -493,7 +524,7 @@ noinline proc handler (var k:socket_t) () //cat(s, DEVNULL); fprint$ cerr,"fthread closing socket "+str k+"\n"; - Faio::sleep(clock,0.1); // give OS time to empty its buffers + Faio::sleep(clock,1.0); // give OS time to empty its buffers ioclose(k); fprint$ cerr,"fthread "+str k+" terminating!\n"; }; commit 0afe2ccec025ee6b3411767c6b003cf5ec1c7617 Author: skaller <Max...@gm...> Date: Mon Oct 18 02:37:14 2010 +1100 Add --options flag to flx to list current values of variant options. A bit of a hack. diff --git a/lpsrc/flx_maker.pak b/lpsrc/flx_maker.pak index 8db770d..5b82247 100644 --- a/lpsrc/flx_maker.pak +++ b/lpsrc/flx_maker.pak @@ -133,7 +133,6 @@ var FLX_INSTALL_DIR = Env::getenv("FLX_INSTALL_DIR", INSTALL_ROOT); // check for test mode: this argument must come first -var TESTMODE=0; var RECOMPILE=0; var DEBUG=0; var DEBUG_COMPILER=0; @@ -227,12 +226,10 @@ whilst grab == 1 and argno<System::argc do elif prefix(arg,"--test=") do dbug?? println "Set test directory"; - TESTMODE=1; FLX_INSTALL_DIR=arg.[7 to]; elif arg=="--test" do dbug?? println "Set test directory"; - TESTMODE=1; FLX_INSTALL_DIR="."; elif arg=="--install" do @@ -347,6 +344,27 @@ whilst grab == 1 and argno<System::argc do println$ "DEBUG_FLAGS = "+str DEBUG_FLAGS; System::exit(0); + elif arg == "--options" do + println$ "NOOPTIMISE = "+str NOOPTIMISE; + println$ "STATIC = "+str STATIC; + println$ "ECHO = "+str ECHO; + println$ "NOSTDLIB = "+str NOSTDLIB; + println$ "DEBUG = "+str DEBUG; + println$ "DEBUG_COMPILER = "+str DEBUG_COMPILER; + println$ "STDIMPORT = "+str STDIMPORT; + println$ "IMPORTS = "+str IMPORTS; + println$ "RECOMPILE = "+str RECOMPILE; + println$ "cpps = "+str cpps; + println$ "cppos = "+str cppos; + println$ "TIME = "+str TIME; + println$ "OUTPUT_DIR = "+str OUTPUT_DIR; + println$ "RUNIT = "+str RUNIT; + println$ "INCLUDE_DIRS = "+str INCLUDE_DIRS; + println$ "FELIX = "+str FELIX; + println$ "LINKER_SWITCHES = "+str LINKER_SWITCHES; + println$ "MACROS = "+str MACROS; + System::exit(0) + elif arg == "--where" do dbug?? println "Print location of install directory and exit"; println(FLX_INSTALL_DIR); @@ -928,7 +946,6 @@ FLX_INSTALL_DIR = os.getenv("FLX_INSTALL_DIR", INSTALL_ROOT) # check for test mode: this argument must come first -TESTMODE=0 RECOMPILE=0 DEBUG=0 DEBUG_COMPILER=0 @@ -1002,11 +1019,9 @@ while grab == 1 and argno<len(sys.argv): IMPORTS=IMPORTS + " " + arg[9:] elif prefix(arg,"--test="): - TESTMODE=1 FLX_INSTALL_DIR=arg[7:] elif arg=="--test": - TESTMODE=1 FLX_INSTALL_DIR="." elif prefix(arg,"--stdout="): @@ -1410,7 +1425,6 @@ sys.exit(0!=result) # check for test mode: this argument must come first -TESTMODE=0 RECOMPILE=0 DEBUG=0 INLINE=100 @@ -1454,13 +1468,11 @@ do ;; x--test=*) - TESTMODE=1 FLX_INSTALL_DIR=${1:7} shift ;; x--test) - TESTMODE=1 FLX_INSTALL_DIR=. shift ;; @@ -1986,132 +1998,6 @@ echo "---------" exit $bad -@head(1,'Package Manager Meta Info') -@head(2,'GODI') -This is the Godiva file originally used to -create the GODI data. At the moment this is -the authoritative meta data. However, -godiva may not handle all the options -we need -- so the generated makefile -is included as well. - -@select(tangler('meta/godiva/flx.godiva','data')) -Package: apps-felix -@tangle('Version: '+config.flx_version) -Revision: 0 -Depends: -Build-Depends: godi-ocaml (> 3.08) -@tangle('Sources: http://felix.sf.net/flx_'+config.flx_version+'_src.tgz') -@tangle('Unpacks-To: flx_'+config.flx_version) -Bytecode-Target: all -Opt-Target: all -Homepage: http://felix.sf.net -Maintainer: John Skaller <sk...@us...> -Options: configure -Description: Felix Compiler -Felix Compiler -. - -@select(tangler('meta/godiva/flx.godiva_camlsyntax','data')) -name = "felix"; -@tangle('version = "'+config.flx_version+'";') -revision = 0; -category = `apps; -depends = []; -build_depends = [`godi,"ocaml", Some (`gt, "3.08")]; -sources_site ="http://felix.sf.net/"; -@tangle('sources_basename= "flx_'+config.flx_version+'_src";') -sources_extension = ".tgz"; -@tangle('sources_unpacksto = "flx_'+config.flx_version+'";') -all_target= "all"; -opt_target= "all"; -homepage= "http://felix.sf.net"; -maintainer = "John Skaller <sk...@us...>"; -options= [`configure]; -short_desc = "Felix Compiler"; -long_desc = "Felix Compiler"; -confopts = [ - { - name = "SUPPORT_DYNAMIC_LOADING"; - default = "1"; - description = "Whether to support dlopen loading"; - implementation = `configarg "--SUPPORT_DYNAMIC_LOADING" - } -]; -specfile = "meta/godiva/flx.godiva_camlsyntax"; -patches = []; -filesdir = None; - -@select(tangler('meta/godi/DESCR','data')) -Felix Compiler and tools. - -@doc() -This makefile only here for reference (don't use it, -it should be generated). - -@select(tangler('meta/godi/Makefile','data')) -# This file was automatically generated by GODIVA -.include "../../mk/bsd.prefs.mk" -.include "../../mk/godi.pkg.mk" - -@tangle('VERSION= '+config.flx_version) -PKGNAME= apps-felix-${VERSION} -@tangle('PKGREVISION= '+config.godi_revision) -@tangle('DISTNAME= flx_'+config.flx_version) -@tangle('DISTFILES= flx_'+config.flx_version+'_src.tgz') -CATEGORIES= apps -MASTER_SITES= http://felix.sf.net/ -MAINTAINER= John Skaller <sk...@us...> -HOMEPAGE= http://felix.sf.net -COMMENT= Felix Compiler - -# confopt defaults: - - -AUTOGENERATE_PLIST = yes -PKG = apps-felix -MAKE_FLAGS= PREFIX=${PREFIX} - - - -PATH:= ${LOCALBASE}/bin:"${PATH}" -HAS_CONFIGURE = yes -CONFIGURE_ARGS+= --prefix ${PREFIX} -CONFIGURE_ENV+= ${BUILD_OCAMLFIND_ENV} -USE_GMAKE = yes - -MAKE_ENV+= ${BUILD_OCAMLFIND_ENV} PKGBASE=${PKGBASE:Q} - -pre-configure-copy: -. if exists(files) - cd files && ${PAX} -rw -pp . ${WRKSRC} -. endif - -pre-configure: pre-configure-copy - -pre-install-mkdirs: -. for d in bin lib/ocaml/pkg-lib doc share man etc info sbin include - ${_PKG_SILENT}${_PKG_DEBUG}mkdir -p ${PREFIX}/${d} -. endfor -. for n in 1 2 3 4 5 6 7 8 9 - ${_PKG_SILENT}${_PKG_DEBUG}mkdir -p ${PREFIX}/man/man${n} -. endfor - -pre-install: pre-install-mkdirs - -ALL_TARGET= all -.if ${GODI_HAVE_OCAMLOPT} == "yes" -# ALL_TARGET+= all -.endif - -post-install: - mkdir -p ${PREFIX}/doc/${PKG} -. for DOC in - install -m 0644 ${WRKSRC}/${DOC} ${PREFIX}/doc/${PKG} -. endfor - -.include "../../mk/bsd.pkg.mk" - @head(1,'Finish up') Just cleaning up script now. @make_executable(os.path.join('bin', 'flx.sh')) commit 2960d0fcaa8810ebbdb48fd4975ab26e0ceb0e21 Author: skaller <Max...@gm...> Date: Mon Oct 18 00:19:48 2010 +1100 Install option to retain file time stamps is "cp -Rp b src/ dst" diff --git a/lpsrc/flx_maker.pak b/lpsrc/flx_maker.pak index 3d3c967..8db770d 100644 --- a/lpsrc/flx_maker.pak +++ b/lpsrc/flx_maker.pak @@ -252,7 +252,7 @@ whilst grab == 1 and argno<System::argc do println$ "Cannot create directory " + INSTALL_ROOT_TOPDIR; System::exit 1; done - result=system("cp -ra "+FLX_INSTALL_DIR+" "+INSTALL_ROOT); + result=system("cp -Rp "+FLX_INSTALL_DIR+" "+INSTALL_ROOT); if result == 0 do println "Install succeeded" else println$ "Install failed, code = " + str(result); done commit 2dc112fbe00f65851115609be12549335a60cff5 Author: skaller <Max...@gm...> Date: Mon Oct 18 00:12:48 2010 +1100 Remove all the spkg stuff cause it isn't used now. Gets rid of some files too. diff --git a/lpsrc/flx_demux.pak b/lpsrc/flx_demux.pak index 38c763d..7d844e1 100644 --- a/lpsrc/flx_demux.pak +++ b/lpsrc/flx_demux.pak @@ -1,129 +1,6 @@ @import config @head(1,'demux') -@h = tangler('spkgs/demux.py') -@select(h) -import os - -import config - -DEMUXRTL_INTERFACES = [ - 'config/target/flx_demux_config.hpp', # portable - - # portable - 'src/demux/flx_demux.hpp', - 'src/demux/demux_demuxer.hpp', - 'src/demux/demux_timer_queue.hpp', - 'src/demux/demux_quitter.hpp', - - # windows (monolithic) - 'src/demux/win/demux_iocp_demuxer.hpp', - 'src/demux/win/demux_overlapped.hpp', - 'src/demux/win/demux_win_timer_queue.hpp', - 'src/demux/win/demux_wself_piper.hpp', - - # posix - 'src/demux/posix/demux_posix_demuxer.hpp', - 'src/demux/posix/demux_posix_timer_queue.hpp', - 'src/demux/posix/demux_pfileio.hpp', - 'src/demux/posix/demux_select_demuxer.hpp', - 'src/demux/posix/demux_sockety.hpp', - 'src/demux/posix/demux_self_piper.hpp', - 'src/demux/posix/demux_ts_select_demuxer.hpp', - - # linux, 10.3 (select impl), 10.4 real. - 'src/demux/poll/demux_poll_demuxer.hpp', - 'src/demux/poll/demux_ts_poll_demuxer.hpp', - - 'src/demux/epoll/demux_epoll_demuxer.hpp', # linux (>= 2.6) - 'src/demux/kqueue/demux_kqueue_demuxer.hpp', # osx(10.3 onwards)/bsd - 'src/demux/evtport/demux_evtport_demuxer.hpp', # solaris (9 onwards?) -] - -DEMUX_CPPS = [ - 'src/demux/flx_demux', - 'src/demux/demux_quitter', -] - -POSIX_DEMUX_CPPS = [ - 'src/demux/posix/demux_posix_demuxer', # posix - 'src/demux/posix/demux_select_demuxer', # posix - 'src/demux/posix/demux_posix_timer_queue', # posix - 'src/demux/posix/demux_sockety', # posix - 'src/demux/posix/demux_self_piper', # posix - 'src/demux/posix/demux_pfileio', # posix - 'src/demux/posix/demux_ts_select_demuxer', # posix -] - -POLL_DEMUX_CPPS = [ - # I've seen poll on linux and osx10.4 systems. - # conditionally compiled and used. - 'src/demux/poll/demux_poll_demuxer', # I've seen this on linux and osx10.4 - 'src/demux/poll/demux_ts_poll_demuxer', # ditto -] - -WINDOWS_DEMUX_CPPS = [ - 'src/demux/win/demux_iocp_demuxer', # windows - 'src/demux/win/demux_overlapped', # windows - 'src/demux/win/demux_wself_piper', # windows - 'src/demux/win/demux_win_timer_queue', # windows -] - -EXTRA_SYS_LIBS = '' -if config.WIN32: - DEMUX_CPPS = DEMUX_CPPS + WINDOWS_DEMUX_CPPS - if config.HAVE_MSVC: - EXTRA_SYS_LIBS = '/DEFAULTLIB:ws2_32 /DEFAULTLIB:mswsock ' - else: - # mingw - EXTRA_SYS_LIBS = '-lws2_32 -lmswsock ' - - -if config.POSIX: - DEMUX_CPPS = DEMUX_CPPS + POSIX_DEMUX_CPPS - -if config.TARGET_CXX.options.HAVE_KQUEUE_DEMUXER: - DEMUX_CPPS = DEMUX_CPPS + [ 'src/demux/kqueue/demux_kqueue_demuxer' ] - -if config.TARGET_CXX.options.HAVE_POLL: - DEMUX_CPPS = DEMUX_CPPS + POLL_DEMUX_CPPS - -if config.TARGET_CXX.options.HAVE_EPOLL: - DEMUX_CPPS = DEMUX_CPPS + [ 'src/demux/epoll/demux_epoll_demuxer' ] # Linux 2.6 + - -if config.TARGET_CXX.options.HAVE_EVTPORTS: - DEMUX_CPPS = DEMUX_CPPS + [ 'src/demux/evtport/demux_evtport_demuxer'] # solaris 10 - -if config.SOLARIS: - # RF: this might not be necessary anymore. - EXTRA_SYS_LIBS = '-lsocket -lnsl ' - -cpp_cpps = DEMUX_CPPS -lib_requires = ['libflx_pthread'] # however libflx not needed -pkg_requires = ['flx_pthread', 'flx_rtl'] # flx_rtl for config .hpp -dflags = EXTRA_SYS_LIBS -sflags = EXTRA_SYS_LIBS -build_macro = 'DEMUX' - -rtl_interfaces = DEMUXRTL_INTERFACES -felix_rtl = ['src/demux/flx_demux.flx'] - -iscr_source = [ - 'lpsrc/flx_demux.pak', -] - -weaver_directory = 'doc/rtl/flx_demux/' -tmpdir = ['demux'] -xfiles = [ - os.path.join('src', 'demux', '*'), - os.path.join('src', 'demux', 'epoll', '*'), - os.path.join('src', 'demux', 'evtport', '*'), - os.path.join('src', 'demux', 'kqueue', '*'), - os.path.join('src', 'demux', 'poll', '*'), - os.path.join('src', 'demux', 'posix', '*'), - os.path.join('src', 'demux', 'win', '*'), -] - @h = tangler('config/demux.fpc') @select(h) Name: demux diff --git a/lpsrc/flx_devutil.pak b/lpsrc/flx_devutil.pak deleted file mode 100644 index 875e9d7..0000000 --- a/lpsrc/flx_devutil.pak +++ /dev/null @@ -1,1220 +0,0 @@ -@head(1,"Developer utility scripts") -These script are NOT required to build Felix: they're utilies -related to munging the original code into interscript format, -and doing miscellaneous things during development. - -@head(2,'Attempt to produce Texinfo document') -Hack up the tutorial: phase 1 of conversion to texinfo format. -@h = tangler('script/mktitut.sed','python') -@select(h) -s/@set_title('\(.*\)')/\\input texinfo\n@settitle \1/ -s/@head(1,'\(.*\)')/@node \1\n@chapter \1/ -s/@head(1,"\(.*\)")/@node \1\n@chapter \1/ -s/@head(2,'\(.*\)')/@node \1\n@section \1/ -s/@head(2,"\(.*\)")/@node \1\n@section \1/ -s/@head(3,'\(.*\)')/@node \1\n@subsection \1/ -s/@head(3,"\(.*\)")/@node \1\n@subsection \1/ -s/@p()// -s/@select.*/@verbatim/ -s/@doc()/@end verbatim/ -s/@begin_displayed_code()/@verbatim/ -s/@end_displayed_code()/@end verbatim/ -s/@tdir =.*// -s/@execfile.*// -s/@h=// -s/@begin_table("\(.*\)","\(.*\)","\(.*\)")/@multitable @columnfractions .33 .33 .33\n@headitem \1 @tab \2 @tab \3/ -s/@begin_table("\(.*\)","\(.*\)")/@multitable @columnfractions .5 .5\n@headitem \1 @tab \2/ -s/@end_table()/@end multitable/ -s/@table_row("\(.*\)","\(.*\)","\(.*\)")/@item \1 @tab \2 @tab \3/ -s/@table_row("\(.*\)","\(.*\)")/@item \1 @tab \2/ -s/\\uXXXX/@verb{ \\uXXXX }/ -s/\\UXXXXXXXX/@verb{ \\UXXXXXXXX }/ -s/\\\\, \\', \\", \\r, \\n, \\t, \\b, \\v, \\f/@verb{ \\\\, \\', \\", \\r, \\n, \\t, \\b, \\v, \\f }/ -s/'else {}'\./@verb{.else {}.}/ -s/"{" /@verb{ { }/ -s/"}" /@verb{ } }/ -@doc() -Now phase 2, a python script. There are some caveats -on what it can handle: it generates just one menu. -@h = tangler('script/mktitut.py','python') -@select(h) -# modify an texinfo file to add a menu of all the nodes -# read from stdin, write to stout -import sys - -crap1 = """ -@@copying -This manual is for Felix version 1.1.0. -Copyright @copyright{} 2005 John Skaller -@@quotation -All rights relinquished, you can do whatever you -like with this manual. -@@end quotation -@@end copying - -@@titlepage -@@title Felix Overview -@@subtitle A quick guide to the basic ideas -@@author John Skaller -@@page -@@vskip 0pt plus 1filll -@@insertcopying -@@end titlepage -@@contents - -@@ifnottex -""" - -crap2 = """ -@@top Overview - -@@insertcopying -@@end ifnottex -""" - -cache = [] -menu = [] - -def nav(node): - for i in xrange(0,n): - if node == menu[i]: - if i == 0: prev = "Top" - else: prev = menu[i-1] - if i == n-1: next = "Top" - else: next = menu[i+1] - return next,prev - -flag = 0 -for line in sys.stdin: - if flag == 0: - if '@node ' == line[:6]: - node = line[6:-1] - flag = 1 - cache.append(line) - else: - cache.append(line) - else: - if '@chapter' == line[:8]: - #print "chapter" - menu.append(node) - elif '@section' == line[:8]: - #print "section" - menu.append(node) - elif '@subsection' == line[:11]: - #print "subsection" - menu.append(node) - else: - print "ERROR, need chapter section or subsection here" - print "got ",line, - raise "error" - cache.append('@section\n') - flag = 0 - - -n = len(menu) - -for line in cache: - if '@settitle' == line[:9]: - print '@setfilename ' + sys.argv[1] # the filename - print line, - print crap1 - print "@node Top, "+menu[0]+", "+menu[n-1]+",(dir)" - print crap2 - print '@menu' - for item in menu: - print "* " + item+":: "+item - print '@end menu' - elif '@node ' == line[:6]: - node = line[6:-1] - next,prev = nav(node) - print '@node '+node+', '+next+', '+prev+', Top' - else: - print line, -print - -@doc() -This is input to equiv-build, which makes -a dummy ocaml-3.08.3 package to satisfy -the debian package dependencies. -@h = tangler('misc/ocaml-nox-3.08.3-equiv','data') -@select(h) -Source: ocaml-nox-3.08.3 -Section: devel -Priority: optional -Maintainer: John Skaller <sk...@us...> -Standards-Version: 3.6.1 - -Package: ocaml-nox-3.08.3 -Architecture: any -Description: hack to tell system 0caml3.08.3... [truncated message content] |
From: <no...@fe...> - 2010-10-10 14:48:20
|
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "An advanced programming language". The branch, master has been updated via fc4f483d20272dbf308e40cb352558b82fc2ca0c (commit) from 801ca9addac13c0a767adb90818b2c2e3bfbcf9b (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit fc4f483d20272dbf308e40cb352558b82fc2ca0c Author: Erick Tryzelaar <ida...@us...> Date: Sun Oct 10 07:47:47 2010 -0700 Changed gitmodules to point to github. diff --git a/.gitmodules b/.gitmodules index edd7262..cac49b9 100644 --- a/.gitmodules +++ b/.gitmodules @@ -1,9 +1,9 @@ [submodule "fbuild"] path = fbuild - url = http://git.felix-lang.org/r/fbuild.git + url = git://github.com/erickt/fbuild.git [submodule "src/compiler/dypgen"] path = src/compiler/dypgen - url = http://git.felix-lang.org/r/dypgen.git + url = git://github.com/erickt/dypgen.git [submodule "src/compiler/ocs"] path = src/compiler/ocs - url = http://git.felix-lang.org/r/ocs.git + url = git://github.com/erickt/ocs.git ----------------------------------------------------------------------- Summary of changes: .gitmodules | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) hooks/post-receive -- An advanced programming language |
From: <no...@fe...> - 2010-10-08 00:07:41
|
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "An advanced programming language". The branch, master has been updated via 801ca9addac13c0a767adb90818b2c2e3bfbcf9b (commit) from 49bd05cf9e9ffe8d0b41451d93bc045465cc8e2d (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 801ca9addac13c0a767adb90818b2c2e3bfbcf9b Author: John Skaller <sk...@us...> Date: Fri Oct 8 11:06:57 2010 +1100 Fiddle lots of stuff to get the flx command line harness to work. diff --git a/buildsystem/flx.py b/buildsystem/flx.py index 49fdcc8..d789ffa 100644 --- a/buildsystem/flx.py +++ b/buildsystem/flx.py @@ -36,8 +36,8 @@ class Builder(fbuild.db.PersistentObject): buildroot = buildroot or self.ctx.buildroot src = Path(src) - # first, copy the src file into the buildroot src_buildroot = src.addroot(buildroot) + dst = src.addroot(buildroot) if preparse: @@ -154,8 +154,45 @@ class Builder(fbuild.db.PersistentObject): cxx_cflags=[], cxx_libs=[], cxx_lflags=[]): + obj = self.compile(src, includes=includes, flags=flags) + # run flx_pkgconfig to generate the include files, normally done + # by flx command line harness but we're probably building it here + + flx_pkgconfig = Path("bin/flx_pkgconfig") + flx_pkgconfig = flx_pkgconfig.addroot(self.ctx.buildroot) + config = Path("config") + config = config.addroot(self.ctx.buildroot) + cmd = [flx_pkgconfig, "--path+="+config, "--field=includes", "@"+src[:-4]+".resh"] + stdout, stderr =self.ctx.execute(cmd) + output = stdout.decode()[:-1] # strip trailing newline + includes = output.split(' ') + files = ["#include "+i+"\n" for i in includes] + fn = src[:-4]+".includes" + f = open(fn,"w") + for file in files: f.write(file) + f.close() + + return function(obj, dst, + async=async, + includes=cxx_includes, + libs=cxx_libs, + cflags=cxx_cflags, + lflags=cxx_lflags, + ) + + def _build_flx_pkgconfig_link(self, function, src, dst=None, *, + async=True, + includes=[], + flags=[], + cxx_includes=[], + cxx_cflags=[], + cxx_libs=[], + cxx_lflags=[]): + + obj = self.compile(src, includes=includes, flags=flags) + return function(obj, dst, async=async, includes=cxx_includes, @@ -170,6 +207,9 @@ class Builder(fbuild.db.PersistentObject): def build_exe(self, *args, **kwargs): return self._build_link(self.link_exe, *args, **kwargs) + def build_flx_pkgconfig_exe(self, *args, **kwargs): + return self._build_flx_pkgconfig_link(self.link_exe, *args, **kwargs) + # ------------------------------------------------------------------------------ def build(ctx, flxg, cxx, drivers): @@ -184,7 +224,7 @@ def build(ctx, flxg, cxx, drivers): ) def build_flx_pkgconfig(phase): - return phase.flx.build_exe( + return phase.flx.build_flx_pkgconfig_exe( dst='bin/flx_pkgconfig', src='src/flx_pkgconfig/flx_pkgconfig.flx', includes=[phase.ctx.buildroot / 'lib'], @@ -192,6 +232,16 @@ def build_flx_pkgconfig(phase): cxx_libs=[call('buildsystem.flx_rtl.build_runtime', phase).static], ) + +def build_flx(phase): + return phase.flx.build_exe( + dst='bin/flx', + src=Path('src/flx/flx.flx').addroot(phase.ctx.buildroot), + includes=[phase.ctx.buildroot / 'lib'], + cxx_includes=['tools', phase.ctx.buildroot / 'lib/rtl'], + cxx_libs=[call('buildsystem.flx_rtl.build_runtime', phase).static], + ) + # ------------------------------------------------------------------------------ def test_flx(phase, src, *args, **kwargs): diff --git a/fbuildroot.py b/fbuildroot.py index 2efddf4..1c37a42 100644 --- a/fbuildroot.py +++ b/fbuildroot.py @@ -386,6 +386,7 @@ def build(ctx): call('buildsystem.' + module + '.build_flx', target) flx_pkgconfig = call('buildsystem.flx.build_flx_pkgconfig', target) + flx= call('buildsystem.flx.build_flx', target) # -------------------------------------------------------------------------- # build the secondary libraries @@ -402,7 +403,7 @@ def build(ctx): # now, try building a file target.felix = call('fbuild.builders.felix.Felix', ctx, - exe=ctx.buildroot / 'bin/flx.py', + exe=ctx.buildroot / 'bin/flx', # changed by JMS to now use the Felix version debug=ctx.options.debug, flags=['--test=' + ctx.buildroot]) diff --git a/lpsrc/flx_maker.pak b/lpsrc/flx_maker.pak index ed378e1..5404421 100644 --- a/lpsrc/flx_maker.pak +++ b/lpsrc/flx_maker.pak @@ -106,6 +106,7 @@ dot=(b"\x47\x49\x46\x38\x39\x61\x09\x00" mkgif("plus",plus) mkgif("minus",plus) mkgif("dot",dot) +// @head(1,'Run script') @# ------------- RUN SCRIPT flx.bat FOR WINDOWS COMMAND LINE @@ -488,6 +489,7 @@ def xqt(cmd, default=None): if ECHO==1:print("output="+output) return output if default is None: + print("Cmd FAILED") sys.stderr.write("Error "+repr(result)+" executing command " + cmd+"\n") if not hasattr(os, 'WEXITSTATUS'): sys.exit(1) @@ -677,8 +679,8 @@ else: sys.exit(0!=system(cmd)) sys.exit(0!=result) -@# ------------- UNIX RUN SCRIPT flx REQUIRES bash -@select(tangler('bin/flx','data')) +@# ------------- UNIX RUN SCRIPT flx.sh REQUIRES bash +@select(tangler('bin/flx.sh','data')) #!/usr/bin/env bash # flx - felix script harness @tangle("INSTALL_ROOT="+config.PREFIX+"/lib/felix/felix-"+config.flx_version) @@ -1228,9 +1230,9 @@ fi @# ------------- a native shell script to drive it (but now leaving the binary to @# ------------- do most of the command line argument processing etc -@select(tangler('tools/flx.flx','data')) +@select(tangler('src/flx/flx.flx','data')) -dbug := true; // switch off for production +dbug := false; // switch off for production False := false; True := true; @@ -1238,7 +1240,8 @@ True := true; @def tv(x): ts("var "+x) -@tv("INSTALL_ROOT=Filename::join(%r, '/lib/felix/felix-%s')" % (config.PREFIX, config.flx_version)) +@tv("INSTALL_ROOT_TOPDIR=Filename::join(%r, 'lib/felix')" % (config.PREFIX)) +@tv("INSTALL_ROOT=Filename::join(INSTALL_ROOT_TOPDIR, 'felix-%s')" % (config.flx_version)) var FLX_INSTALL_DIR = Env::getenv("FLX_INSTALL_DIR", INSTALL_ROOT); @tv("CYGWIN="+str(config.CYGWIN)) @tv("WIN32="+str(config.WIN32)) @@ -1328,6 +1331,7 @@ fun prefix(arg:string,key:string)=> var compile_exts = List::list ('cpp','cxx'); var linkexts = List::list ('o','obj','lib','dll','a','so'); var arg = ""; +var result = 0; whilst grab == 1 and argno<System::argc do arg = System::argv argno; @@ -1336,6 +1340,8 @@ whilst grab == 1 and argno<System::argc do var dir,base = Filename::split1(path); dbug?? println$ "path="+path+", ext="+ext+",dir="+dir+",base="+base; if ext != "flx" and ext != "" do + // add to list of things to link, and also things to compile + // if the extension is appropriate if List::mem eq of (string * string) compile_exts ext do cpps = cpps + " " + arg; cppos = cppos + " " + path + "." + EXT_OBJ; @@ -1345,6 +1351,9 @@ whilst grab == 1 and argno<System::argc do elif arg == "--nostdimport" do dbug?? println "No standard library import"; + // Note: currently, Felix compiler generates code that REQUIRES + // the standard library, eg the driver passes a gc_profile_t record + // and the compiler generates _uctor_ objects, etc etc STDIMPORT=""; elif prefix(arg,"--import=") do @@ -1361,16 +1370,44 @@ whilst grab == 1 and argno<System::argc do TESTMODE=1; FLX_INSTALL_DIR="."; + elif arg=="--install" do + dbug?? println "Intall Felix"; + println "Install Felix: ONLY ON UNIX (you may need to be superuser)"; + println "Always installs the --test directory to the configured install target"; + println "Because that is hard coded into this program"; + println "Note: does NOT install this program 'flx' into your PATH!"; + println$ "FROM: " + FLX_INSTALL_DIR; + println$ "TO : " + INSTALL_ROOT; + if FLX_INSTALL_DIR == INSTALL_ROOT do + println "Can't install, src and dst are the same"; + System::exit(1); + else + result=System::system("mkdir -pv "+INSTALL_ROOT_TOPDIR); + if result != 0 do + println$ "Cannot create directory " + INSTALL_ROOT_TOPDIR; + System::exit 1; + done + result=System::system("cp -r "+FLX_INSTALL_DIR+" "+INSTALL_ROOT); + if result == 0 do println "Install succeeded" + else println$ "Install failed, code = " + str(result); + done + System::exit(result); + done + elif prefix(arg,"--stdout=") do dbug?? println "Redirect standard output"; + // of the Felix program only: used for saving the output + // to a file so the test harness can compare it with an .expect file STDOUT=arg.[9 to]; elif arg=="--force" do dbug?? println "Force recompilation"; + // of the felix code, runs Felix unless --nofelix is set + // the C++ compiler is run unless the felix compile failed RECOMPILE=1; elif arg=="--debug" do - dbug?? println "Enable debugging"; + dbug?? println "Enable runtime debugging"; DEBUG=1; elif arg=="--debug-compiler" do @@ -1508,7 +1545,7 @@ if ECHO == 1 do done CONFIG_DIR = Filename::join(FLX_INSTALL_DIR,'config'); -dbug?? println$ "Felix package manager config directory is "+PKGCONFIG; +dbug?? println$ "Felix package manager config directory is "+CONFIG_DIR; // make a list of any *.cpp files (or other g++ options ..) var EXT_OBJ = @@ -1549,6 +1586,7 @@ gen get_stdout(x:string) = { if os.name == "nt": // popen doesn't work on Windows */ result := system(x + " >tmp.out"); + Stdout::flush(); output := Text_file::load "tmp.out"; /* else: @@ -1565,7 +1603,7 @@ gen get_stdout(x:string) = { } gen xqt(cmd:string) = { - ECHO == 1 ?? print(cmd); + ECHO == 1 or dbug ?? println("cmd="+cmd); var result,output = get_stdout(cmd); if result == 0 do n := @@ -1575,8 +1613,9 @@ gen xqt(cmd:string) = { endmatch ; output = output.[to n]; // first line excluding newline - ECHO==1 ?? println("output="+output); + ECHO==1 or dbug ?? println("output='"+output+"'"); else + dbug ?? println ("COMMAND FAILED"); fprint$ cerr, ("Error "+repr(result)+" executing command " + cmd + "/n"); System::exit result; done @@ -1763,7 +1802,6 @@ done var FLXFLAGS="--inline="+str(INLINE) + ' ' + str(OUTPUT_DIR); -var result = 0; var FCMD=""; var LCMD=""; var CCMD=""; @@ -2064,7 +2102,7 @@ post-install: @head(1,'Finish up') Just cleaning up script now. -@make_executable(os.path.join('bin', 'flx')) +@make_executable(os.path.join('bin', 'flx.sh')) @make_executable(os.path.join('bin', 'flx.py')) diff --git a/lpsrc/flx_stdlib.pak b/lpsrc/flx_stdlib.pak index 75c4b38..53d8020 100644 --- a/lpsrc/flx_stdlib.pak +++ b/lpsrc/flx_stdlib.pak @@ -269,9 +269,9 @@ module Filename # Felix then reduces it to \\ !!! # and finally C reduces it to plain \ !!! # don't remove the !!! because \ at eol is continuation .. argg - tangle(' const sep : charp = c"\\"\\\\\\\\\\"";') + tangle(' val sep = "\\\\\\\\"') else: - tangle(' const sep : charp = c"\\"'+os.sep+'\\"";') + tangle(' val sep = "'+os.sep+'";') @# // these seem more generic @@ -285,7 +285,7 @@ module Filename // since split pulls components off from the RHS we have to // fold them back from the left - fun split1(s:string)=> match find_last_of(s,sep) with + fun split1(s:string)=> match find_last_of(s,char sep) with | Some ?pos => if pos==0 then str sep else s.[to pos] endif, s.[pos+1 to] diff --git a/src/compiler/drivers/flxg.ml b/src/compiler/drivers/flxg.ml index 7556229..4af052d 100644 --- a/src/compiler/drivers/flxg.ml +++ b/src/compiler/drivers/flxg.ml @@ -449,14 +449,23 @@ let codegen_bsyms state bsym_table root_proc = plh ("//Timestamp: " ^ state.compile_start_gm_string); plh ("//Timestamp: " ^ state.compile_start_local_string); plh ""; + + (* THE PRESENCE OF THESE LINES IS A BUG .. defeats the FLX_NO_INCLUDES switch + * but we need this temporarily, at least until we fix flx.flx compilation + * in the build system to generate dependencies in the flx.includes file + * + * Also we need to fix the compiler and all libraries to use explicit + * qualification everywhere which will make the code a bit of a mess.. + *) + plh "//FELIX RUNTIME"; + plh "#include \"flx_rtl.hpp\""; + plh "using namespace ::flx::rtl;"; + plh "#include \"flx_gc.hpp\""; + plh "using namespace ::flx::gc::generic;"; + plh "#ifndef FLX_NO_INCLUDES"; plh ("#include \"" ^ state.module_name ^ ".includes\""); plh "#endif"; - plh "//FELIX RUNTIME"; - (* plh "#include \"flx_rtl.hpp\""; *) - plh "using namespace ::flx::rtl;"; - (* plh "#include \"flx_gc.hpp\""; *) - plh "using namespace ::flx::gc::generic;"; plh ""; plh "\n//-----------------------------------------"; diff --git a/src/lib/std/env.flx b/src/lib/std/env.flx index 66fa71a..fa5f4ec 100644 --- a/src/lib/std/env.flx +++ b/src/lib/std/env.flx @@ -1,7 +1,7 @@ module Env { fun getenv:string -> string = - "::flx::rtl::strutil::atostr(getenv($1.data()))" + "::flx::rtl::strutil::atostr(std::getenv($1.data()))" requires flx_strutil, cstdlib; fun getenv(name:string,dflt:string)=>let ?result = getenv(name) in ----------------------------------------------------------------------- Summary of changes: buildsystem/flx.py | 54 ++++++++++++++++++++++++++++++++++++- fbuildroot.py | 3 +- lpsrc/flx_maker.pak | 60 ++++++++++++++++++++++++++++++++++------- lpsrc/flx_stdlib.pak | 6 ++-- src/compiler/drivers/flxg.ml | 19 ++++++++++--- src/lib/std/env.flx | 2 +- 6 files changed, 121 insertions(+), 23 deletions(-) hooks/post-receive -- An advanced programming language |
From: <no...@fe...> - 2010-10-07 05:26:06
|
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "An advanced programming language". The branch, master has been updated via 49bd05cf9e9ffe8d0b41451d93bc045465cc8e2d (commit) from 8b7a4f79a2af14c2a4e0a27f157889416bc6e779 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 49bd05cf9e9ffe8d0b41451d93bc045465cc8e2d Author: John Skaller <sk...@us...> Date: Thu Oct 7 16:25:22 2010 +1100 Get rid of the old fbuild.. not maintained anymore so useless. diff --git a/fbuild_old/bin/fbuild b/fbuild_old/bin/fbuild deleted file mode 100755 index 57676ab..0000000 --- a/fbuild_old/bin/fbuild +++ /dev/null @@ -1,489 +0,0 @@ -#!/usr/bin/env python -############################################################### -# FELIX MAKE SCRIPT -############################################################### -import os -import sys -import glob -import shutil -import imp - -# forward import of set -try: - set -except NameError: - from sets import Set as set - -sys.path.append( - os.path.join(os.path.dirname(sys.argv[0]), '..', 'lib') -) - -os.environ['PYTHONPATH'] = os.pathsep.join([ - os.path.join(os.path.dirname(sys.argv[0]), '..', 'lib'), - os.environ.get('PYTHONPATH', ''), -]) - -if '' not in sys.path: sys.path.insert(0, '') - -import fbuild.flxbuild.flxutil -import fbuild.flxbuild.package -import fbuild.flxbuild.process - -# ------------------------------------------------ -# Load the initial config -# ------------------------------------------------ - -try: - import config -except Exception: - import traceback - xt,xv,tb = sys.exc_info() - print "ERROR IN config/__init__.py" - traceback.print_exception(xt,xv,tb) - print "You must either" - print "(a) edit config/__init__.py and fix the error, or," - print "(b) i) delete it, and," - print " ii) run 'python script/make_config.py'" - print " again to reset it:" - print " this is done automatically by 'make boot'" - sys.exit(1) - -# ------------------------------------------------ -# PROCESS COMMAND LINE OPTIONS -# ------------------------------------------------ - -import fbuildroot_old as fbuildroot - -def load_options(): - from optparse import OptionParser, make_option - - parser = OptionParser() - parser.add_options([ - make_option('-v', '--verbose', - action='count', - default=1, - help='print out extra debugging info'), - make_option('-q', '--quiet', - action='count', - default=0, - help='do not print out extra debugging info'), - make_option('--force', - action='store_true', - default=False, - help='force running a process'), - make_option('--phase', dest='selected_phases', metavar='PHASE', - action='append', - default=[], - help='run only this phase of the build'), - make_option('--pkg', dest='selected_packages', metavar='PKG', - action='append', - default=[], - help='build only these packages'), - make_option('--model', dest='selected_models', metavar='MODEL', - action='append', - default=[], - help='build only the phases which can run on this platform'), - make_option('--lparchive', - metavar='PATH', - help='use an alternative lpsrc directory'), - ]) - - try: - extra_options = fbuildroot.options - except AttributeError: - pass - else: - option_group = parser.add_option_group('project options') - option_group.add_options(extra_options) - - options, args = parser.parse_args() - - #### - - # temporary hack to set up the shell - config.HOST_OCAML.verbose = options.verbose - config.HOST_CC.verbose = options.verbose - config.HOST_CXX.verbose = options.verbose - config.TARGET_CC.verbose = options.verbose - config.TARGET_CXX.verbose = options.verbose - - config.HOST_OCAML.quiet = options.quiet - config.HOST_CC.quiet = options.quiet - config.HOST_CXX.quiet = options.quiet - config.TARGET_CC.quiet = options.quiet - config.TARGET_CXX.quiet = options.quiet - - #### - - #### - - # if the user didn't say which phases to run - # then run all the phases that use the host model - # On building on Linux for MinGW, the build model - # is linux, and the host model is mingw, the target - # is win32. We'd be running all MinGW phases. - # note a phase is named for the targetted product - # NOT the machine that generates it. Thus host - # phase mingw means 'compile on Linux for MingW' - - if not options.selected_phases and not options.selected_models: - options.selected_models = [config.host_model] - - for model in options.selected_models: - if model == config.build_model and 'build' not in options.selected_phases: - options.selected_phases.append('build') - - if model == config.host_model and 'host' not in options.selected_phases: - options.selected_phases.append('host') - - if model == config.target_model and 'target' not in options.selected_phases: - options.selected_phases.append('target') - - if model == config.run_model and 'run' not in options.selected_phases: - options.selected_phases.append('run') - - #### - - if options.lparchive: - config.FLX_LPARCHIVE = options.lparchive - - return options, args - - -def initial_extraction(options, args): - if "extract" not in args and "force_extract" not in args: - return - - paks = glob.glob(os.path.join(config.FLX_LPARCHIVE, "lpsrc", "*.pak")) - - for p in paks: - print "EXTRACTING", p, "from", config.FLX_LPARCHIVE - if "force_extract" in args: - cmd = (config.ISCR, '--force', '--break-on-error', p) - else: - cmd = (config.ISCR, '--break-on-error', p) - - fbuild.flxbuild.flxutil.execute(cmd, - verbose=options.verbose, - quiet=options.quiet, - ) - - # this is a hack - fbuild.flxbuild.flxutil.mkdirs('speed') - shutil.copy(os.path.join('misc', 'interscript.css'), 'speed') - - print "EXTRACTION COMPLETE" - sys.exit(0) - -# ---------------------------------------------------------------------------- - -def load_packages(): - pkgs = [] - unsorted_pkgs = [] - - for i in glob.glob(os.path.join(config.src_dir, "cpkgs", "target", "*.py")): - shutil.copy(i, os.path.join('cpkgs', 'target')) - - for i in \ - glob.glob(os.path.join("spkgs", "*.py")) + \ - glob.glob(os.path.join(config.src_dir, "spkgs", "*.py")): - pkg = os.path.basename(os.path.splitext(i)[0]) - if pkg == '__init__': - continue - - f, filename, description = imp.find_module(pkg, [os.path.dirname(i)]) - try: - module = imp.load_module(pkg, f, filename, description) - finally: - if f: - f.close() - - unsorted_pkgs.append(pkg) - d = {} - for k, v in module.__dict__.iteritems(): - if k[0] != '_': - d[k] = v - fbuild.flxbuild.package.pkgd[pkg] = d - - def addpkg_r(pkg): - if pkg not in pkgs: - if pkg not in fbuild.flxbuild.package.pkgd.keys(): - print "Unknown package", pkg - print "Please extract!" - sys.exit(1) - else: - reqs = fbuild.flxbuild.package.pkgd[pkg].get('pkg_requires',[]) - fbuild.flxbuild.package.pkgreqs[pkg]=reqs - for i in reqs: - addpkg_r(i) - pkgs.append(pkg) - - for pkg in unsorted_pkgs: - addpkg_r(pkg) - - # invert the requirements in order to determine the dependencies - fbuild.flxbuild.package.pkgdeps.update( - fbuild.flxbuild.flxutil.invert(fbuild.flxbuild.package.pkgreqs)) - - return pkgs - -# ------------------------------------------------- -# LOAD PROCESSES - - -def load_process(process_path, options, args): - p = process_path.split('.') - module, name = '.'.join(p[:-1]), p[-1] - process = getattr(__import__(module, {}, {}, ['']), name) - - if not callable(process): - raise ImportError - - if type(process) is type: - process = process( - verbose=options.verbose, - quiet=options.quiet, - optimise=options.optimise, - debug=options.debug, - force=options.force or 'force' in args, - options=args, - ) - - return process - - -def load_processes(options, args): - for phase, processes in fbuildroot.fbuild_processes.iteritems(): - for process_path in processes: - process = load_process(process_path, options, args) - fbuild.flxbuild.process.processes[process_path] = process - -# ----------------------------------------------------------------------------- - -def calculate_packages(options, phase, packages): - if options.selected_packages: - selected_pkgs = options.selected_packages - else: - selected_pkgs = [] - max_len = max([len(s) for s in fbuild.flxbuild.package.pkgd]) - - for pkg, pkgdict in fbuild.flxbuild.package.pkgd.iteritems(): - latest_src_time = fbuild.flxbuild.process.get_latest_src_time(pkgdict) - - stamp = os.path.join('pkg-stamps', '%s.%s' % (pkg, phase)) - latest_build_time = fbuild.flxbuild.flxutil.filetime(stamp) - - if latest_build_time == 0: - print "Pak %s (virtual)" % pkg.ljust(max_len), 'UNBUILT' - selected_pkgs.append(pkg) - elif latest_build_time < latest_src_time: - print "Pak %s changed: %s" % ( - pkg.ljust(max_len), - fbuild.flxbuild.flxutil.fmtime(latest_src_time), - ) - selected_pkgs.append(pkg) - - return [pkg for pkg in packages if pkg in selected_pkgs] - -#### - -def run_phase(options, phase, packages): - print '-' * 78 - print '***** MAKING PHASE', phase - pkgsummary = {} - for pkg in calculate_packages(options, phase, packages): - pkgdict = fbuild.flxbuild.package.pkgd[pkg] - print "*****", phase, "MAKING PACKAGE", pkg, "************" - stamp = os.path.join("pkg-stamps", "%s.%s" % (pkg, phase)) - if os.path.exists(stamp): - os.remove(stamp) - - for process in fbuildroot.fbuild_processes.get(phase, []): - #print "*****", phase, "RUNNING PROCESS", process, "************" - result = fbuild.flxbuild.process.processes[process](pkg, pkgdict, pkgsummary) - if not result and result is not None: - break - else: - fbuild.flxbuild.process.enstamp(stamp, options.quiet) - - if pkgsummary: - pkgsummary = pkgsummary.items() - pkgsummary.sort() - print phase, "PHASE SUMMARY" - for (package, process), summary in pkgsummary: - if summary != "no tests": - print ' ', process.ljust(25), ':', package.ljust(20), ':', summary - -# ----------------------------------------------------------------------------- - -def print_failures(): - # print out all the failed processes - failure_log = fbuild.flxbuild.flxutil.Tee() - - failure_log.write("----- PROCESS STATUS -----\n") - failed = 0 - fatal = 0 - for name, process in fbuild.flxbuild.process.processes.iteritems(): - try: - failures = process.failures - successes = process.successes - dummy = process.dummy - used = process.used - except AttributeError: - continue - - nfail = len(failures) - npass = len(successes) - failed = failed or nfail > 0 - fatal = fatal or (nfail > 0 and not dummy) - - if nfail: - print >> failure_log, 'PROCESS **FAILED**:', name, nfail, '/', npass + nfail, 'failures' - if dummy: - print >> failure_log, " [Expected failure, doesn't break build]" - else: - if used: - print >> failure_log, 'PROCESS PASSED :', name, npass, 'passed' - else: - pass - #print >> failure_log, 'PROCESS UNUSED :', name - - if not failed: - return False - - print '^^^^^^^^^^ FAILURES by group ^^^^^^^^^^^^^' - - kats = {} - for name, process in fbuild.flxbuild.process.processes.iteritems(): - try: - failures = process.failures - except AttributeError: - continue - - if not failures: - continue - - print '- %s: %s' % (len(failures), name) - - lookup = {} - - for pkg, file, failure in failures: - files = lookup.get(pkg, []) - files.append(file) - lookup[pkg] = files - - lookup = lookup.items() - lookup.sort() - - for pkg, files in lookup: - files.sort() - - print ' - %s: %s' % (len(files), pkg) - lastfile = "" - for file in files: - if file == lastfile: continue - lastfile = file - # grab the first line of the file to see if it - # has a #line number to print out as well, so we can - # localize the error to the interscript file location - - emitted = 0 - f = open(file) - eat = 1 - while eat == 1: - try: - line = f.readline() - except: - eat = 0 - if emitted == 0: fstring = file - else: fstring = "" - if emitted == 0 and line[0:5] == '#line': - comment = line - elif line [0:8] == "//Check ": - kat = line[8:].strip() - l = kats.get(kat,[]) - l.append(file) - kats[kat]=l - comment = kat - else: - if emitted == 0: - comment = line - else: - comment = "" - eat = 0 - if line != "": - print ' %-35s %s' % (fstring, comment.strip()) - emitted = 1 - if emitted == 0: - print ' %s' % file - f.close() - - print '^^^^^^^^^^ FAILURES by category ^^^^^^^^^^' - keys = kats.keys() - keys.sort() - for kat in keys: - print kat+":" - files = kats[kat] - for file in files: - print " ",file - - f = open('errors.log', 'w') - try: - f.write(failure_log.getvalue()) - finally: - f.close() - - return fatal - -# ----------------------------------------------------------------------------- - -def main(): - # add the felix directory to the search paths - sys.path.append(config.src_dir) - os.environ['PYTHONPATH'] = \ - config.src_dir + os.pathsep + os.environ.get('PYTHONPATH', '') - - options, args = load_options() - initial_extraction(options, args) - packages = load_packages() - load_processes(options, args) - - #### - - # run all the pre-processes - for process in fbuildroot.fbuild_preprocesses: - load_process(process, options, args).preprocess() - - # execute user selected processes in command line order - for arg in args: - # check in the build system stock process set - # these are run independently of the selected phase - if arg in fbuild.flxbuild.process.processes: - fbuild.flxbuild.process.processes[arg].runme() - else: - # try seeing if it's a process - try: - process = load_process(arg, options, args) - except ImportError: - pass - else: - process.runme() - - else: - # otherwise run all the processes specified by the application - # for the selected phases, in the application specified phase order - for phase in options.selected_phases: - run_phase(options, phase, packages) - - #### - - if print_failures(): - print "********* BUILD FAILED ********************" - return 1 - else: - print "*********** RUN COMPLETE: NO UNEXPECTED FAILURES DETECTED *******" - - return 0 - -if __name__ == '__main__': - sys.exit(main()) diff --git a/fbuild_old/bin/make_config.py b/fbuild_old/bin/make_config.py deleted file mode 100755 index 67e3bb7..0000000 --- a/fbuild_old/bin/make_config.py +++ /dev/null @@ -1,600 +0,0 @@ -#!/usr/bin/env python - -import os -import sys -import time -import glob -from optparse import OptionParser, make_option -import shutil - -sys.path.append( - os.path.join(os.path.dirname(sys.argv[0]), '..', 'lib') -) - -os.environ['PYTHONPATH'] = os.pathsep.join([ - os.path.join(os.path.dirname(sys.argv[0]), '..', 'lib'), - os.environ.get('PYTHONPATH', ''), -]) - -from fbuild.flxbuild.flxutil import xqt, ExecutionError -from fbuild.flxbuild.config_support import pr, pa - -import version - -# ----------------------------------------------------------------------------- - -# supported platforms - -platforms = [ - "posix", - "cygwin", - "nocygwin", - "mingw", - "win32", - "win64", - "osx", - "detect" - "solaris", - "bsd", - "linux" - ] - -# map other names for them, obtained from -# various place like os.platform, os.system("mname -u"), etc - -archmap = { - "irix":"posix", - "irix64":"posix", - "unix":"posix", - "posix":"posix", - "linux":"linux", - "gnu/linux":"linux", - "solaris":"solaris", - "sunos":"solaris", - "cygwin":"cygwin", - "nocygwin":"nocygwin", - "mingw":"mingw", - "windows":"win32", - "nt":"win32", - "win32":"win32", - "win64":"win64", - "darwin":"osx", - "freebsd":"bsd", - "netbsd":"bsd", - "openbsd":"bsd", - "osx":"osx", - "detect":"detect" - } - -ALL_PHASES = ["build", "host", "target", "run"] - -# ----------------------------------------------------------------------------- - -def check_model(m): - try: - m = archmap[m] - except KeyError: - print "Unknown model '"+m+"' please choose one of:" - for m in platforms: print " * " + m - sys.exit(1) - return m - -def load_options(): - parser = OptionParser() - parser.add_options([ - make_option('-v', '--verbose', - action='count', - default=0, - help='print out extra debugging info'), - make_option('-q', '--quiet', - dest='verbose', - action='store_const', - const=0, - help='do not print out extra debugging info'), - make_option('--prefix', - help='install into this prefixed directory', - default=os.environ.get('PREFIX', '/usr/local')), - make_option('-I', '--include-path', - metavar='INCLUDE_PATH', - dest='include_paths', - action='append', - help='additionally search these paths for headers'), - make_option('-L', '--lib-path', - metavar='LIB_PATH', - dest='lib_paths', - action='append', - help='additionally search these paths for libraries'), - make_option('--build', - dest='build_model', - help='specify the build model'), - make_option('--host', - dest='host_model', - help='specify the host model'), - make_option('--target', - dest='target_model', - help='specify the target model'), - make_option('--run', - dest='run_model', - help='specify the run model'), - make_option('--buildcc', - metavar='CC', - help='specify the build c compiler'), - make_option('--hostcc', - metavar='CC', - help='specify the host c compiler'), - make_option('--targetcc', - metavar='CC', - help='specify the target c compiler'), - make_option('--buildcxx', - metavar='CXX', - help='specify the build c++ compiler'), - make_option('--hostcxx', - metavar='CXX', - help='specify the host c++ compiler'), - make_option('--targetcxx', - metavar='CXX', - help='specify the target c++ compiler'), - make_option('--boot', - dest='bootfile', - help='add a config bootfile for additional config modification'), - make_option('--src-dir', - metavar='PATH', - default=os.environ.get("SRC_DIR", os.curdir), - help='specify the source directory'), - make_option('--lparchive', - metavar='PATH', - default=os.environ.get("FLX_LPARCHIVE", os.curdir), - help='specify the location of the interscript files'), - make_option('--phase', - dest='phases', - action='append', - default=[], - help='specify which phases to configure'), - ]) - - options, args = parser.parse_args() - - if not options.verbose: - options.quiet = 2 - else: - options.quiet = 0 - - if options.build_model: - print "Specified build model", options.build_model - options.build_model = check_model(options.build_model) - else: - # attempt to find the Felix name for the build OS - # using uname -s, or, if that fails, Python os.name - # if the final result isn't a name we recognize - # set the build_model to 'detect' to indicate C level - # testing is to be used. Note that these C tests are - # done anyhow, and may verify, refine, or otherwise - # munge this result .. however we need some initial - # indication HOW to perform these tests. - - try: - output = xqt('uname', '-s') - except ExecutionError: - try: - options.build_model = archmap[os.name] - except KeyError: - print "uname -s and Python returns unknown OS type, assuming 'detect'" - options.build_model = "detect" - else: - output = output[0].strip().lower() - options.build_model = archmap[output] - - # attempt to find the Felix name for the build OS - # using uname -s, or, if that fails, Python os.name - # if the final result isn't a name we recognize - # set the build_model to 'detect' to indicate C level - # testing is to be used. Note that these C tests are - # done anyhow, and may verify, refine, or otherwise - # munge this result .. however we need some initial - # indication HOW to perform these tests. - - try: - output = xqt('uname', '-s') - except ExecutionError: - try: - options.build_model = archmap[os.name] - except KeyError: - print "uname -s and Python returns unknown OS type, assuming 'detect'" - options.build_model = "detect" - else: - output = output[0].strip().lower() - options.build_model = archmap[output] - - print "Build platform: " + options.build_model - - if options.host_model: - print "Specified host model", options.host_model - options.host_model = check_model(options.host_model) - - if options.target_model: - print "Specified target model", options.target_model - options.target_model = check_model(options.target_model) - - if options.run_model: - print "Specified run model", options.run_model - options.run_model = check_model(options.run_model) - - for phase in options.phases: - if phase not in ALL_PHASES: - print "UNKNOWN PHASE", phase,"not in", ALL_PHASES - sys.exit(1) - - if not options.phases: - options.phases = ALL_PHASES - - if options.bootfile: - try: - execfile(options.bootfile) - print "Loaded", options.bootfile - except: - print "Cannot execute specified bootstrap file: ", options.bootfile - sys.exit(1) - - return options, args - -#--------------------------------------------------- -# Discover C/C++ compilers, linker, and other 'binutils' - -import fbuild.flxbuild.ocaml_class -import fbuild.flxbuild.gcc_class -import fbuild.flxbuild.gxx_class -import fbuild.flxbuild.msvcc_class -import fbuild.flxbuild.msvcxx_class - -# -# Detect the native build model -# -# This model has two uses: first, to build any build time -# tools needed to assist in generating the sources -# and second, to aid in selecting the options to cross-compile -# for the chosen host and target -# - -def load_compiler(options, compiler, cc, input_cc, phase, model, suffix): - CC = compiler(verbose=options.verbose, quiet=options.quiet) - - CC.set_options( - COM=cc, - include_paths=options.include_paths, - lib_paths=options.lib_paths, - use=phase, - model=model, - build=options.build_model, - ) - - if input_cc and ( - input_cc.options.COM == CC.options.COM and - input_cc.options.model == CC.options.model and - input_cc.options.build == CC.options.build): - print 'using the', input_cc.options.use, 'CC options for the', phase, 'CC' - CC.load_options(os.path.join('config', input_cc.options.use + suffix)) - CC.options.use = phase - else: - CC.check_options() - CC.report_config() - CC.save_options(os.path.join('config', phase + suffix)) - - return CC - - - -def load_compilers(options, phase, model, cc, cxx, - input_cc=None, - input_cxx=None): - print - print "++++++++++++++++++++++++++++++++++++++++++++++++++++++++" - print "Checking", phase, "MODEL", model - print "++++++++++++++++++++++++++++++++++++++++++++++++++++++++" - print - - if model in ["win32","win64"]: - CC = fbuild.flxbuild.msvcc_class.msvcc - CXX = fbuild.flxbuild.msvcxx_class.msvcxx - else: - CC = fbuild.flxbuild.gcc_class.gcc - CXX = fbuild.flxbuild.gxx_class.gxx - - CC = load_compiler(options, CC, cc, input_cc, phase, model, '_cc.py') - CXX = load_compiler(options, CXX, cxx, input_cxx, phase, model, '_cxx.py') - - return CC, CXX - -# ----------------------------------------------------------------------------- - -def configure_build(options): - BUILD_CC, BUILD_CXX = load_compilers(options, - phase='build', - model=options.build_model, - cc=options.buildcc, - cxx=options.buildcxx, - ) - - BUILD_CC.check_options() - BUILD_CC.report_config() - BUILD_CC.save_options("config/build_cc.py") - options.build_model = BUILD_CC.options.model - - BUILD_CXX.check_options() - BUILD_CXX.report_config() - BUILD_CXX.save_options("config/build_cxx.py") - options.build_model = BUILD_CXX.options.model - - cpkgs = glob.glob("cpkgs"+os.sep+"build"+os.sep+"*.py") - for cpkgf in cpkgs: - cpkg = os.path.splitext(os.path.basename(cpkgf))[0] - print "build CONFIGURING", cpkg - __import__('cpkgs.build.' + cpkg) - - return BUILD_CC, BUILD_CXX - - -def configure_host(options, BUILD_CC, BUILD_CXX): - # - # Now create the host model: the compiler has to run - # on the build machine, but can cross compile for - # the host (if so, we can build but not test Felix) - # - # Cross compilation of the host tools may prevent any - # testing of the tools - # - - if not options.host_model: - options.host_model = options.build_model - print "Defaulting host model to build model:", options.host_model - - HOST_CC, HOST_CXX = load_compilers(options, - phase='host', - model=options.host_model, - cc=options.hostcc, - cxx=options.hostcxx, - input_cc=BUILD_CC, - input_cxx=BUILD_CXX, - ) - - HOST_OCAML = fbuild.flxbuild.ocaml_class.ocaml(verbose=options.verbose, quiet=options.quiet) - - camllinkopts = "" - if HOST_CXX.options.CYGWIN: - camllinkopts = '-ccopt "-Wl,--stack -Wl,10485760" ' - - HOST_OCAML.autodetect(camllinkopts) - HOST_OCAML.report_config() - HOST_OCAML.save_options("config/ocaml_config.py") - - options.host_model = HOST_CXX.options.model - - cpkgs = glob.glob("cpkgs"+os.sep+"host"+os.sep+"*.py") - for cpkgf in cpkgs: - cpkg = os.path.splitext(os.path.basename(cpkgf))[0] - print "host CONFIGURING", cpkg - __import__('cpkgs.host.' + cpkg) - - return HOST_CC, HOST_CXX, HOST_OCAML - - -def configure_target(options, HOST_CC, HOST_CXX): - # - # Now create the target model: the compiler has to run - # on the build machine, but can cross compile for - # the target - # - # cross compilation of C++ generated by Felix allows us to - # check the generated code compiles, but not that it runs - # [but the output is largely portable so we can still try] - # - - if not options.target_model: - options.target_model = options.host_model - print "Defaulting target model to host model:", options.target_model - - TARGET_CC, TARGET_CXX = load_compilers(options, - phase='target', - model=options.target_model, - cc=options.targetcc, - cxx=options.targetcxx, - input_cc=HOST_CC, - input_cxx=HOST_CXX, - ) - - options.target_model = TARGET_CXX.options.model - - return TARGET_CC, TARGET_CXX - - -def configure_run(options): - if not options.run_model: - options.run_model = options.target_model - print "Defaulting run model to target model:", options.run_model - -# ----------------------------------------------------------------------------- - -def write_config(options, CONFIG_TIME, - BUILD_CC, BUILD_CXX, - HOST_CC, HOST_CXX, HOST_OCAML, - TARGET_CC, TARGET_CXX): - try: - print "Writing main config file" - f = open("config"+os.sep+"__init__.py","w") - pr(f,'import sys') - pr(f,"if '' not in sys.path: sys.path = [''] + sys.path") - pr(f,'#'+CONFIG_TIME) - pr(f,"CONFIG_TIME = " + repr(CONFIG_TIME)) - pr(f,"flx_version = " + repr(version.flx_version)) - pr(f,"flx_version_major = " + repr(version.flx_version_major)) - pr(f,"godi_revision = " + repr(version.godi_revision)) - pr(f,"debian_revision = " + repr(version.debian_revision)) - if options.bootfile: - pr(f,"try:") - pr(f," execfile('config/config_bootstrap.py')") - pr(f,"except: pass") - pr(f,"import fbuild.flxbuild.gcc_class") - pr(f,"import fbuild.flxbuild.msvcc_class") - pr(f,"import fbuild.flxbuild.gxx_class") - pr(f,"import fbuild.flxbuild.msvcxx_class") - pr(f,"import fbuild.flxbuild.ocaml_class") - pr(f,"from fbuild.flxbuild.config_support import *") - - pr(f,"") - pr(f,"#User configurable section") - pr(f,"SUPPORT_DYNAMIC_LOADING = " + repr(TARGET_CXX.options.SUPPORT_DYNAMIC_LOADING)) - pr(f,"SUPPORT_STATIC_LINKAGE = 1") - if TARGET_CXX.options.SUPPORT_DYNAMIC_LOADING: - pr(f,"DEFAULT_LINK_MODEL = 'dynamic'") - else: - pr(f,"DEFAULT_LINK_MODEL = 'static'") - pr(f,"build_model = " + repr(options.build_model)) - pr(f,"host_model = " + repr(options.host_model)) - pr(f,"target_model = " + repr(options.target_model)) - pr(f,"run_model = " + repr(options.run_model)) - - # target model switches - pr(f,"CYGWIN = " + repr(TARGET_CXX.options.CYGWIN)) - pr(f,"MACOSX = " + repr(TARGET_CXX.options.MACOSX)) - pr(f,"WIN32 = " + repr(TARGET_CXX.options.WIN32)) - pr(f,"WIN64 = " + repr(TARGET_CXX.options.WIN64)) - pr(f,"POSIX = " + repr(TARGET_CXX.options.POSIX)) - pr(f,"SOLARIS = " + repr(TARGET_CXX.options.SOLARIS)) - pr(f,"BSD = " + repr(TARGET_CXX.options.BSD)) - pr(f,"LINUX = " + repr(TARGET_CXX.options.LINUX)) - pr(f,"PREFIX = " + repr(options.prefix)) - pr(f,"src_dir = " + repr(options.src_dir)) - pr(f,"FLX_LPARCHIVE = " + repr(options.lparchive)) - pr(f,"FLX_RTL_DIR = " + repr(os.path.join('lib', 'rtl'))) - pr(f,"FLX_HOST_CONFIG_DIR = " + repr(os.path.join('config', 'host'))) - pr(f,"FLX_TARGET_CONFIG_DIR = " + repr(os.path.join('config', 'target'))) - pr(f,"") - - pr(f,"HOST_OCAML = fbuild.flxbuild.ocaml_class.ocaml()") - pr(f,"HOST_OCAML.load_options("+repr('config'+os.sep+'ocaml_config.py')+")") - - cc = HOST_CC.__class__.__name__ - pr(f,"HOST_CC = fbuild.flxbuild."+cc+"_class."+cc+"()") - pr(f,"HOST_CC.load_options("+repr('config'+os.sep+'host_cc.py')+")") - cc = TARGET_CC.__class__.__name__ - pr(f,"TARGET_CC = fbuild.flxbuild."+cc+"_class."+cc+"()") - pr(f,"TARGET_CC.load_options("+repr('config'+os.sep+'target_cc.py')+")") - - cxx = HOST_CXX.__class__.__name__ - pr(f,"HOST_CXX = fbuild.flxbuild."+cxx+"_class."+cxx+"()") - pr(f,"HOST_CXX.load_options("+repr('config'+os.sep+'host_cxx.py')+")") - cxx = TARGET_CXX.__class__.__name__ - pr(f,"TARGET_CXX = fbuild.flxbuild."+cxx+"_class."+cxx+"()") - pr(f,"TARGET_CXX.load_options("+repr('config'+os.sep+'target_cxx.py')+")") - pr(f,"") - - if options.target_model in ["win32","win64"]: - HAVE_MSVC = 1 - HAVE_GNU = 0 - DIFF = 'FC /L /W' - else: - HAVE_MSVC = 0 - HAVE_GNU = 1 - - #DIFF = "diff -a -b " # build system is Unix Python - # RF - trying out args that work on solaris (-a = not cool) - # could use that sys type stuff here? - DIFF = 'diff -b' - - pr(f,"HAVE_GNU = " + repr(HAVE_GNU)) - pr(f,"FLXCC_CPP='cpp '") - pr(f,"HAVE_MSVC = " + repr(HAVE_MSVC)) - pr(f,"DIFF = " + repr(DIFF)) - - ISCR = sys.executable + ' ' + \ - os.path.join(options.src_dir, 'interscript', 'bin', 'iscr.py') + \ - ' --cache-prefix=lpsrc-cache' - pr(f,"ISCR = " + repr(ISCR)) - - # --------------------------------------------------- - - # RF: noone seems to be using the results of this - # JS: Not yet: policy is to test it out anyhow, in case needed - # - # uname -s: kernel name "linux" on linux - # uname -n: network node name "rosella" on JS box - # uname -r: kernel-release "2.6.12-10-amd64-k8" on JS box - # uname -v: kernel-version " #1 Thu Dec 22 11:12:06 UTC 2005" on JS box - # uname -m: machine hardware name: "x86_64" on JS box - # uname -o: operating system: "GNU/Linux" on JS box - # uname -p: OSX only? on osx reports broad cpu type (e.g. powerpc) - # not sure what it reports on intel macs. - # machine command reports very specific cpu type, e.g. ppc7450, ppc7400 - - try: - output = xqt('uname', '-m') - except ExecutionError: - ARCH = "unknown" - else: - ARCH = output[0].strip().lower() - pr(f,"ARCH = " + repr(ARCH)) - - try: - if options.build_model == 'osx': - output = xqt('uname -p') - else: - output = xqt('uname -o') - except ExecutionError: - OS = 'unknown' - else: - OS = output[0].strip().lower() - - pr(f,"OS = " + repr(OS)) - - pr(f,"") - pr(f,"# HACK to get all the target variables into global namespace") - f.close() - if options.bootfile: - print "Copying bootfile :", options.bootfile - shutil.copy(options.bootfile, os.path.join('config', 'config_bootstrap.py')) - except EnvironmentError: - print "Unable to create config"+os.sep+"__init__.py" - sys.exit(1) - - print "Created config"+os.sep+"__init__.py" - print "Edit this file to set your preferences" - print "This file will not be clobbered by the Felix build process" - - cpkgs = glob.glob("cpkgs"+os.sep+"target"+os.sep+"*.py") - for cpkgf in cpkgs: - cpkg = os.path.splitext(os.path.basename(cpkgf))[0] - print "target CONFIGURING", cpkg - __import__('cpkgs.target.' + cpkg) - -# ----------------------------------------------------------------------------- - -def main(): - time_stamp_format = "%Y/%m/%d %H:%M:%S UTC" - config_time = time.gmtime(time.time()) - CONFIG_TIME = time.strftime(time_stamp_format, config_time) - - options, args = load_options() - - if args: - print 'Unknown configure args:', args - return 1 - - if not os.path.exists('config'): - os.path.mkdir('config') - - BUILD_CC, BUILD_CXX = configure_build(options) - HOST_CC, HOST_CXX, HOST_OCAML = configure_host(options, BUILD_CC, BUILD_CXX) - TARGET_CC, TARGET_CXX = configure_target(options, HOST_CC, HOST_CXX) - configure_run(options) - - write_config(options, CONFIG_TIME, - BUILD_CC, BUILD_CXX, - HOST_CC, HOST_CXX, HOST_OCAML, - TARGET_CC, TARGET_CXX, - ) - - return 0 - -# ----------------------------------------------------------------------------- - -if __name__ == '__main__': - sys.exit(main()) diff --git a/fbuild_old/lib/fbuild/__init__.py b/fbuild_old/lib/fbuild/__init__.py deleted file mode 100644 index e69de29..0000000 diff --git a/fbuild_old/lib/fbuild/compat/_Queue.py b/fbuild_old/lib/fbuild/compat/_Queue.py deleted file mode 100644 index e4b9d3a..0000000 --- a/fbuild_old/lib/fbuild/compat/_Queue.py +++ /dev/null @@ -1,221 +0,0 @@ -"""A multi-producer, multi-consumer queue.""" - -from time import time as _time - -try: - from collections import deque -except ImportError: - class deque(list): - def popleft(self): - return self.pop(0) - -__all__ = ['Empty', 'Full', 'Queue'] - -class Empty(Exception): - "Exception raised by Queue.get(block=0)/get_nowait()." - pass - -class Full(Exception): - "Exception raised by Queue.put(block=0)/put_nowait()." - pass - -class Queue: - """Create a queue object with a given maximum size. - - If maxsize is <= 0, the queue size is infinite. - """ - def __init__(self, maxsize=0): - try: - import threading - except ImportError: - import dummy_threading as threading - self._init(maxsize) - # mutex must be held whenever the queue is mutating. All methods - # that acquire mutex must release it before returning. mutex - # is shared between the three conditions, so acquiring and - # releasing the conditions also acquires and releases mutex. - self.mutex = threading.Lock() - # Notify not_empty whenever an item is added to the queue; a - # thread waiting to get is notified then. - self.not_empty = threading.Condition(self.mutex) - # Notify not_full whenever an item is removed from the queue; - # a thread waiting to put is notified then. - self.not_full = threading.Condition(self.mutex) - # Notify all_tasks_done whenever the number of unfinished tasks - # drops to zero; thread waiting to join() is notified to resume - self.all_tasks_done = threading.Condition(self.mutex) - self.unfinished_tasks = 0 - - def task_done(self): - """Indicate that a formerly enqueued task is complete. - - Used by Queue consumer threads. For each get() used to fetch a task, - a subsequent call to task_done() tells the queue that the processing - on the task is complete. - - If a join() is currently blocking, it will resume when all items - have been processed (meaning that a task_done() call was received - for every item that had been put() into the queue). - - Raises a ValueError if called more times than there were items - placed in the queue. - """ - self.all_tasks_done.acquire() - try: - unfinished = self.unfinished_tasks - 1 - if unfinished <= 0: - if unfinished < 0: - raise ValueError('task_done() called too many times') - self.all_tasks_done.notifyAll() - self.unfinished_tasks = unfinished - finally: - self.all_tasks_done.release() - - def join(self): - """Blocks until all items in the Queue have been gotten and processed. - - The count of unfinished tasks goes up whenever an item is added to the - queue. The count goes down whenever a consumer thread calls task_done() - to indicate the item was retrieved and all work on it is complete. - - When the count of unfinished tasks drops to zero, join() unblocks. - """ - self.all_tasks_done.acquire() - try: - while self.unfinished_tasks: - self.all_tasks_done.wait() - finally: - self.all_tasks_done.release() - - def qsize(self): - """Return the approximate size of the queue (not reliable!).""" - self.mutex.acquire() - n = self._qsize() - self.mutex.release() - return n - - def empty(self): - """Return True if the queue is empty, False otherwise (not reliable!).""" - self.mutex.acquire() - n = self._empty() - self.mutex.release() - return n - - def full(self): - """Return True if the queue is full, False otherwise (not reliable!).""" - self.mutex.acquire() - n = self._full() - self.mutex.release() - return n - - def put(self, item, block=True, timeout=None): - """Put an item into the queue. - - If optional args 'block' is true and 'timeout' is None (the default), - block if necessary until a free slot is available. If 'timeout' is - a positive number, it blocks at most 'timeout' seconds and raises - the Full exception if no free slot was available within that time. - Otherwise ('block' is false), put an item on the queue if a free slot - is immediately available, else raise the Full exception ('timeout' - is ignored in that case). - """ - self.not_full.acquire() - try: - if not block: - if self._full(): - raise Full - elif timeout is None: - while self._full(): - self.not_full.wait() - else: - if timeout < 0: - raise ValueError("'timeout' must be a positive number") - endtime = _time() + timeout - while self._full(): - remaining = endtime - _time() - if remaining <= 0.0: - raise Full - self.not_full.wait(remaining) - self._put(item) - self.unfinished_tasks += 1 - self.not_empty.notify() - finally: - self.not_full.release() - - def put_nowait(self, item): - """Put an item into the queue without blocking. - - Only enqueue the item if a free slot is immediately available. - Otherwise raise the Full exception. - """ - return self.put(item, False) - - def get(self, block=True, timeout=None): - """Remove and return an item from the queue. - - If optional args 'block' is true and 'timeout' is None (the default), - block if necessary until an item is available. If 'timeout' is - a positive number, it blocks at most 'timeout' seconds and raises - the Empty exception if no item was available within that time. - Otherwise ('block' is false), return an item if one is immediately - available, else raise the Empty exception ('timeout' is ignored - in that case). - """ - self.not_empty.acquire() - try: - if not block: - if self._empty(): - raise Empty - elif timeout is None: - while self._empty(): - self.not_empty.wait() - else: - if timeout < 0: - raise ValueError("'timeout' must be a positive number") - endtime = _time() + timeout - while self._empty(): - remaining = endtime - _time() - if remaining <= 0.0: - raise Empty - self.not_empty.wait(remaining) - item = self._get() - self.not_full.notify() - return item - finally: - self.not_empty.release() - - def get_nowait(self): - """Remove and return an item from the queue without blocking. - - Only get an item if one is immediately available. Otherwise - raise the Empty exception. - """ - return self.get(False) - - # Override these methods to implement other queue organizations - # (e.g. stack or priority queue). - # These will only be called with appropriate locks held - - # Initialize the queue representation - def _init(self, maxsize): - self.maxsize = maxsize - self.queue = deque() - - def _qsize(self): - return len(self.queue) - - # Check whether the queue is empty - def _empty(self): - return not self.queue - - # Check whether the queue is full - def _full(self): - return self.maxsize > 0 and len(self.queue) == self.maxsize - - # Put a new item in the queue - def _put(self, item): - self.queue.append(item) - - # Get an item from the queue - def _get(self): - return self.queue.popleft() diff --git a/fbuild_old/lib/fbuild/compat/__init__.py b/fbuild_old/lib/fbuild/compat/__init__.py deleted file mode 100644 index e69de29..0000000 diff --git a/fbuild_old/lib/fbuild/compat/_subprocess.py b/fbuild_old/lib/fbuild/compat/_subprocess.py deleted file mode 100644 index fa3d683..0000000 --- a/fbuild_old/lib/fbuild/compat/_subprocess.py +++ /dev/null @@ -1,1246 +0,0 @@ -# subprocess - Subprocesses with accessible I/O streams -# -# For more information about this module, see PEP 324. -# -# This module should remain compatible with Python 2.2, see PEP 291. -# -# Copyright (c) 2003-2005 by Peter Astrand <as...@ly...> -# -# Licensed to PSF under a Contributor Agreement. -# See http://www.python.org/2.4/license for licensing details. - -r"""subprocess - Subprocesses with accessible I/O streams - -This module allows you to spawn processes, connect to their -input/output/error pipes, and obtain their return codes. This module -intends to replace several other, older modules and functions, like: - -os.system -os.spawn* -os.popen* -popen2.* -commands.* - -Information about how the subprocess module can be used to replace these -modules and functions can be found below. - - - -Using the subprocess module -=========================== -This module defines one class called Popen: - -class Popen(args, bufsize=0, executable=None, - stdin=None, stdout=None, stderr=None, - preexec_fn=None, close_fds=False, shell=False, - cwd=None, env=None, universal_newlines=False, - startupinfo=None, creationflags=0): - - -Arguments are: - -args should be a string, or a sequence of program arguments. The -program to execute is normally the first item in the args sequence or -string, but can be explicitly set by using the executable argument. - -On UNIX, with shell=False (default): In this case, the Popen class -uses os.execvp() to execute the child program. args should normally -be a sequence. A string will be treated as a sequence with the string -as the only item (the program to execute). - -On UNIX, with shell=True: If args is a string, it specifies the -command string to execute through the shell. If args is a sequence, -the first item specifies the command string, and any additional items -will be treated as additional shell arguments. - -On Windows: the Popen class uses CreateProcess() to execute the child -program, which operates on strings. If args is a sequence, it will be -converted to a string using the list2cmdline method. Please note that -not all MS Windows applications interpret the command line the same -way: The list2cmdline is designed for applications using the same -rules as the MS C runtime. - -bufsize, if given, has the same meaning as the corresponding argument -to the built-in open() function: 0 means unbuffered, 1 means line -buffered, any other positive value means use a buffer of -(approximately) that size. A negative bufsize means to use the system -default, which usually means fully buffered. The default value for -bufsize is 0 (unbuffered). - -stdin, stdout and stderr specify the executed programs' standard -input, standard output and standard error file handles, respectively. -Valid values are PIPE, an existing file descriptor (a positive -integer), an existing file object, and None. PIPE indicates that a -new pipe to the child should be created. With None, no redirection -will occur; the child's file handles will be inherited from the -parent. Additionally, stderr can be STDOUT, which indicates that the -stderr data from the applications should be captured into the same -file handle as for stdout. - -If preexec_fn is set to a callable object, this object will be called -in the child process just before the child is executed. - -If close_fds is true, all file descriptors except 0, 1 and 2 will be -closed before the child process is executed. - -if shell is true, the specified command will be executed through the -shell. - -If cwd is not None, the current directory will be changed to cwd -before the child is executed. - -If env is not None, it defines the environment variables for the new -process. - -If universal_newlines is true, the file objects stdout and stderr are -opened as a text files, but lines may be terminated by any of '\n', -the Unix end-of-line convention, '\r', the Macintosh convention or -'\r\n', the Windows convention. All of these external representations -are seen as '\n' by the Python program. Note: This feature is only -available if Python is built with universal newline support (the -default). Also, the newlines attribute of the file objects stdout, -stdin and stderr are not updated by the communicate() method. - -The startupinfo and creationflags, if given, will be passed to the -underlying CreateProcess() function. They can specify things such as -appearance of the main window and priority for the new process. -(Windows only) - - -This module also defines two shortcut functions: - -call(*popenargs, **kwargs): - Run command with arguments. Wait for command to complete, then - return the returncode attribute. - - The arguments are the same as for the Popen constructor. Example: - - retcode = call(["ls", "-l"]) - -check_call(*popenargs, **kwargs): - Run command with arguments. Wait for command to complete. If the - exit code was zero then return, otherwise raise - CalledProcessError. The CalledProcessError object will have the - return code in the returncode attribute. - - The arguments are the same as for the Popen constructor. Example: - - check_call(["ls", "-l"]) - -Exceptions ----------- -Exceptions raised in the child process, before the new program has -started to execute, will be re-raised in the parent. Additionally, -the exception object will have one extra attribute called -'child_traceback', which is a string containing traceback information -from the childs point of view. - -The most common exception raised is OSError. This occurs, for -example, when trying to execute a non-existent file. Applications -should prepare for OSErrors. - -A ValueError will be raised if Popen is called with invalid arguments. - -check_call() will raise CalledProcessError, if the called process -returns a non-zero return code. - - -Security --------- -Unlike some other popen functions, this implementation will never call -/bin/sh implicitly. This means that all characters, including shell -metacharacters, can safely be passed to child processes. - - -Popen objects -============= -Instances of the Popen class have the following methods: - -poll() - Check if child process has terminated. Returns returncode - attribute. - -wait() - Wait for child process to terminate. Returns returncode attribute. - -communicate(input=None) - Interact with process: Send data to stdin. Read data from stdout - and stderr, until end-of-file is reached. Wait for process to - terminate. The optional input argument should be a string to be - sent to the child process, or None, if no data should be sent to - the child. - - communicate() returns a tuple (stdout, stderr). - - Note: The data read is buffered in memory, so do not use this - method if the data size is large or unlimited. - -The following attributes are also available: - -stdin - If the stdin argument is PIPE, this attribute is a file object - that provides input to the child process. Otherwise, it is None. - -stdout - If the stdout argument is PIPE, this attribute is a file object - that provides output from the child process. Otherwise, it is - None. - -stderr - If the stderr argument is PIPE, this attribute is file object that - provides error output from the child process. Otherwise, it is - None. - -pid - The process ID of the child process. - -returncode - The child return code. A None value indicates that the process - hasn't terminated yet. A negative value -N indicates that the - child was terminated by signal N (UNIX only). - - -Replacing older functions with the subprocess module -==================================================== -In this section, "a ==> b" means that b can be used as a replacement -for a. - -Note: All functions in this section fail (more or less) silently if -the executed program cannot be found; this module raises an OSError -exception. - -In the following examples, we assume that the subprocess module is -imported with "from subprocess import *". - - -Replacing /bin/sh shell backquote ---------------------------------- -output=`mycmd myarg` -==> -output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0] - - -Replacing shell pipe line -------------------------- -output=`dmesg | grep hda` -==> -p1 = Popen(["dmesg"], stdout=PIPE) -p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) -output = p2.communicate()[0] - - -Replacing os.system() ---------------------- -sts = os.system("mycmd" + " myarg") -==> -p = Popen("mycmd" + " myarg", shell=True) -pid, sts = os.waitpid(p.pid, 0) - -Note: - -* Calling the program through the shell is usually not required. - -* It's easier to look at the returncode attribute than the - exitstatus. - -A more real-world example would look like this: - -try: - retcode = call("mycmd" + " myarg", shell=True) - if retcode < 0: - print >>sys.stderr, "Child was terminated by signal", -retcode - else: - print >>sys.stderr, "Child returned", retcode -except OSError, e: - print >>sys.stderr, "Execution failed:", e - - -Replacing os.spawn* -------------------- -P_NOWAIT example: - -pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg") -==> -pid = Popen(["/bin/mycmd", "myarg"]).pid - - -P_WAIT example: - -retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg") -==> -retcode = call(["/bin/mycmd", "myarg"]) - - -Vector example: - -os.spawnvp(os.P_NOWAIT, path, args) -==> -Popen([path] + args[1:]) - - -Environment example: - -os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env) -==> -Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"}) - - -Replacing os.popen* -------------------- -pipe = os.popen(cmd, mode='r', bufsize) -==> -pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout - -pipe = os.popen(cmd, mode='w', bufsize) -==> -pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin - - -(child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize) -==> -p = Popen(cmd, shell=True, bufsize=bufsize, - stdin=PIPE, stdout=PIPE, close_fds=True) -(child_stdin, child_stdout) = (p.stdin, p.stdout) - - -(child_stdin, - child_stdout, - child_stderr) = os.popen3(cmd, mode, bufsize) -==> -p = Popen(cmd, shell=True, bufsize=bufsize, - stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) -(child_stdin, - child_stdout, - child_stderr) = (p.stdin, p.stdout, p.stderr) - - -(child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize) -==> -p = Popen(cmd, shell=True, bufsize=bufsize, - stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) -(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout) - - -Replacing popen2.* ------------------- -Note: If the cmd argument to popen2 functions is a string, the command -is executed through /bin/sh. If it is a list, the command is directly -executed. - -(child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) -==> -p = Popen(["somestring"], shell=True, bufsize=bufsize - stdin=PIPE, stdout=PIPE, close_fds=True) -(child_stdout, child_stdin) = (p.stdout, p.stdin) - - -(child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode) -==> -p = Popen(["mycmd", "myarg"], bufsize=bufsize, - stdin=PIPE, stdout=PIPE, close_fds=True) -(child_stdout, child_stdin) = (p.stdout, p.stdin) - -The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen, -except that: - -* subprocess.Popen raises an exception if the execution fails -* the capturestderr argument is replaced with the stderr argument. -* stdin=PIPE and stdout=PIPE must be specified. -* popen2 closes all filedescriptors by default, but you have to specify - close_fds=True with subprocess.Popen. - - -""" - -import sys -mswindows = (sys.platform == "win32") - -import os -import types -import traceback - -# Exception classes used by this module. -class CalledProcessError(Exception): - """This exception is raised when a process run by check_call() returns - a non-zero exit status. The exit status will be stored in the - returncode attribute.""" - def __init__(self, returncode, cmd): - self.returncode = returncode - self.cmd = cmd - def __str__(self): - return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode) - - -if mswindows: - import threading - import msvcrt - if 0: # <-- change this to use pywin32 instead of the _subprocess driver - import pywintypes - from win32api import GetStdHandle, STD_INPUT_HANDLE, \ - STD_OUTPUT_HANDLE, STD_ERROR_HANDLE - from win32api import GetCurrentProcess, DuplicateHandle, \ - GetModuleFileName, GetVersion - from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE - from win32pipe import CreatePipe - from win32process import CreateProcess, STARTUPINFO, \ - GetExitCodeProcess, STARTF_USESTDHANDLES, \ - STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE - from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0 - else: - from _subprocess import * - class STARTUPINFO: - dwFlags = 0 - hStdInput = None - hStdOutput = None - hStdError = None - wShowWindow = 0 - class pywintypes: - error = IOError -else: - import select - import errno - import fcntl - import pickle - -__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "CalledProcessError"] - -try: - MAXFD = os.sysconf("SC_OPEN_MAX") -except: - MAXFD = 256 - -# True/False does not exist on 2.2.0 -try: - False -except NameError: - False = 0 - True = 1 - -_active = [] - -def _cleanup(): - for inst in _active[:]: - if inst.poll(_deadstate=sys.maxint) >= 0: - try: - _active.remove(inst) - except ValueError: - # This can happen if two threads create a new Popen instance. - # It's harmless that it was already removed, so ignore. - pass - -PIPE = -1 -STDOUT = -2 - - -def call(*popenargs, **kwargs): - """Run command with arguments. Wait for command to complete, then - return the returncode attribute. - - The arguments are the same as for the Popen constructor. Example: - - retcode = call(["ls", "-l"]) - """ - return Popen(*popenargs, **kwargs).wait() - - -def check_call(*popenargs, **kwargs): - """Run command with arguments. Wait for command to complete. If - the exit code was zero then return, otherwise raise - CalledProcessError. The CalledProcessError object will have the - return code in the returncode attribute. - - The arguments are the same as for the Popen constructor. Example: - - check_call(["ls", "-l"]) - """ - retcode = call(*popenargs, **kwargs) - cmd = kwargs.get("args") - if cmd is None: - cmd = popenargs[0] - if retcode: - raise CalledProcessError(retcode, cmd) - return retcode - - -def list2cmdline(seq): - """ - Translate a sequence of arguments into a command line - string, using the same rules as the MS C runtime: - - 1) Arguments are delimited by white space, which is either a - space or a tab. - - 2) A string surrounded by double quotation marks is - interpreted as a single argument, regard... [truncated message content] |