Re: [Tcl9-colibri] [TCLCORE] ANNOUNCE: Colibri version 0.12

[Frédéric B. <fre...@fr...>]

Hi Larry,


Le 30/11/2011 15:59, Larry McVoy a écrit :
> Strangely enough, we're playing similar games with write protection.  Can
> you tell me
>
> - which platforms have you tested this on?

Linux/x86 and Windows/x86. It compiles on Linux/x86-64 but the linker spits a 
strange error on a missing symbol (oddly enough it compiled just fine on version 
0.11), so I haven't had the opportunity to test it. I have no Win64 toolchain 
but given the overall portability of the code it should work OK 
(platform-specific code is minimal).

As for Unix flavours, as long as you have mmap, pthreads and sigaction it should 
work fine, 32 or 64 bits.

> - Is there a regression suite that tests the protection stuff?  If so,
>    a readme on how to run it and we'll see if we can test it on our
>    cluster (all of the above and more)

Yes, there are both a regression test suite and a benchmark suite. The latter is 
mostly for data structure comparison, so we can ignore it for now. The test 
suite is in the /Test dir, there is a series of .c for each test group plus a 
main test.c file. By default it runs the whole Colibri test suite testColibri(), 
but you can comment it out and run individual tests.

Additionnally, there is a group of individual stress tests at the end of 
TestMain() that aren't by nature integrated in the regression test suite and can 
run independently. The ones we want are testChildVect() and testChildren(). The 
former allocates mutable elements and links them so as to buid a chain in 
reverse order (older vectors point to newer ones). The latter does more or less 
the same thing but with "word synonym chains" (in short, a circular linked list 
of related objects). Both hit memory heavily and make an intense use of parent 
tracking.

To run these tests, simply build the whole stuff with the provided makefiles, 
run the regression test once to check that everything is OK, then modify test.c 
to run the tests you want. You can also tweak the constants in the tests if you 
want to adjust the memory usage, but defaults are fine.


> - did you see any perf gain or loss?

On regular usage (mutable hash and trie maps, both string- and integer-based), 
my benchmarks show a speedup of up to 25% on write access compared to the 
previous version 0.11 where mutable objects and client code had to explicitly 
call Col_WordSetModified() upon each mutable operation (updating the parent 
tracking structures). Read access is mostly unchanged. But this doesn't take 
into account the GC process. For that, the stress tests mentioned above are 
interesting because they are designed to spend most of their time in GC. Overall 
I've observed a performance penalty of about 10%, but:

- In real world applications, GC run much less often than actual read/write 
access code, so this penalty is more than compensated by the speedup in write 
access.

- This penalty is mostly due not to the page write tracking itself but to the 
changes in internal structures: I've made tests with manual vs. automatic write 
tracking using the same internals (calling the same book-keeping code 
respectively from client code and exception/signal handler), and found the 
latter to be slightly faster, to my surprise: I thought at first that 
exception/signal handling would induce a lot of lost CPU cycles, but since it 
only occurs on the first write, the cost seems amortized compared to the inline 
overhead of calling Col_WordSetModified() explicitly.

> Cool stuff!

Thanks! Feel free to steal the code, it's BSD-licensed (the stuff you want is in 
platform/unix/colUnixPlatform.c and plaform/win32colWin32Platform.c for 
platform-specific stuff, and colAlloc.c for memory management)


Cheers, Fred