Re: [SharpOS Developers] Proposal

[Pieter M. <pie...@in...>]

Chriss,

I don't think anyone here is questioning your contributions. I for one
think you've done a great job with developing something like the AOT.
Due to the very fact that we don't understand how it really works (yet),
makes it impossible for us to check how good it really is. So we can't
comment on the quality of your work.

By making at least a general architecture design, or even just document
its current design a lot more people will have the chance to help you
out. Most people can only assist in small areas (due to knowledge or
time constraints), so they won't go as far as to read all the code to be
able to add some lines. If on the other hand we had some design
documents (e.g. a general map of the bytecode->asm flow, some class
diagrams, ...) those people will be "in the loop" with much less effort.
What we would do here is instead of all of us doing the same work
(trying to understand the structure), some of us document it for the
rest so they can understand in a fraction of that time.

This does not only hold true for AOT, but also for the kernel.

I believe we can all agree that good documentation is a MUST to keep
everything clear and organizable. So I think the question that has to be
answered first is:
Do we continue development and start documenting by following its trail?
Or do we ask Chriss and the others to halt for a moment so we can all
catch up?

I think the last option is the best one because otherwise we wouldn't
notice architectural problems until it's too late.

Just my 2 cents...

Insite Mees-Delbeke VOF
Zomerstraat 29A
9270 Laarne
Telefoon: 09 367 96 34
GSM: 0478 44 96 04
E-mail: pie...@in...
Site: http://www.insitehosting.be


-----Original Message-----
From: sha...@li...
[mailto:sha...@li...] On Behalf Of
Darx Kies
Sent: woensdag 29 augustus 2007 4:23
To: sha...@li...
Subject: Re: [SharpOS Developers] Proposal

Scott Balmos wrote:
> Darx Kies wrote:
>  =20
>> Scott Balmos wrote:
>>  =20
>>    =20
>>> Chad Z. Hower aka Kudzu wrote:
>>>  =20
>>>    =20
>>>      =20
>>>> I'd like to make a few proposals.
>>>>
>>>> 1) We establish a loose but workable team structure which includes
conflict
>>>> resolution.
>>>>
>>>>  =20
>>>>    =20
>>>>      =20
>>>>        =20
>>> Someone will probably say this already exists. I don't know. My
major=20
>>> recommendation right now is establishment of a build master, who
owns=20
>>> the trunk. All changes should be done in local repos. When something

>>> should be promoted to the trunk, it is debated here, goes through a
code=20
>>> check, and the build master normalizes the code to a coding standard

>>> before committing it to the trunk. Trying to get rid of William
breaking=20
>>> the kernel again (heh), or our infamous problems of things compiling

>>> under Mono but not VStudio or vice versa.
>>>
>>> Between that, and API / architecture docs before you start coding a=20
>>> major piece. Everyone else can tell you how much I've grumbled
lately=20
>>> about not being able to get my VM code going because the trunk repo
got=20
>>> reworked, the base kernel's in flux, and the AOT is still buggy. :)
>>>  =20
>>>    =20
>>>      =20
>> And now guess why I was begging for test cases and test cases and I=20
>> still do.
>>  =20
>>    =20
>>>  =20
>>>    =20
>>>      =20
>>>> 2) We establish a highly documented, well debated kernel design
doc. Then
>>>> proceed to make the kernel rock solid before putting too much
effort into
>>>> anything beyond the command line.
>>>>
>>>>  =20
>>>>    =20
>>>>      =20
>>>>        =20
>>> I've been kicking for this for awhile. Currently it's more a
code-first,=20
>>> write-architecture-later. The largest part of the system, the
compiler,=20
>>> is AOT-only, and only one person knows its architecture. I've been
very=20
>>> vocal about that. Don't know if Chriss has gotten anywhere with=20
>>> documenting the architecture.
>>>  =20
>>>    =20
>>>      =20
>> Like I already said. The AOT itself is still subject to change, and=20
>> right now I am rewriting parts of it.
>> The code is partly documented and it does even have some inline
examples=20
>> to make it more understandable
>> and I everyone that wants to understand it better just write me on
IRC.
>>
>>  =20
>>    =20
>>> Personally, I would halt all current kernel-related development, go
back=20
>>> and tear back apart the compiler and get that completely
rearchitected,=20
>>> redocumented, and retested.=20
>>>    =20
>>>      =20
>> That is what actually is happening right now, even though I had to=20
>> suspend the work on it for a while.... again.
>>
>>  =20
>>    =20
>>> Start with reading in the source bytecode at=20
>>> a method level from multiple sources, whether it's a byte stream,
file,=20
>>> whatever. This ensures an easy separation of code into an embeddable
JIT=20
>>> engine, with the AOT system simply being a file-feeder shell.
Convert to=20
>>> unoptimized IR code, then register transfer code. Spit out direct,=20
>>> unoptimized machine code from there. Possibly hand-verify if
necessary=20
>>> the generated machine code. That way we know the basic compiler
works=20
>>> and implements all CIL opcodes. Optimizations such as SSA, dead code

>>> elimination, loop unwinding, etc can all be implemented as
transforms on=20
>>> top of the base IR code stream. From the testing I've seen, I can't
tell=20
>>> whether bugs are in simple bytecode conversion or various IR=20
>>> optimization algorithms. They all seem to be intertwined currently.
>>>
>>> But I'm liable to get my head chopped off by Chriss for the above.
:)
>>>  =20
>>>    =20
>>>      =20
>> I am not that cruel. ;)
>>
>> The idea was actually to get something working ASAP so that the ppl
that=20
>> wanted to work on the kernel
>> can do something too. That is why there are lots of generated test
cases=20
>> for the x86 encoding but pertty few test
>> cases for the rest of the AOT. I would just suggest to look in the=20
>> source code for the TODO lines and there are
>> lots of them. Most of it was just implemented when an exception
raised=20
>> because some part was not implemented
>> yet. And most of the time the trunk kernel was the main source of
code=20
>> to test and develop the AOT. Johann and
>> William felt pitiful enough to write test cases.;) Thank you guys!
>>
>> What I am trying to say is that I am the only one that is working on
it=20
>> and any help is more than welcome.
>>
>> Chriss.
>>  =20
>>    =20
>
> Not to be chicken-and-egg, but almost all good test cases cannot be=20
> covered unless there was a set of overall architecture designs. And=20
> getting that written in writing, rather than saying "just talk to me
on=20
> IRC", will ultimately benefit us more. Likewise with "just read the=20
> code". That's just not good enough with a multi-member team. Plus, how

> will we ever know whether we've written enough unit tests?
>  =20
We need at least more than 150 that there are now (not counting the x86=20
ones). And besides
what I meant were NOT test cases for the AOT internals but for .NET.=20
Just take a look at the
current test cases and the test cases for example of mono, pnet and=20
rotor. And once again, yes,
I will document the AOT once it is stable and more or less feature=20
complete and when I said
"talk to me on IRC" I was saying that I want to help everyone that wants

to get involved
with the AOT and start document it where needed which was done partly by

others as well.

> You acknowledge that you're the only one working on it currently, and=20
> would welcome the help. Personally, I'd love to help, as compiler
theory=20
> as well as OS design was my emphasis threads in college. But, as Bruce

> just noted in his email, everything's black box. You yourself once
noted=20
> to me that most of the compiler design was taken from another project
of=20
> yours that converted CIL to JavaScript, parts of which you can't=20
> release, and the architecture of which you don't clearly remember. :)=20
>  =20
It's not about not being able but not wanting to release it. And I don't

get what you mean about
not remembering the architecture.

> Plus, seeing huge code files that are over 400k in size scares the
crap=20
> out of anyone. Heck, you remember my first few weeks of talking. I=20
> couldn't follow much of anything, and it took longer for me to realize

> and remember various compiler optimization tricks such as SSA.
>
> Even now, you're rewriting the compiler.=20
Yes I am rewriting parts of it, which is mostly refactoring, removing=20
some hacks and
at the same time I continue to implement the missing parts. And I am=20
also working on making it
possible to turn on and off the whole constant propagation, copy=20
propagation, constant folding and
so on.

> Okay, but again, what=20
> architecture? How is it designed, the flow of code from a bytecode=20
> stream into native platform code? etc. We won't truly know how to test

> it unless we know intimately how it is designed to work.=20
Read above about the test cases.
> That's the=20
> whole point behind writing a true written architecture document,
posted=20
> up on our site. That way everyone knows how it works, everyone knows
how=20
> to test it, and everyone knows (generally) where a fix would have to
be=20
> made in what code transformation layer in order to fix a bug. Hence
also=20
> my advocation of starting out with "raw, unoptimized" code. Layer on
the=20
> optimizers one by one as they're tested, in order to minimize the
number=20
> of variables when tracking down a bug.
>  =20
Read the above again. That is what I am working on as well. I am=20
rewriting parts of it to make it
more easily for anyone to start working on the AOT.
> Again, yeah, we've got a working compiler now and it was produced
"just=20
> to get things going". But now we're taking a step back, looking to see

> how well everything's designed in the first place (if at all), in
order=20
> to make for a better long-term system.
>  =20

Sorry but that sounds harsh!

The AOT is by far not complete and not perfect. And I tried to do my=20
best to contribute to the
project instead of whining and doing nothing. Not all of the AOT is a=20
"black box", tough I see
no attempt to at least document that for the others, (except the ones by

xfury and illuminus).
Illuminus for example documented how to write the AOT test cases. And=20
once again, if anyone
volunteers to write the documentation of the AOT, yes....., write me on
IRC.

Everyone does what he cans, as good as he cans, when he cans.

Chriss.

------------------------------------------------------------------------
-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
SharpOS-Developers mailing list
Sha...@li...
https://lists.sourceforge.net/lists/listinfo/sharpos-developers