Menu
▾
▴
sharpos-developers
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-28 20:24:56
|
I'd like to make a few proposals. 1) We establish a loose but workable team structure which includes conflict resolution. 2) We establish a highly documented, well debated kernel design doc. Then proceed to make the kernel rock solid before putting too much effort into anything beyond the command line. 3) We establish a build process to produce ready to go VMWare images as well as easier to build source scripts. -- Chad Z. Hower aka Kudzu "Programming is an art form that fights back" http://www.KudzuWorld.com/ http://www.Woo-Hoo.net/ http://www.DelphiToDotNet.com/ |
|
From: Scott B. <sb...@me...> - 2007-08-28 20:39:53
|
Chad Z. Hower aka Kudzu wrote: > I'd like to make a few proposals. > > 1) We establish a loose but workable team structure which includes conflict > resolution. > > Someone will probably say this already exists. I don't know. My major recommendation right now is establishment of a build master, who owns the trunk. All changes should be done in local repos. When something should be promoted to the trunk, it is debated here, goes through a code check, and the build master normalizes the code to a coding standard before committing it to the trunk. Trying to get rid of William breaking the kernel again (heh), or our infamous problems of things compiling under Mono but not VStudio or vice versa. Between that, and API / architecture docs before you start coding a major piece. Everyone else can tell you how much I've grumbled lately about not being able to get my VM code going because the trunk repo got reworked, the base kernel's in flux, and the AOT is still buggy. :) > 2) We establish a highly documented, well debated kernel design doc. Then > proceed to make the kernel rock solid before putting too much effort into > anything beyond the command line. > > I've been kicking for this for awhile. Currently it's more a code-first, write-architecture-later. The largest part of the system, the compiler, is AOT-only, and only one person knows its architecture. I've been very vocal about that. Don't know if Chriss has gotten anywhere with documenting the architecture. Personally, I would halt all current kernel-related development, go back and tear back apart the compiler and get that completely rearchitected, redocumented, and retested. Start with reading in the source bytecode at a method level from multiple sources, whether it's a byte stream, file, whatever. This ensures an easy separation of code into an embeddable JIT engine, with the AOT system simply being a file-feeder shell. Convert to unoptimized IR code, then register transfer code. Spit out direct, unoptimized machine code from there. Possibly hand-verify if necessary the generated machine code. That way we know the basic compiler works and implements all CIL opcodes. Optimizations such as SSA, dead code elimination, loop unwinding, etc can all be implemented as transforms on top of the base IR code stream. From the testing I've seen, I can't tell whether bugs are in simple bytecode conversion or various IR optimization algorithms. They all seem to be intertwined currently. But I'm liable to get my head chopped off by Chriss for the above. :) > 3) We establish a build process to produce ready to go VMWare images as well > as easier to build source scripts. > Already exists, for both VMWare and Bochs |
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-28 20:47:59
|
> Someone will probably say this already exists. I don't know. My major > recommendation right now is establishment of a build master, who owns Where is it documented? :) What is the process of conflict resolution? I don't just mean build team vs doc team type things. > check, and the build master normalizes the code to a coding standard > before committing it to the trunk. Trying to get rid of William We can use code formatters for such items in many places. > I've been kicking for this for awhile. Currently it's more a code- > write-architecture-later. The largest part of the system, the compiler, > is AOT-only, and only one person knows its architecture. I've been very > vocal about that. Don't know if Chriss has gotten anywhere with > documenting the architecture. The problem is that docs get too far behind too quickly and people forget why/what. I can take a stab at this - but it will also force a lot of review IMO of architecture and I wont be hesitant to bring issues up and push for change when needed, especially in core. We HAVE to get core RIGHT. > Already exists, for both VMWare and Bochs Ok, then change Item 3 to - Make a new, better, easier to understand website. I can do that too, if you don't care what tool I use. That failing I can make a fork with my own info. :) |
|
From: Scott B. <sb...@me...> - 2007-08-28 20:56:15
|
Chad Z. Hower aka Kudzu wrote: >> Someone will probably say this already exists. I don't know. My major >> recommendation right now is establishment of a build master, who owns >> > > Where is it documented? :) What is the process of conflict resolution? I > don't just mean build team vs doc team type things. > > >> check, and the build master normalizes the code to a coding standard >> before committing it to the trunk. Trying to get rid of William >> > > We can use code formatters for such items in many places. > > >> I've been kicking for this for awhile. Currently it's more a code- >> write-architecture-later. The largest part of the system, the compiler, >> is AOT-only, and only one person knows its architecture. I've been very >> vocal about that. Don't know if Chriss has gotten anywhere with >> documenting the architecture. >> > > The problem is that docs get too far behind too quickly and people forget > why/what. I can take a stab at this - but it will also force a lot of review > IMO of architecture and I wont be hesitant to bring issues up and push for > change when needed, especially in core. We HAVE to get core RIGHT. > > >> Already exists, for both VMWare and Bochs >> > > Ok, then change Item 3 to - Make a new, better, easier to understand > website. I can do that too, if you don't care what tool I use. That failing > I can make a fork with my own info. :) > > Definitely not disagreeing with any of the above. I'm taking all next week off for a vacation, so I have the time to do this. But within all the flurry of email, we haven't heard from William or Chriss, who so far are the two main devs of the existing codebase. There's been a general consensus of "what we have already works", even if it isn't formalized or documented, which makes the "professional software engineer" side of me grumble. The biggest thing is whether William and Chriss would agree to stop kernel and compiler development until we have a chance to look back over the architecture, agree and document said new architecture, and look back over existing code to massage it accordingly. |
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-28 20:59:15
|
> The biggest thing is whether William and Chriss would agree to stop > kernel and compiler development until we have a chance to look back > over > the architecture, agree and document said new architecture, and look > back over existing code to massage it accordingly. Considering there are only 6 of us, if they don't, sounds like the other 4 might be interested in forking. :) |
|
From: Darx K. <dar...@gm...> - 2007-08-28 22:08:43
|
Scott Balmos wrote: > Chad Z. Hower aka Kudzu wrote: > >> I'd like to make a few proposals. >> >> 1) We establish a loose but workable team structure which includes conflict >> resolution. >> >> >> > Someone will probably say this already exists. I don't know. My major > recommendation right now is establishment of a build master, who owns > the trunk. All changes should be done in local repos. When something > should be promoted to the trunk, it is debated here, goes through a code > check, and the build master normalizes the code to a coding standard > before committing it to the trunk. Trying to get rid of William breaking > the kernel again (heh), or our infamous problems of things compiling > under Mono but not VStudio or vice versa. > > Between that, and API / architecture docs before you start coding a > major piece. Everyone else can tell you how much I've grumbled lately > about not being able to get my VM code going because the trunk repo got > reworked, the base kernel's in flux, and the AOT is still buggy. :) > And now guess why I was begging for test cases and test cases and I still do. > >> 2) We establish a highly documented, well debated kernel design doc. Then >> proceed to make the kernel rock solid before putting too much effort into >> anything beyond the command line. >> >> >> > I've been kicking for this for awhile. Currently it's more a code-first, > write-architecture-later. The largest part of the system, the compiler, > is AOT-only, and only one person knows its architecture. I've been very > vocal about that. Don't know if Chriss has gotten anywhere with > documenting the architecture. > Like I already said. The AOT itself is still subject to change, and right now I am rewriting parts of it. The code is partly documented and it does even have some inline examples to make it more understandable and I everyone that wants to understand it better just write me on IRC. > Personally, I would halt all current kernel-related development, go back > and tear back apart the compiler and get that completely rearchitected, > redocumented, and retested. That is what actually is happening right now, even though I had to suspend the work on it for a while.... again. > Start with reading in the source bytecode at > a method level from multiple sources, whether it's a byte stream, file, > whatever. This ensures an easy separation of code into an embeddable JIT > engine, with the AOT system simply being a file-feeder shell. Convert to > unoptimized IR code, then register transfer code. Spit out direct, > unoptimized machine code from there. Possibly hand-verify if necessary > the generated machine code. That way we know the basic compiler works > and implements all CIL opcodes. Optimizations such as SSA, dead code > elimination, loop unwinding, etc can all be implemented as transforms on > top of the base IR code stream. From the testing I've seen, I can't tell > whether bugs are in simple bytecode conversion or various IR > optimization algorithms. They all seem to be intertwined currently. > > But I'm liable to get my head chopped off by Chriss for the above. :) > I am not that cruel. ;) The idea was actually to get something working ASAP so that the ppl that wanted to work on the kernel can do something too. That is why there are lots of generated test cases for the x86 encoding but pertty few test cases for the rest of the AOT. I would just suggest to look in the source code for the TODO lines and there are lots of them. Most of it was just implemented when an exception raised because some part was not implemented yet. And most of the time the trunk kernel was the main source of code to test and develop the AOT. Johann and William felt pitiful enough to write test cases.;) Thank you guys! What I am trying to say is that I am the only one that is working on it and any help is more than welcome. Chriss. |
|
From: Bruce <ill...@gm...> - 2007-08-28 22:53:58
|
On 8/28/07, Darx Kies <dar...@gm...> wrote: > > Scott Balmos wrote: > > Chad Z. Hower aka Kudzu wrote: > > > >> I'd like to make a few proposals. > >> > >> 1) We establish a loose but workable team structure which includes > conflict > >> resolution. > >> > >> > >> > > Someone will probably say this already exists. I don't know. My major > > recommendation right now is establishment of a build master, who owns > > the trunk. All changes should be done in local repos. When something > > should be promoted to the trunk, it is debated here, goes through a code > > check, and the build master normalizes the code to a coding standard > > before committing it to the trunk. Trying to get rid of William breaking > > the kernel again (heh), or our infamous problems of things compiling > > under Mono but not VStudio or vice versa. Though I am not personally proficient with branching and then re-merging for the purpose of trunk preservation, I'm a big fan of moving in that direction. I tried setting up a continuous integration system, (CruiseControl.NET), for the project - and CruiseControl.NET from what I understand is the better of the open-source options - but it was a pain in the arse to keep it configured properly, especially with the build scripts changing all the time. If we think we are worth it, we should check into getting a free license, available to open source projects, of a commercial continuous integration suite, such as Jira. > Between that, and API / architecture docs before you start coding a > > major piece. Everyone else can tell you how much I've grumbled lately > > about not being able to get my VM code going because the trunk repo got > > reworked, the base kernel's in flux, and the AOT is still buggy. :) > > > And now guess why I was begging for test cases and test cases and I > still do. These are the things that we ideally need to test: -1- test kernel API functionality -2- test AOT output validity -3- test AOT API functionality (And ideally, the testing process needs to be fully automated, and fully attempted and reported after changes to the trunk.) #1 can be tested, as we've discussed, by conditionally compiling test cases into the kernel. We have not done any work in this area yet. As far as #2, we've done this thus far by compiling a tests kernel that runs the set of test methods, and then printing if one fails. This can be automated by writing to a serial port on the virtual machine which is routed to a named pipe or log file on the host machine, and then processed. The problem with what we have been doing is, some bugs only arise when compiling multiple tests. AOTing test A actually makes test B either fail to AOT or fail to succeed. So we may need a more intelligent testing process that attempts to resolve failed-to-AOT-tests attempts by removing test cases. *shrug*. It has also been suggested that we need to "hand verify" our machine code output, so perhaps a way to compare x86 code output from our AOT with output from .NET or Mono could be possible. As far as #3 (testing the AOT's API functionality) - this is more of a sanity issue, might not need to be done off-the-bat, and it is certainly agreeable that alot of it would be redundant with testing AOT output. Furthermore, regarding all this testing, is that the amount of testing done will compound with every architecture we eventually support. All the more reason to establish as much automation as possible. > > >> 2) We establish a highly documented, well debated kernel design doc. > Then > >> proceed to make the kernel rock solid before putting too much effort > into > >> anything beyond the command line. > >> I firmly agree with this. The problem is, most of us aren't organized enough or well-practiced enough to draw up diagrams of an entire kernel. But I agree that it does need to be done, it does need to be debated. But once settled, it should not be changed for a long time. Projects that keep re-hashing their core design, before they get "off the ground", never get anywhere. They fall apart. We will have to accept that not everyone will be happy with the final design. Thats why forks were invented. >> > > I've been kicking for this for awhile. Currently it's more a code-first, > > write-architecture-later. The largest part of the system, the compiler, > > is AOT-only, and only one person knows its architecture. I've been very > > vocal about that. Don't know if Chriss has gotten anywhere with > > documenting the architecture. > > > Like I already said. The AOT itself is still subject to change, and > right now I am rewriting parts of it. > The code is partly documented and it does even have some inline examples > to make it more understandable > and I everyone that wants to understand it better just write me on IRC. > > > Personally, I would halt all current kernel-related development, go back > > and tear back apart the compiler and get that completely rearchitected, > > redocumented, and retested. > That is what actually is happening right now, even though I had to > suspend the work on it for a while.... again. But in essence, I have to agree. No more kernel work, yet. The AOT is the heart of the project. We have to keep it beating. And we have to get that blood pressure up. I think if we setup a comprehensive test automation process, we can polish the AOT with ease, and then proceed with the kernel, and he monstrous task of at least *trying* to get everyone to agree on its design. > Start with reading in the source bytecode at > > a method level from multiple sources, whether it's a byte stream, file, > > whatever. This ensures an easy separation of code into an embeddable JIT > > engine, with the AOT system simply being a file-feeder shell. Convert to > > unoptimized IR code, then register transfer code. Spit out direct, > > unoptimized machine code from there. Possibly hand-verify if necessary > > the generated machine code. That way we know the basic compiler works > > and implements all CIL opcodes. Optimizations such as SSA, dead code > > elimination, loop unwinding, etc can all be implemented as transforms on > > top of the base IR code stream. From the testing I've seen, I can't tell > > whether bugs are in simple bytecode conversion or various IR > > optimization algorithms. They all seem to be intertwined currently. > > > > But I'm liable to get my head chopped off by Chriss for the above. :) > > > I am not that cruel. ;) > > The idea was actually to get something working ASAP so that the ppl that > wanted to work on the kernel > can do something too. That is why there are lots of generated test cases > for the x86 encoding but pertty few test > cases for the rest of the AOT. I would just suggest to look in the > source code for the TODO lines and there are > lots of them. Most of it was just implemented when an exception raised > because some part was not implemented > yet. And most of the time the trunk kernel was the main source of code > to test and develop the AOT. Johann and > William felt pitiful enough to write test cases.;) Thank you guys! > > What I am trying to say is that I am the only one that is working on it > and any help is more than welcome. I have to say, Chriss, you are very helpful. But the AOT is a beast. A black-box beast. Try not to blame a few of us for being intimidated. ;-) The function-pointer-stub fiasco has scared me quite thoroughly into a position of lurking... So we have to keep our task list, (regardless as to whether or not we keep it on Trac), needs to be fleshed out as thoroughly as possible. If someone decides to work on something (preferably something that has been discussed) before a task has been made for it, then they should, at the very least, make a task for it first. Alot of the craziness around here is simply because we don't keep things organized. No one really knows what someone else is doing until the trunk gets broken... |
|
From: Scott B. <sb...@me...> - 2007-08-28 23:03:16
|
Darx Kies wrote: > Scott Balmos wrote: > >> Chad Z. Hower aka Kudzu wrote: >> >> >>> I'd like to make a few proposals. >>> >>> 1) We establish a loose but workable team structure which includes conflict >>> resolution. >>> >>> >>> >>> >> Someone will probably say this already exists. I don't know. My major >> recommendation right now is establishment of a build master, who owns >> the trunk. All changes should be done in local repos. When something >> should be promoted to the trunk, it is debated here, goes through a code >> check, and the build master normalizes the code to a coding standard >> before committing it to the trunk. Trying to get rid of William breaking >> the kernel again (heh), or our infamous problems of things compiling >> under Mono but not VStudio or vice versa. >> >> Between that, and API / architecture docs before you start coding a >> major piece. Everyone else can tell you how much I've grumbled lately >> about not being able to get my VM code going because the trunk repo got >> reworked, the base kernel's in flux, and the AOT is still buggy. :) >> >> > And now guess why I was begging for test cases and test cases and I > still do. > >> >> >>> 2) We establish a highly documented, well debated kernel design doc. Then >>> proceed to make the kernel rock solid before putting too much effort into >>> anything beyond the command line. >>> >>> >>> >>> >> I've been kicking for this for awhile. Currently it's more a code-first, >> write-architecture-later. The largest part of the system, the compiler, >> is AOT-only, and only one person knows its architecture. I've been very >> vocal about that. Don't know if Chriss has gotten anywhere with >> documenting the architecture. >> >> > Like I already said. The AOT itself is still subject to change, and > right now I am rewriting parts of it. > The code is partly documented and it does even have some inline examples > to make it more understandable > and I everyone that wants to understand it better just write me on IRC. > > >> Personally, I would halt all current kernel-related development, go back >> and tear back apart the compiler and get that completely rearchitected, >> redocumented, and retested. >> > That is what actually is happening right now, even though I had to > suspend the work on it for a while.... again. > > >> Start with reading in the source bytecode at >> a method level from multiple sources, whether it's a byte stream, file, >> whatever. This ensures an easy separation of code into an embeddable JIT >> engine, with the AOT system simply being a file-feeder shell. Convert to >> unoptimized IR code, then register transfer code. Spit out direct, >> unoptimized machine code from there. Possibly hand-verify if necessary >> the generated machine code. That way we know the basic compiler works >> and implements all CIL opcodes. Optimizations such as SSA, dead code >> elimination, loop unwinding, etc can all be implemented as transforms on >> top of the base IR code stream. From the testing I've seen, I can't tell >> whether bugs are in simple bytecode conversion or various IR >> optimization algorithms. They all seem to be intertwined currently. >> >> But I'm liable to get my head chopped off by Chriss for the above. :) >> >> > I am not that cruel. ;) > > The idea was actually to get something working ASAP so that the ppl that > wanted to work on the kernel > can do something too. That is why there are lots of generated test cases > for the x86 encoding but pertty few test > cases for the rest of the AOT. I would just suggest to look in the > source code for the TODO lines and there are > lots of them. Most of it was just implemented when an exception raised > because some part was not implemented > yet. And most of the time the trunk kernel was the main source of code > to test and develop the AOT. Johann and > William felt pitiful enough to write test cases.;) Thank you guys! > > What I am trying to say is that I am the only one that is working on it > and any help is more than welcome. > > Chriss. > Not to be chicken-and-egg, but almost all good test cases cannot be covered unless there was a set of overall architecture designs. And getting that written in writing, rather than saying "just talk to me on IRC", will ultimately benefit us more. Likewise with "just read the code". That's just not good enough with a multi-member team. Plus, how will we ever know whether we've written enough unit tests? You acknowledge that you're the only one working on it currently, and would welcome the help. Personally, I'd love to help, as compiler theory as well as OS design was my emphasis threads in college. But, as Bruce just noted in his email, everything's black box. You yourself once noted to me that most of the compiler design was taken from another project of yours that converted CIL to JavaScript, parts of which you can't release, and the architecture of which you don't clearly remember. :) Plus, seeing huge code files that are over 400k in size scares the crap out of anyone. Heck, you remember my first few weeks of talking. I couldn't follow much of anything, and it took longer for me to realize and remember various compiler optimization tricks such as SSA. Even now, you're rewriting the compiler. Okay, but again, what architecture? How is it designed, the flow of code from a bytecode stream into native platform code? etc. We won't truly know how to test it unless we know intimately how it is designed to work. That's the whole point behind writing a true written architecture document, posted up on our site. That way everyone knows how it works, everyone knows how to test it, and everyone knows (generally) where a fix would have to be made in what code transformation layer in order to fix a bug. Hence also my advocation of starting out with "raw, unoptimized" code. Layer on the optimizers one by one as they're tested, in order to minimize the number of variables when tracking down a bug. Again, yeah, we've got a working compiler now and it was produced "just to get things going". But now we're taking a step back, looking to see how well everything's designed in the first place (if at all), in order to make for a better long-term system. |
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-29 02:28:28
|
> Like I already said. The AOT itself is still subject to change, and > right now I am rewriting parts of it. > The code is partly documented and it does even have some inline > examples > to make it more understandable > and I everyone that wants to understand it better just write me on IRC. It will likely be Nov before I could so any serious work. But I spent several hours tonight digging through mostly AOT with Scott helping me navigate. We can greatly simplify AOT in many areas. AOT is critical to get right and I think we must address it very soon. In addition, the method of using the attributes to match up unsafe code into safe code. There are more expedient ways to do this which also include static checking of interfaces which is not present now. |
|
From: William L. <xfu...@gm...> - 2007-08-30 05:25:17
|
On 8/28/07, Scott Balmos <sb...@me...> wrote: > Chad Z. Hower aka Kudzu wrote: > > I'd like to make a few proposals. > > > > 1) We establish a loose but workable team structure which includes conflict > > resolution. > > > > > Someone will probably say this already exists. I don't know. My major > recommendation right now is establishment of a build master, who owns > the trunk. All changes should be done in local repos. When something > should be promoted to the trunk, it is debated here, goes through a code > check, and the build master normalizes the code to a coding standard > before committing it to the trunk. Trying to get rid of William breaking > the kernel again (heh), or our infamous problems of things compiling > under Mono but not VStudio or vice versa. To my defense, that topic change was voluntary! Honestly I should be developing on my own branch, but when I started there wasn't any other branches or development. They started popping up before I was done :). > Between that, and API / architecture docs before you start coding a > major piece. Everyone else can tell you how much I've grumbled lately > about not being able to get my VM code going because the trunk repo got > reworked, the base kernel's in flux, and the AOT is still buggy. :) API design by spec does not work, that I am certain of. I do think having a plan is good, and writing extensive docs for what we have is good, but outlining the entire API before we do it is rather pointless if we throw in one new use case, because that use case will change the API. I'm a strong believer in the feature freeze conventions. > > 2) We establish a highly documented, well debated kernel design doc. Then > > proceed to make the kernel rock solid before putting too much effort into > > anything beyond the command line. Now a design doc sounds good, but recall that we have not even fulfilled that command line milestone. We don't have a working shell. > I've been kicking for this for awhile. Currently it's more a code-first, > write-architecture-later. The largest part of the system, the compiler, > is AOT-only, and only one person knows its architecture. I've been very > vocal about that. Don't know if Chriss has gotten anywhere with > documenting the architecture. Architecture is completely different from "API docs". I agree entirely on outlining how the thing will work, just not too specifically. There's a sweet spot somewhere. > Personally, I would halt all current kernel-related development, go back > and tear back apart the compiler and get that completely rearchitected, > redocumented, and retested. Start with reading in the source bytecode at > a method level from multiple sources, whether it's a byte stream, file, > whatever. Now would be a good time, because Chriss is reworking the whole thing. But you cannot demand such things of one developer. He needs more help. Chriss has been calling for testcases the entire time, and only just recently have me and Bruce answered those calls, and we still require TONS more. He was having a hard time keeping the AOT stable throughout his reworking without enough testcases. Unfortunately that was right at the time I no longer had stable internet access. This is further complicated by the entry barriers of the AOT (which are very high). I've only just begun to gain a real extensive grasp on how it works. > This ensures an easy separation of code into an embeddable JIT > engine, with the AOT system simply being a file-feeder shell. It's already in the plans to refactor the AOT into a JIT with a more or less dumb AOT program that embeds it. Like I said, now is a good time. > Convert to > unoptimized IR code, then register transfer code. Spit out direct, > unoptimized machine code from there. Possibly hand-verify if necessary > the generated machine code. Chriss has just recently made some commits to make it more possible to disable the optimizations btw. > That way we know the basic compiler works > and implements all CIL opcodes. Hand verification isn't useful in this case, because we'll break stuff as we continue development. Testcases are the way to go, and we are doing that but by far not enough. > Optimizations such as SSA, dead code > elimination, loop unwinding, etc can all be implemented as transforms on > top of the base IR code stream. Agreed. One of the first projects I took on here was to make them all optional, but that wasn't possible because of the design (at the time). > From the testing I've seen, I can't tell > whether bugs are in simple bytecode conversion or various IR > optimization algorithms. They all seem to be intertwined currently. Yes, it's been a big issue. > > But I'm liable to get my head chopped off by Chriss for the above. :) He might. But if he doesn't, I at least nibbled at your ear :) > > 3) We establish a build process to produce ready to go VMWare images as well > > as easier to build source scripts. > > > Already exists, for both VMWare and Bochs Oh, and QEmu works, but it always does (no config files!) > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > SharpOS-Developers mailing list > Sha...@li... > https://lists.sourceforge.net/lists/listinfo/sharpos-developers > -- fury long name: William Lahti handle :: fury freenode :: xfury blog :: http://xfurious.blogspot.com/ |
|
From: Darx K. <dar...@gm...> - 2007-08-29 02:22:48
|
Scott Balmos wrote: > Darx Kies wrote: > >> Scott Balmos wrote: >> >> >>> Chad Z. Hower aka Kudzu wrote: >>> >>> >>> >>>> I'd like to make a few proposals. >>>> >>>> 1) We establish a loose but workable team structure which includes conflict >>>> resolution. >>>> >>>> >>>> >>>> >>>> >>> Someone will probably say this already exists. I don't know. My major >>> recommendation right now is establishment of a build master, who owns >>> the trunk. All changes should be done in local repos. When something >>> should be promoted to the trunk, it is debated here, goes through a code >>> check, and the build master normalizes the code to a coding standard >>> before committing it to the trunk. Trying to get rid of William breaking >>> the kernel again (heh), or our infamous problems of things compiling >>> under Mono but not VStudio or vice versa. >>> >>> Between that, and API / architecture docs before you start coding a >>> major piece. Everyone else can tell you how much I've grumbled lately >>> about not being able to get my VM code going because the trunk repo got >>> reworked, the base kernel's in flux, and the AOT is still buggy. :) >>> >>> >>> >> And now guess why I was begging for test cases and test cases and I >> still do. >> >> >>> >>> >>> >>>> 2) We establish a highly documented, well debated kernel design doc. Then >>>> proceed to make the kernel rock solid before putting too much effort into >>>> anything beyond the command line. >>>> >>>> >>>> >>>> >>>> >>> I've been kicking for this for awhile. Currently it's more a code-first, >>> write-architecture-later. The largest part of the system, the compiler, >>> is AOT-only, and only one person knows its architecture. I've been very >>> vocal about that. Don't know if Chriss has gotten anywhere with >>> documenting the architecture. >>> >>> >>> >> Like I already said. The AOT itself is still subject to change, and >> right now I am rewriting parts of it. >> The code is partly documented and it does even have some inline examples >> to make it more understandable >> and I everyone that wants to understand it better just write me on IRC. >> >> >> >>> Personally, I would halt all current kernel-related development, go back >>> and tear back apart the compiler and get that completely rearchitected, >>> redocumented, and retested. >>> >>> >> That is what actually is happening right now, even though I had to >> suspend the work on it for a while.... again. >> >> >> >>> Start with reading in the source bytecode at >>> a method level from multiple sources, whether it's a byte stream, file, >>> whatever. This ensures an easy separation of code into an embeddable JIT >>> engine, with the AOT system simply being a file-feeder shell. Convert to >>> unoptimized IR code, then register transfer code. Spit out direct, >>> unoptimized machine code from there. Possibly hand-verify if necessary >>> the generated machine code. That way we know the basic compiler works >>> and implements all CIL opcodes. Optimizations such as SSA, dead code >>> elimination, loop unwinding, etc can all be implemented as transforms on >>> top of the base IR code stream. From the testing I've seen, I can't tell >>> whether bugs are in simple bytecode conversion or various IR >>> optimization algorithms. They all seem to be intertwined currently. >>> >>> But I'm liable to get my head chopped off by Chriss for the above. :) >>> >>> >>> >> I am not that cruel. ;) >> >> The idea was actually to get something working ASAP so that the ppl that >> wanted to work on the kernel >> can do something too. That is why there are lots of generated test cases >> for the x86 encoding but pertty few test >> cases for the rest of the AOT. I would just suggest to look in the >> source code for the TODO lines and there are >> lots of them. Most of it was just implemented when an exception raised >> because some part was not implemented >> yet. And most of the time the trunk kernel was the main source of code >> to test and develop the AOT. Johann and >> William felt pitiful enough to write test cases.;) Thank you guys! >> >> What I am trying to say is that I am the only one that is working on it >> and any help is more than welcome. >> >> Chriss. >> >> > > Not to be chicken-and-egg, but almost all good test cases cannot be > covered unless there was a set of overall architecture designs. And > getting that written in writing, rather than saying "just talk to me on > IRC", will ultimately benefit us more. Likewise with "just read the > code". That's just not good enough with a multi-member team. Plus, how > will we ever know whether we've written enough unit tests? > We need at least more than 150 that there are now (not counting the x86 ones). And besides what I meant were NOT test cases for the AOT internals but for .NET. Just take a look at the current test cases and the test cases for example of mono, pnet and rotor. And once again, yes, I will document the AOT once it is stable and more or less feature complete and when I said "talk to me on IRC" I was saying that I want to help everyone that wants to get involved with the AOT and start document it where needed which was done partly by others as well. > You acknowledge that you're the only one working on it currently, and > would welcome the help. Personally, I'd love to help, as compiler theory > as well as OS design was my emphasis threads in college. But, as Bruce > just noted in his email, everything's black box. You yourself once noted > to me that most of the compiler design was taken from another project of > yours that converted CIL to JavaScript, parts of which you can't > release, and the architecture of which you don't clearly remember. :) > It's not about not being able but not wanting to release it. And I don't get what you mean about not remembering the architecture. > Plus, seeing huge code files that are over 400k in size scares the crap > out of anyone. Heck, you remember my first few weeks of talking. I > couldn't follow much of anything, and it took longer for me to realize > and remember various compiler optimization tricks such as SSA. > > Even now, you're rewriting the compiler. Yes I am rewriting parts of it, which is mostly refactoring, removing some hacks and at the same time I continue to implement the missing parts. And I am also working on making it possible to turn on and off the whole constant propagation, copy propagation, constant folding and so on. > Okay, but again, what > architecture? How is it designed, the flow of code from a bytecode > stream into native platform code? etc. We won't truly know how to test > it unless we know intimately how it is designed to work. Read above about the test cases. > That's the > whole point behind writing a true written architecture document, posted > up on our site. That way everyone knows how it works, everyone knows how > to test it, and everyone knows (generally) where a fix would have to be > made in what code transformation layer in order to fix a bug. Hence also > my advocation of starting out with "raw, unoptimized" code. Layer on the > optimizers one by one as they're tested, in order to minimize the number > of variables when tracking down a bug. > Read the above again. That is what I am working on as well. I am rewriting parts of it to make it more easily for anyone to start working on the AOT. > Again, yeah, we've got a working compiler now and it was produced "just > to get things going". But now we're taking a step back, looking to see > how well everything's designed in the first place (if at all), in order > to make for a better long-term system. > Sorry but that sounds harsh! The AOT is by far not complete and not perfect. And I tried to do my best to contribute to the project instead of whining and doing nothing. Not all of the AOT is a "black box", tough I see no attempt to at least document that for the others, (except the ones by xfury and illuminus). Illuminus for example documented how to write the AOT test cases. And once again, if anyone volunteers to write the documentation of the AOT, yes....., write me on IRC. Everyone does what he cans, as good as he cans, when he cans. Chriss. |
|
From: Pieter M. <pie...@in...> - 2007-08-29 07:46:25
|
Chriss, I don't think anyone here is questioning your contributions. I for one think you've done a great job with developing something like the AOT. Due to the very fact that we don't understand how it really works (yet), makes it impossible for us to check how good it really is. So we can't comment on the quality of your work. By making at least a general architecture design, or even just document its current design a lot more people will have the chance to help you out. Most people can only assist in small areas (due to knowledge or time constraints), so they won't go as far as to read all the code to be able to add some lines. If on the other hand we had some design documents (e.g. a general map of the bytecode->asm flow, some class diagrams, ...) those people will be "in the loop" with much less effort. What we would do here is instead of all of us doing the same work (trying to understand the structure), some of us document it for the rest so they can understand in a fraction of that time. This does not only hold true for AOT, but also for the kernel. I believe we can all agree that good documentation is a MUST to keep everything clear and organizable. So I think the question that has to be answered first is: Do we continue development and start documenting by following its trail? Or do we ask Chriss and the others to halt for a moment so we can all catch up? I think the last option is the best one because otherwise we wouldn't notice architectural problems until it's too late. Just my 2 cents... Insite Mees-Delbeke VOF Zomerstraat 29A 9270 Laarne Telefoon: 09 367 96 34 GSM: 0478 44 96 04 E-mail: pie...@in... Site: http://www.insitehosting.be -----Original Message----- From: sha...@li... [mailto:sha...@li...] On Behalf Of Darx Kies Sent: woensdag 29 augustus 2007 4:23 To: sha...@li... Subject: Re: [SharpOS Developers] Proposal Scott Balmos wrote: > Darx Kies wrote: > =20 >> Scott Balmos wrote: >> =20 >> =20 >>> Chad Z. Hower aka Kudzu wrote: >>> =20 >>> =20 >>> =20 >>>> I'd like to make a few proposals. >>>> >>>> 1) We establish a loose but workable team structure which includes conflict >>>> resolution. >>>> >>>> =20 >>>> =20 >>>> =20 >>>> =20 >>> Someone will probably say this already exists. I don't know. My major=20 >>> recommendation right now is establishment of a build master, who owns=20 >>> the trunk. All changes should be done in local repos. When something >>> should be promoted to the trunk, it is debated here, goes through a code=20 >>> check, and the build master normalizes the code to a coding standard >>> before committing it to the trunk. Trying to get rid of William breaking=20 >>> the kernel again (heh), or our infamous problems of things compiling >>> under Mono but not VStudio or vice versa. >>> >>> Between that, and API / architecture docs before you start coding a=20 >>> major piece. Everyone else can tell you how much I've grumbled lately=20 >>> about not being able to get my VM code going because the trunk repo got=20 >>> reworked, the base kernel's in flux, and the AOT is still buggy. :) >>> =20 >>> =20 >>> =20 >> And now guess why I was begging for test cases and test cases and I=20 >> still do. >> =20 >> =20 >>> =20 >>> =20 >>> =20 >>>> 2) We establish a highly documented, well debated kernel design doc. Then >>>> proceed to make the kernel rock solid before putting too much effort into >>>> anything beyond the command line. >>>> >>>> =20 >>>> =20 >>>> =20 >>>> =20 >>> I've been kicking for this for awhile. Currently it's more a code-first,=20 >>> write-architecture-later. The largest part of the system, the compiler,=20 >>> is AOT-only, and only one person knows its architecture. I've been very=20 >>> vocal about that. Don't know if Chriss has gotten anywhere with=20 >>> documenting the architecture. >>> =20 >>> =20 >>> =20 >> Like I already said. The AOT itself is still subject to change, and=20 >> right now I am rewriting parts of it. >> The code is partly documented and it does even have some inline examples=20 >> to make it more understandable >> and I everyone that wants to understand it better just write me on IRC. >> >> =20 >> =20 >>> Personally, I would halt all current kernel-related development, go back=20 >>> and tear back apart the compiler and get that completely rearchitected,=20 >>> redocumented, and retested.=20 >>> =20 >>> =20 >> That is what actually is happening right now, even though I had to=20 >> suspend the work on it for a while.... again. >> >> =20 >> =20 >>> Start with reading in the source bytecode at=20 >>> a method level from multiple sources, whether it's a byte stream, file,=20 >>> whatever. This ensures an easy separation of code into an embeddable JIT=20 >>> engine, with the AOT system simply being a file-feeder shell. Convert to=20 >>> unoptimized IR code, then register transfer code. Spit out direct,=20 >>> unoptimized machine code from there. Possibly hand-verify if necessary=20 >>> the generated machine code. That way we know the basic compiler works=20 >>> and implements all CIL opcodes. Optimizations such as SSA, dead code >>> elimination, loop unwinding, etc can all be implemented as transforms on=20 >>> top of the base IR code stream. From the testing I've seen, I can't tell=20 >>> whether bugs are in simple bytecode conversion or various IR=20 >>> optimization algorithms. They all seem to be intertwined currently. >>> >>> But I'm liable to get my head chopped off by Chriss for the above. :) >>> =20 >>> =20 >>> =20 >> I am not that cruel. ;) >> >> The idea was actually to get something working ASAP so that the ppl that=20 >> wanted to work on the kernel >> can do something too. That is why there are lots of generated test cases=20 >> for the x86 encoding but pertty few test >> cases for the rest of the AOT. I would just suggest to look in the=20 >> source code for the TODO lines and there are >> lots of them. Most of it was just implemented when an exception raised=20 >> because some part was not implemented >> yet. And most of the time the trunk kernel was the main source of code=20 >> to test and develop the AOT. Johann and >> William felt pitiful enough to write test cases.;) Thank you guys! >> >> What I am trying to say is that I am the only one that is working on it=20 >> and any help is more than welcome. >> >> Chriss. >> =20 >> =20 > > Not to be chicken-and-egg, but almost all good test cases cannot be=20 > covered unless there was a set of overall architecture designs. And=20 > getting that written in writing, rather than saying "just talk to me on=20 > IRC", will ultimately benefit us more. Likewise with "just read the=20 > code". That's just not good enough with a multi-member team. Plus, how > will we ever know whether we've written enough unit tests? > =20 We need at least more than 150 that there are now (not counting the x86=20 ones). And besides what I meant were NOT test cases for the AOT internals but for .NET.=20 Just take a look at the current test cases and the test cases for example of mono, pnet and=20 rotor. And once again, yes, I will document the AOT once it is stable and more or less feature=20 complete and when I said "talk to me on IRC" I was saying that I want to help everyone that wants to get involved with the AOT and start document it where needed which was done partly by others as well. > You acknowledge that you're the only one working on it currently, and=20 > would welcome the help. Personally, I'd love to help, as compiler theory=20 > as well as OS design was my emphasis threads in college. But, as Bruce > just noted in his email, everything's black box. You yourself once noted=20 > to me that most of the compiler design was taken from another project of=20 > yours that converted CIL to JavaScript, parts of which you can't=20 > release, and the architecture of which you don't clearly remember. :)=20 > =20 It's not about not being able but not wanting to release it. And I don't get what you mean about not remembering the architecture. > Plus, seeing huge code files that are over 400k in size scares the crap=20 > out of anyone. Heck, you remember my first few weeks of talking. I=20 > couldn't follow much of anything, and it took longer for me to realize > and remember various compiler optimization tricks such as SSA. > > Even now, you're rewriting the compiler.=20 Yes I am rewriting parts of it, which is mostly refactoring, removing=20 some hacks and at the same time I continue to implement the missing parts. And I am=20 also working on making it possible to turn on and off the whole constant propagation, copy=20 propagation, constant folding and so on. > Okay, but again, what=20 > architecture? How is it designed, the flow of code from a bytecode=20 > stream into native platform code? etc. We won't truly know how to test > it unless we know intimately how it is designed to work.=20 Read above about the test cases. > That's the=20 > whole point behind writing a true written architecture document, posted=20 > up on our site. That way everyone knows how it works, everyone knows how=20 > to test it, and everyone knows (generally) where a fix would have to be=20 > made in what code transformation layer in order to fix a bug. Hence also=20 > my advocation of starting out with "raw, unoptimized" code. Layer on the=20 > optimizers one by one as they're tested, in order to minimize the number=20 > of variables when tracking down a bug. > =20 Read the above again. That is what I am working on as well. I am=20 rewriting parts of it to make it more easily for anyone to start working on the AOT. > Again, yeah, we've got a working compiler now and it was produced "just=20 > to get things going". But now we're taking a step back, looking to see > how well everything's designed in the first place (if at all), in order=20 > to make for a better long-term system. > =20 Sorry but that sounds harsh! The AOT is by far not complete and not perfect. And I tried to do my=20 best to contribute to the project instead of whining and doing nothing. Not all of the AOT is a=20 "black box", tough I see no attempt to at least document that for the others, (except the ones by xfury and illuminus). Illuminus for example documented how to write the AOT test cases. And=20 once again, if anyone volunteers to write the documentation of the AOT, yes....., write me on IRC. Everyone does what he cans, as good as he cans, when he cans. Chriss. ------------------------------------------------------------------------ - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ SharpOS-Developers mailing list Sha...@li... https://lists.sourceforge.net/lists/listinfo/sharpos-developers |
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-29 09:27:22
|
> I don't think anyone here is questioning your contributions. I for one > think you've done a great job with developing something like the AOT. After some pretty hefty review I have some pretty hefty thoughts on AOT. But Id like to talk to Chris 1:1 and get some more backgrounders before spewing my thoughts about. But I'll say this. AOT is core to this project. It needs to be modularized, opened up and heavily worked on. I also believe significant portions need thought about and likely redone. But if we modularize it a bit more and some of us help, the load wont all be on Chris and we can do it. |
|
From: Pieter M. <pie...@in...> - 2007-08-29 10:48:59
|
Well I agree that AOT and its optimizations should be modular. But I think that most of the code can be salvaged for this purpose. I see it this way: Read IL (fixed) -> Transform to blocks (fixed) -> Block optimizations (modular) -> Transform to SSA (fixed) -> SSA optimizations (modular) -> Register allocation (modular) -> Transform out of SSA (fixed) -> Write assembly (modular) Where optimizations in the block/SSA domain could be sequentially executed and the sequence repeated until the code no longer changes. Insite Mees-Delbeke VOF Zomerstraat 29A 9270 Laarne Telefoon: 09 367 96 34 GSM: 0478 44 96 04 E-mail: pie...@in... Site: http://www.insitehosting.be -----Original Message----- From: sha...@li... [mailto:sha...@li...] On Behalf Of Chad Z. Hower aka Kudzu Sent: woensdag 29 augustus 2007 11:27 To: sha...@li... Subject: Re: [SharpOS Developers] Proposal > I don't think anyone here is questioning your contributions. I for one > think you've done a great job with developing something like the AOT. After some pretty hefty review I have some pretty hefty thoughts on AOT. But Id like to talk to Chris 1:1 and get some more backgrounders before spewing my thoughts about. But I'll say this. AOT is core to this project. It needs to be modularized, opened up and heavily worked on. I also believe significant portions need thought about and likely redone. But if we modularize it a bit more and some of us help, the load wont all be on Chris and we can do it. ------------------------------------------------------------------------ - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ SharpOS-Developers mailing list Sha...@li... https://lists.sourceforge.net/lists/listinfo/sharpos-developers |
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-29 11:07:22
|
> Well I agree that AOT and its optimizations should be modular. But I > think that most of the code can be salvaged for this purpose. Absolutely code can be used. > I see it this way: > > Read IL (fixed) -> Transform to blocks (fixed) -> Block optimizations > (modular) -> Transform to SSA (fixed) -> SSA optimizations (modular) -> > Register allocation (modular) -> Transform out of SSA (fixed) -> Write > assembly (modular) That would be my plan - Id like to make a very simple proof of concept first though. |
|
From: Darx K. <dar...@gm...> - 2007-08-29 10:26:59
|
Pieter Mees wrote: > Chriss, > > I don't think anyone here is questioning your contributions. I for one > think you've done a great job with developing something like the AOT. > Due to the very fact that we don't understand how it really works (yet), > makes it impossible for us to check how good it really is. So we can't > comment on the quality of your work. > > By making at least a general architecture design, or even just document > its current design a lot more people will have the chance to help you > out. Most people can only assist in small areas (due to knowledge or > time constraints), so they won't go as far as to read all the code to be > able to add some lines. If on the other hand we had some design > documents (e.g. a general map of the bytecode->asm flow, some class > diagrams, ...) those people will be "in the loop" with much less effort. > What we would do here is instead of all of us doing the same work > (trying to understand the structure), some of us document it for the > rest so they can understand in a fraction of that time. > > This does not only hold true for AOT, but also for the kernel. > > I believe we can all agree that good documentation is a MUST to keep > everything clear and organizable. So I think the question that has to be > answered first is: > Do we continue development and start documenting by following its trail? > Or do we ask Chriss and the others to halt for a moment so we can all > catch up? > > I think the last option is the best one because otherwise we wouldn't > notice architectural problems until it's too late. I agree on that too. So what we would need now is volunteers to write the documentation. Anyone? Anyone interested, I am on IRC. Or perhaps we should all gather on IRC for Q&A Session this weekend? That way someone can write the documentation, so that everyone can understand it and use it? Chriss. |
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-29 10:31:31
|
> So what we would need now is volunteers to write the documentation. > Anyone? I already volunteered on some - but you have to be part of it. And I have bigger thoughts on AOT as well... In fact let me try to do a prototype with Matthijs on seomthing. > Or perhaps we should all gather on IRC for Q&A Session this weekend? > That way > someone can write the documentation, so that everyone can understand it > and use it? Might be good. |
|
From: Darx K. <dar...@gm...> - 2007-08-29 11:18:40
|
Hi Chad Z. Hower aka Kudzu wrote: >> So what we would need now is volunteers to write the documentation. >> Anyone? >> > > I already volunteered on some - but you have to be part of it. And I have > bigger thoughts on AOT as well... In fact let me try to do a prototype with > Matthijs on seomthing. > Sounds great! I am on IRC as you are even though it seems you don't get my messages. Chriss. |
|
From: Johann M. <jo...@ma...> - 2007-08-30 02:24:04
|
Here was my original idea for the AOT compiler. I started work on this before Chriss came in and showed off his work. And then as time flew by (and I was assigned to three research projects this semester), I completely ran out of time and energy to keep this project running. Anyway, my idea was very simple, but inefficient: If you look at the IL code, it's completely stack based. You push on values, pop values off, different operations do different things to the stack, etc... why not translate the IL into simple x86 stack operations? For example, when you: int i = 5; in C#, the IL pushes that value onto the "IL stack". Why not simply translate that into an x86 push operation? In fact, almost all IL commands can easily be translated to simple x86 operations. The only real issue is with the decimal type (80-bit ?), which doesn't translate well to x86. The only thing we would have to standardize on is the memory layout of an assembly. Is there a header which includes the entry points? Is there a table which includes locations of all the classes and methods? etc... We'd use a standard set of registers for each operation. Meaning, when we want to do the IL add operation, we pop the last two values from the stack off into a documented set of registers, add them, and then push them back. As time went on, we could combine common sets of operations for more efficiency. Like I said, I ran out of time and Chriss's solutions seemed to work better at the time. Another issue was my frustration with developing in MonoDevelop. Now I'm running Visual Studio through a VM on my MacBook Pro (btw: svn supports conversion from CRLF's to LF's automatically, so we can easily incorporate Windows and *nix developers). Chriss's AOT compiler does a lot more than my simple compiler would (mostly dealing with optimizations). If I had more time I'd love to test out my idea though. Johann On Aug 29, 2007, at 2:27 AM, Chad Z. Hower aka Kudzu wrote: >> I don't think anyone here is questioning your contributions. I for >> one >> think you've done a great job with developing something like the AOT. > > After some pretty hefty review I have some pretty hefty thoughts on > AOT. But > Id like to talk to Chris 1:1 and get some more backgrounders before > spewing > my thoughts about. > > But I'll say this. AOT is core to this project. It needs to be > modularized, > opened up and heavily worked on. I also believe significant > portions need > thought about and likely redone. But if we modularize it a bit more > and some > of us help, the load wont all be on Chris and we can do it. > > > > > ---------------------------------------------------------------------- > --- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a > browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > SharpOS-Developers mailing list > Sha...@li... > https://lists.sourceforge.net/lists/listinfo/sharpos-developers |
|
From: William L. <xfu...@gm...> - 2007-08-30 05:49:03
|
Yeah, I put a lot of effort into Scott's message before I realized the thread had grown larger than I thought. What Chriss said sums up much of my points. > Where optimizations in the block/SSA domain could be sequentially > executed and the sequence repeated until the code no longer changes. This is done in the AOT, but it's not readily apparent. On 8/29/07, Johann MacDonagh <jo...@ma...> wrote: > Here was my original idea for the AOT compiler. I started work on > this before Chriss came in and showed off his work. And then as time > flew by (and I was assigned to three research projects this > semester), I completely ran out of time and energy to keep this > project running. > > Anyway, my idea was very simple, but inefficient: > > If you look at the IL code, it's completely stack based. You push on > values, pop values off, different operations do different things to > the stack, etc... why not translate the IL into simple x86 stack > operations? > > For example, when you: > > int i = 5; > > in C#, the IL pushes that value onto the "IL stack". Why not simply > translate that into an x86 push operation? In fact, almost all IL > commands can easily be translated to simple x86 operations. The only > real issue is with the decimal type (80-bit ?), which doesn't > translate well to x86. > > The only thing we would have to standardize on is the memory layout > of an assembly. Is there a header which includes the entry points? Is > there a table which includes locations of all the classes and > methods? etc... We'd use a standard set of registers for each > operation. Meaning, when we want to do the IL add operation, we pop > the last two values from the stack off into a documented set of > registers, add them, and then push them back. As time went on, we > could combine common sets of operations for more efficiency. > > Like I said, I ran out of time and Chriss's solutions seemed to work > better at the time. Another issue was my frustration with developing > in MonoDevelop. Now I'm running Visual Studio through a VM on my > MacBook Pro (btw: svn supports conversion from CRLF's to LF's > automatically, so we can easily incorporate Windows and *nix > developers). > > Chriss's AOT compiler does a lot more than my simple compiler would > (mostly dealing with optimizations). If I had more time I'd love to > test out my idea though. > > Johann > > On Aug 29, 2007, at 2:27 AM, Chad Z. Hower aka Kudzu wrote: > > >> I don't think anyone here is questioning your contributions. I for > >> one > >> think you've done a great job with developing something like the AOT. > > > > After some pretty hefty review I have some pretty hefty thoughts on > > AOT. But > > Id like to talk to Chris 1:1 and get some more backgrounders before > > spewing > > my thoughts about. > > > > But I'll say this. AOT is core to this project. It needs to be > > modularized, > > opened up and heavily worked on. I also believe significant > > portions need > > thought about and likely redone. But if we modularize it a bit more > > and some > > of us help, the load wont all be on Chris and we can do it. > > > > > > > > > > ---------------------------------------------------------------------- > > --- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a > > browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > SharpOS-Developers mailing list > > Sha...@li... > > https://lists.sourceforge.net/lists/listinfo/sharpos-developers > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > SharpOS-Developers mailing list > Sha...@li... > https://lists.sourceforge.net/lists/listinfo/sharpos-developers > -- fury long name: William Lahti handle :: fury freenode :: xfury blog :: http://xfurious.blogspot.com/ |
|
From: Pieter M. <pie...@in...> - 2007-08-30 06:53:39
|
In fact, I said that because I already noticed the AOT compiler uses it. A modularized AOT would have a hard time implementing the sequence of the current AOT compiler exactly the same way. If the AOT compiler does not know what an optimization module actually does, it can't use it in custom loops. So we can implement this by executing all modular optimizations in sequence and repeating that until nothing changes. The problem I see with that approach is: say optimization 1 likes to see the code in form 1, and optimization 2 likes to see the same code in form 2. Then they will both keep changing the same instructions to their point of view. The loop will continue forever. The current AOT does not have this problem because it can choose which optimizations are looped together. Somehow we should make those modular optimizations complementary, so they don't keep undoing and redoing eachothers work. Insite Mees-Delbeke VOF Zomerstraat 29A 9270 Laarne Telefoon: 09 367 96 34 GSM: 0478 44 96 04 E-mail: pie...@in... Site: http://www.insitehosting.be -----Original Message----- From: sha...@li... [mailto:sha...@li...] On Behalf Of William Lahti Sent: donderdag 30 augustus 2007 7:49 To: sha...@li... Subject: Re: [SharpOS Developers] Proposal Yeah, I put a lot of effort into Scott's message before I realized the thread had grown larger than I thought. What Chriss said sums up much of my points. > Where optimizations in the block/SSA domain could be sequentially > executed and the sequence repeated until the code no longer changes. This is done in the AOT, but it's not readily apparent. On 8/29/07, Johann MacDonagh <jo...@ma...> wrote: > Here was my original idea for the AOT compiler. I started work on > this before Chriss came in and showed off his work. And then as time > flew by (and I was assigned to three research projects this > semester), I completely ran out of time and energy to keep this > project running. > > Anyway, my idea was very simple, but inefficient: > > If you look at the IL code, it's completely stack based. You push on > values, pop values off, different operations do different things to > the stack, etc... why not translate the IL into simple x86 stack > operations? > > For example, when you: > > int i =3D 5; > > in C#, the IL pushes that value onto the "IL stack". Why not simply > translate that into an x86 push operation? In fact, almost all IL > commands can easily be translated to simple x86 operations. The only > real issue is with the decimal type (80-bit ?), which doesn't > translate well to x86. > > The only thing we would have to standardize on is the memory layout > of an assembly. Is there a header which includes the entry points? Is > there a table which includes locations of all the classes and > methods? etc... We'd use a standard set of registers for each > operation. Meaning, when we want to do the IL add operation, we pop > the last two values from the stack off into a documented set of > registers, add them, and then push them back. As time went on, we > could combine common sets of operations for more efficiency. > > Like I said, I ran out of time and Chriss's solutions seemed to work > better at the time. Another issue was my frustration with developing > in MonoDevelop. Now I'm running Visual Studio through a VM on my > MacBook Pro (btw: svn supports conversion from CRLF's to LF's > automatically, so we can easily incorporate Windows and *nix > developers). > > Chriss's AOT compiler does a lot more than my simple compiler would > (mostly dealing with optimizations). If I had more time I'd love to > test out my idea though. > > Johann > > On Aug 29, 2007, at 2:27 AM, Chad Z. Hower aka Kudzu wrote: > > >> I don't think anyone here is questioning your contributions. I for > >> one > >> think you've done a great job with developing something like the AOT. > > > > After some pretty hefty review I have some pretty hefty thoughts on > > AOT. But > > Id like to talk to Chris 1:1 and get some more backgrounders before > > spewing > > my thoughts about. > > > > But I'll say this. AOT is core to this project. It needs to be > > modularized, > > opened up and heavily worked on. I also believe significant > > portions need > > thought about and likely redone. But if we modularize it a bit more > > and some > > of us help, the load wont all be on Chris and we can do it. > > > > > > > > > > ---------------------------------------------------------------------- > > --- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a > > browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > SharpOS-Developers mailing list > > Sha...@li... > > https://lists.sourceforge.net/lists/listinfo/sharpos-developers > > > ------------------------------------------------------------------------ - > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > SharpOS-Developers mailing list > Sha...@li... > https://lists.sourceforge.net/lists/listinfo/sharpos-developers > --=20 fury long name: William Lahti handle :: fury freenode :: xfury blog :: http://xfurious.blogspot.com/ ------------------------------------------------------------------------ - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ SharpOS-Developers mailing list Sha...@li... https://lists.sourceforge.net/lists/listinfo/sharpos-developers |
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-30 12:17:55
|
> A modularized AOT would have a hard time implementing the sequence of > the current AOT compiler exactly the same way. If the AOT compiler does > not know what an optimization module actually does, it can't use it in Modularized doenst have to mean lack of communication. |
|
From: Pieter M. <pie...@in...> - 2007-08-30 12:22:38
|
What I meant was that the current AOT uses specific optimizations in specific ways and combinations so we should consider that when designing a modular one. I didn't say it couldn't be done, it's just something to keep in mind ;) Insite Mees-Delbeke VOF Zomerstraat 29A 9270 Laarne Telefoon: 09 367 96 34 GSM: 0478 44 96 04 E-mail: pie...@in... Site: http://www.insitehosting.be -----Original Message----- From: sha...@li... [mailto:sha...@li...] On Behalf Of Chad Z. Hower aka Kudzu Sent: donderdag 30 augustus 2007 14:18 To: sha...@li... Subject: Re: [SharpOS Developers] Proposal > A modularized AOT would have a hard time implementing the sequence of > the current AOT compiler exactly the same way. If the AOT compiler does > not know what an optimization module actually does, it can't use it in Modularized doenst have to mean lack of communication. ------------------------------------------------------------------------ - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ SharpOS-Developers mailing list Sha...@li... https://lists.sourceforge.net/lists/listinfo/sharpos-developers |
|
From: Chad Z. H. a. K. <Chad@Hower.org> - 2007-08-30 12:25:22
|
> What I meant was that the current AOT uses specific optimizations in > specific ways and combinations so we should consider that when > designing > a modular one. I didn't say it couldn't be done, it's just something to > keep in mind ;) Absolutely. There is a lot of valuable stuff in AOT now and Chriss' head and Ive made him aware of that. :) |
|
From: Sander v. R. <san...@gm...> - 2007-08-30 13:50:35
|
Why is the prototype on codeplex and not on sourceforge? Not that i mind, but it's a bit odd ;) Any idea when vs 2008 will be out? ....presumably in 2008? ;) |
Thanks for helping keep SourceForge clean.
X