I have been advancing fairly quickly on P6 extended features.
Of late, the code base has been going instable.
Why?
At the close of Pascal-p5 1.3, I went through a full testing sequence, which includes a full PRT (rejection test) and a self compile, where P5 is hosted onto itself.
Since starting P6, I haven't been running the PRT or the self compiles. I believe as a result, the code has been slowly loosing stability. I believe this is because of the nature of Pascal and the host compiler, and that is an interesting object lesson that I don't believe I want to repeat.
GPC is the current host compiler. In ISO 7185 mode it is currently the best option available, and the version choosen was the best version available. The difference (functionally speaking) between it and P5 is that P5 attempts to do as much error checking as possible, on the level (or exceeding) that of one of the "super lints" that make the rounds for C in corporations.
Thus it is no surprise, and hopefully no offense is given, when I say that P5 already performs more complete checking than any other Pascal compiler in current use (and I include my own in this, IP Pascal). It checks for undefined (never accessed) memory locations, pointers that were reused after being disposed of, variants referenced that are inactive, and many other cases.
The reasons I haven't been running these tests for P6 are:
The PRT is fairly work intensive to check it ran properly. P6 changes the output considerably, and thus can't be automatically compared (and I even tried to increase the automation a bit to cover it).
The self compile takes a LONG time. I usually run it overnight. I have seen it take a full day (24 hours) to complete.
I have been running the PAT (Pascal Acceptance Test) on each commit, since that is fairly rapid. In fact, I run it for each of the pint, pmach, and cmach cases (different backend interpreters).
The PRT checks for how the compiler handles bad sytax and other issues in the front end. I think not running that has allowed several handling errors to creep into the front end.
The self compile gives the compiler and interpreter the benifit of the extensive checking that is available with the compiler/interpreter itself. That is, even though it runs slowly, it performs a more extensive check on itself than GPC (or other host) can do.
Thus I decided a while ago to stop and get my house in order. The PRT was revamped and passed for P6. I completed the first stage of the self compile (pcom compiles itself and runs on pint), and am going through the fixes on the second stage (pint compiles on pcom and runs on pint).
What else?
From IP Pascal comes a test I call "the spew test", it is based on the idea that you can add repeatable faults to a given Pascal source code. To do this, it counts though all of the characters in the source file, and either:
Subtracts a character.
Adds a character, usually a non-valid pascal character (like "%").
Then it compiles the file and monitors the output for:
Compiler faults.
Crashes (failure to complete).
This is actually a fairly good stability test, abet only for the front end (pcom). It is not a long program, and I will have to rewrite it in C, since the original is written IP Pascal (Pascaline), which P6 will execute one day, but presently does not fully implement. I had planned to wait on that, but I believe the need is now, and the program is not large.
Moral of the story
The first moral is TEST. Test always. Perform the most complete test suite possible. Any program can just be written and then run. The difference between that and a reliable TOOL is testing, testing and more testing. Then perhaps more testing.
The second moral is about the P5/P6 system itself. This system does a SERIOUS amount of checking on the correctness of the running program. In fact, there are only a few items left that aren't checked (7 in fact: See ticket 16 here). Unlike other languages like, say, C, Pascal is designed to be checked completely.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So if you read the commits, you know that:
Why?
At the close of Pascal-p5 1.3, I went through a full testing sequence, which includes a full PRT (rejection test) and a self compile, where P5 is hosted onto itself.
Since starting P6, I haven't been running the PRT or the self compiles. I believe as a result, the code has been slowly loosing stability. I believe this is because of the nature of Pascal and the host compiler, and that is an interesting object lesson that I don't believe I want to repeat.
GPC is the current host compiler. In ISO 7185 mode it is currently the best option available, and the version choosen was the best version available. The difference (functionally speaking) between it and P5 is that P5 attempts to do as much error checking as possible, on the level (or exceeding) that of one of the "super lints" that make the rounds for C in corporations.
Thus it is no surprise, and hopefully no offense is given, when I say that P5 already performs more complete checking than any other Pascal compiler in current use (and I include my own in this, IP Pascal). It checks for undefined (never accessed) memory locations, pointers that were reused after being disposed of, variants referenced that are inactive, and many other cases.
The reasons I haven't been running these tests for P6 are:
I have been running the PAT (Pascal Acceptance Test) on each commit, since that is fairly rapid. In fact, I run it for each of the pint, pmach, and cmach cases (different backend interpreters).
The PRT checks for how the compiler handles bad sytax and other issues in the front end. I think not running that has allowed several handling errors to creep into the front end.
The self compile gives the compiler and interpreter the benifit of the extensive checking that is available with the compiler/interpreter itself. That is, even though it runs slowly, it performs a more extensive check on itself than GPC (or other host) can do.
Thus I decided a while ago to stop and get my house in order. The PRT was revamped and passed for P6. I completed the first stage of the self compile (pcom compiles itself and runs on pint), and am going through the fixes on the second stage (pint compiles on pcom and runs on pint).
What else?
From IP Pascal comes a test I call "the spew test", it is based on the idea that you can add repeatable faults to a given Pascal source code. To do this, it counts though all of the characters in the source file, and either:
Then it compiles the file and monitors the output for:
This is actually a fairly good stability test, abet only for the front end (pcom). It is not a long program, and I will have to rewrite it in C, since the original is written IP Pascal (Pascaline), which P6 will execute one day, but presently does not fully implement. I had planned to wait on that, but I believe the need is now, and the program is not large.
Moral of the story
The first moral is TEST. Test always. Perform the most complete test suite possible. Any program can just be written and then run. The difference between that and a reliable TOOL is testing, testing and more testing. Then perhaps more testing.
The second moral is about the P5/P6 system itself. This system does a SERIOUS amount of checking on the correctness of the running program. In fact, there are only a few items left that aren't checked (7 in fact: See ticket 16 here). Unlike other languages like, say, C, Pascal is designed to be checked completely.