Over the past few days, I have made changes to the compiler, and these changes need to be tested and regression tested. This post explains that process.
You can run the same tests and use the same process to test your own programs.
Enjoy,
Evan
GCBASIC Testing
This automated testing process is designed to ensure the health of the GCBASIC compiler by attempting to build a vast library of demonstration files. It acts as a "smoke test" to verify that the compiler can still generate valid assembly and machine code across many different microcontrollers without crashing or throwing errors.
Introduction
The testcompilealldemos.bat script is a quality assurance tool used to validate GCBASIC code. You will find this in the GCBASIC folder.
Instead of testing a single program, it recursively scans a target directory for all .gcb files and runs them through the compiler. This process helps developers identify if a recent change to the compiler has accidentally "broken" the logic or syntax of existing programs.
How to Use for Overall Testing
To perform a full library test, follow these steps:
Placement: Ensure testcompilealldemos.bat and GCBASIC.EXE are in the GCBASIC folder. By default, they are installed there.
Execution: Run the batch file by double-clicking it or via the Command Line for more control.
Targeting: By default, the script is configured to look for a folder named demos\. You can also drag and drop a specific folder onto the .bat file to test just that directory.
Automatic Filtering:
The script will skip any folder containing a file named skiptest or skiptest.txt.
It will ignore files that lack a #chip directive or a valid hardware header (such as specific includes for the Uno or LGT8F328P), marking them as "Invalid Sources".
Individual files containing the text #SKIPTEST will be bypassed and logged as skipped.
You Can Use the Same Process
You don't have to limit this tool to the built-in demos. You can use it to "stress test" your own project folders or custom libraries to ensure they remain compatible with new compiler builds.
Advanced Command Line Usage:
Open a command prompt and pass a specific directory as a parameter to target your own project folder:
Efficiency: This allows the script to identify and process every .gcb file in your specified directory tree.
Defaults: If no parameter is passed, the script defaults back to the standard demos\ folder.
Clean Workspace: Regardless of which folder you test, the script automatically cleans up temporary files like compiled.asm, compiled.hex, compiled.lst, and compiled.html when it finishes.
Understanding the Outputs
After the test completes, four log files are generated to summarize the results. These files include a "Started @" timestamp at the top and a "Completed @" timestamp at the end.
Log File
Description
success.log
Lists every file that successfully produced both a compiled .asm and a .hex file.
test.log
Records files that encountered errors during compilation, including the specific error messages found in errors.txt. It also logs if a file failed to generate an ASM or HEX file.
skipped.log
Lists files that were intentionally bypassed due to folder-level markers (skiptest) or the internal #SKIPTEST command.
invalidsources.log
Tracks files that do not meet the requirements for a standalone program, such as font files, documentation, or code missing a #chip definition.
The Testing Paradox: "Immune Testing"
While this tool is powerful, users should be aware of the Testing Paradox.
Because this is a compiler-only test, it only confirms that the code is syntactically correct and the toolchain is intact. It does not guarantee that the generated code will behave correctly on a physical chip.
Undetected Bugs: If a compiler or library update changes how a specific command behaves on a hardware level, the program might still compile perfectly (yielding a "Success"), but the actual hardware performance could be broken.
Immunity: We test "known good" code, any testing approach may become immune to detecting new bugs. We regularly add new, more complex demos to the test suite ( the demos ) to "challenge" the compiler and ensure new features are being tested alongside the old ones.
❤️
1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Over the past few days, I have made changes to the compiler, and these changes need to be tested and regression tested. This post explains that process.
You can run the same tests and use the same process to test your own programs.
Enjoy,
Evan
GCBASIC Testing
This automated testing process is designed to ensure the health of the GCBASIC compiler by attempting to build a vast library of demonstration files. It acts as a "smoke test" to verify that the compiler can still generate valid assembly and machine code across many different microcontrollers without crashing or throwing errors.
Introduction
The
testcompilealldemos.batscript is a quality assurance tool used to validate GCBASIC code. You will find this in the GCBASIC folder.Instead of testing a single program, it recursively scans a target directory for all
.gcbfiles and runs them through the compiler. This process helps developers identify if a recent change to the compiler has accidentally "broken" the logic or syntax of existing programs.How to Use for Overall Testing
To perform a full library test, follow these steps:
testcompilealldemos.batandGCBASIC.EXEare in the GCBASIC folder. By default, they are installed there.Execution: Run the batch file by double-clicking it or via the Command Line for more control.
Targeting: By default, the script is configured to look for a folder named
demos\. You can also drag and drop a specific folder onto the.batfile to test just that directory.Automatic Filtering:
The script will skip any folder containing a file named
skiptestorskiptest.txt.It will ignore files that lack a
#chipdirective or a valid hardware header (such as specific includes for the Uno or LGT8F328P), marking them as "Invalid Sources".Individual files containing the text
#SKIPTESTwill be bypassed and logged as skipped.You Can Use the Same Process
You don't have to limit this tool to the built-in demos. You can use it to "stress test" your own project folders or custom libraries to ensure they remain compatible with new compiler builds.
Advanced Command Line Usage:
Open a command prompt and pass a specific directory as a parameter to target your own project folder:
Efficiency: This allows the script to identify and process every
.gcbfile in your specified directory tree.Defaults: If no parameter is passed, the script defaults back to the standard
demos\folder.Clean Workspace: Regardless of which folder you test, the script automatically cleans up temporary files like
compiled.asm,compiled.hex,compiled.lst, andcompiled.htmlwhen it finishes.Understanding the Outputs
After the test completes, four log files are generated to summarize the results. These files include a "Started @" timestamp at the top and a "Completed @" timestamp at the end.
success.log.asmand a.hexfile.test.logerrors.txt. It also logs if a file failed to generate an ASM or HEX file.skipped.logskiptest) or the internal#SKIPTESTcommand.invalidsources.log#chipdefinition.The Testing Paradox: "Immune Testing"
While this tool is powerful, users should be aware of the Testing Paradox.
Because this is a compiler-only test, it only confirms that the code is syntactically correct and the toolchain is intact. It does not guarantee that the generated code will behave correctly on a physical chip.