For technical support, please e-mail tombo@gekkoware.com
My goal is to make a useful compiler generation tool for the international community.
I have always been interested in developing cost effective, custom languages to meet the needs of a user community.
I am greatly encouraged by the geographic and cultural diversity of people downloading this code.
Over time, I hope that inspired compiler developers will be able to develop custom compilers to
meet the need of their users.
As a language designer I am facinated by how languages can shape thoughts.
Fortran users view matricies in ways that a C programmer will never understand.
As you design your languages, you will make decisions about how your users will express concepts.
Some concepts will be expressed easily, others concepts may be very awkward.
Sometimes a single word in your language can express the aspirations of an entire culture.
-------------------------------------------------------------------------------------
The Tombo Compiler Generation Toolkit consists of documentation, a compiler generator, a class
library to support compilation, and sample programs. It is distributed in the Unix archive file,
tombo.tar.gz.
Before unpacking the source, read the license agreement below and agree to its terms. By
unpacking the source, you are agreeing to the licensing terms.
Unpack the source using the following command:
untar Tombo2.0.0.tar.gz
tar -xvf Tombo2.0.0.tar.gz
-------------------------------------------------------------------------------------
Version 2.0.0 Release notes
This realease is intended to:
- Release a joint Visual Studio/Unix/Mac "Solution"
- Improve library structure so that relationships among modules is more appearent.
- Clean-up character string processing for better extended character (e.g. Unicode) support
- Reorganized modules for better flexibility and more consise functionality
!!! Special Note: There are source code changes to code developed
under version 1.0.0 that need modifications to migrate to version 2.0.0.
> Visual Studio users will get an error when compiling T_TEXT.cpp.
These users will need to edit a #define snprintf directive that is commented out near line 20.
This will properly define the _snprintf method.
> Removed control character transtlation in the Symbol Table.
Text retrieved from symbol table will match text from original find.
This ensures that there is always a "1 to 1 relationship" between a text object and
its numeric index value.
> Renamed many classes to clarify relationships between class members
and classes in the translation heirarchy.
The focus is on the byte stream to token and token back to byte stream conversions.
Thus, "your" localized byte streams and grammars can be processed natively by Tombo.
My goal is that you are only restricted by "C" language restrictions on action names.
All of the members of a class are named with a prefix indicating the function of that class.
The first stage is to convert input byte streams into tokens.
The class names are organized as follows:
Bytes to Tokens:
T_BSRC: Byte stream source
T_XSRC: Translate byte streams into extended characters. (i.e. extended character source)
T_TSRC: Translate extended character streams into token streams
Token sources (T_TSRC) are connected to a token sink (T_TSINK).
Parsers are derived fron the T_TSINK class.
Parsing actions are derived form the T_EMITTER class which is derived from the T_TSRC class.
The final action class is attached to a T_TSINK class which supports the Run method to force parsing.
Prior to final output, the compilation process uses the symbol numbers contained in tokens
to build up the text objects for final output.
The final stage is to convert these text objects back into a properly encoded byte stream.
Symbol Text to byte stream sinks:
T_BSRC: Byte stream source
T_XSRC: Translate byte streams into extended characters. (i.e. extended character source)
T_TSRC: Translate extended character streams into token streams
Since the conversion between bytes and extended characters, the encoder and decoders are packaged
in the same module with the following prefix:
T_X: Encoding/Decoding specific module (e.g. T_XUTF8 to process UTF8 encodings)
> Added serialization and deserialization procedures for T_TEXT class.
> Simplified couplings among components to reduce dependancies.
> Cleaned up T_TSRC (Token Source) processing to support multiple source inclusion
Allow multiple intermediate sources for #include and #define reference processing
T_TSRC is a stand-alone class that provides a stream of tokens.
On End of Source, an optional End of Source token can be generated as the final valid token.
All tokens beyond the End of Source will be UNDEFINED and likely halt processing if used.
T_TSRC objects are attached to a T_TOKEN_BASE object which acts as a token consumer
access point. In addition, it contains the token mapping table for mapping a
token symbol to a token type. This mapping table is the mechanism for translating
tokens associated with reserved words and predefined operators into tokens having
the proper token type.
T_TSRC objects are created via a "new" operator, used, and finally discarded by a
T_TSINK object when it is no longer useful.
The T_TSINK object provides a termination point for the final T_TSRC object.
Tokens are supplied to a token processor attached to a sink for parsing, display, or
quiet cosumption.
Usually, a parser acts as a token sink. Thus, the T_PARSER_CFG class is derived from
the T_TSINK class.
This object supports a Run procedure for token consumption and a Dump procedure for
displaying generated tokens by a particular compiler stage.
-------------------------------------------------------------------------------------
Version 2.1.0 Release notes (Minor fixes)
Restructured some of the error processing for better error detection.
Removed special error terminal symbol processing. An error terminal is just another
terminal in the language.
Added backslash encapsulation of hex codes for extended character set in reserved words.
Backslash followed by hex characters form an extended character in a reserved word.
A double backslash is converted to a single backslash character.
-------------------------------------------------------------------------------------
Version 2.1.1 Release notes (Minor fixes)
Implemented secordary token source processing to support features such as #include
-------------------------------------------------------------------------------------
Version 2.1.2 Release notes (Minor fixes)
Improved parser start up to peek at first token to set up current and next token.
This ensures that the parser has valid tokens for emit procedures on start-up and
the token source queue has original contents.
Improved end of source handling
Implemented command line token source to facilitate command line parsing.
-------------------------------------------------------------------------------------
Tombo Compiler Generation Toolset License Agreement
Developed by Gary Block, Gekkoware Systems
Copyright (c) 1992, Gekkoware Systems
Copyright (c) 2003, California Institute of Technology ("Caltech")
Copyright (c) 2012, California Institute of Technology ("Caltech")
U.S. Government sponsorship acknowledged.
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted
provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this list of conditions
and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this list of conditions
and the following disclaimer in the documentation and/or other materials provided with the
distribution.
- Neither the name of Caltech nor its operating division, the Jet Propulsion Laboratory, nor the
names of its contributors may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
- PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY,
- WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-------------------------------------------------------------------------------------
Hozho,
Gary
