Share

Kokogut - a compiler of Kogut

File Release Notes and Changelog

Release Name: 0.7.0

Notes:
Please download kokogut-0.7.0-boot.tar.bz2 unless you already
have Kokogut installed in version 0.6.0 or later.

Changes: A cleaned up release: - Changed the syntax: * 'if' and 'ifDefined' are written without 'else', and are limited to one or two branches: - if cond {expr1} - if cond {expr1} {expr2} Alternatively any number of cases can be written in brackets, with '_' replacing 'else'. If no case matches in the bracketed variant, 'if' fails, and 'ifDefined' is an error. * 'def' is used for functions, 'method' for methods; 'let' remains only in the form 'let pat = expr' * Dispatched parameters of generic functions are written without parentheses: - def DoRemove coll! key {}; and types of specialized parameters of methods are written with '^' instead of '!', also without parentheses: - method DoRemove coll^HASH_DICT key {...}; * Function cases are put in brackets if there is more than one case. The opening bracket may be prefixed with a label, which specifies an internal name for recursive calls, which is 'again' by default. * Constructors are written without the equal sign between the type name and the constructor name. * A constructor name followed by brace-enclosed definitions defines a singleton object with an explicitly specified behavior. * 'subtypes T {definitions}' makes an existing type the implicit supertype of definitions. * '~' has a high priority, equal to 'label:', but when it's used before a function being applied, it covers the whole application. * A space in a string can be escaped by "\s" instead of "\ " (reverted the change made in version 0.6.1), because a backslash followed by spaces and then a newline should be ignored. - New functions: * ReverseGen, IterateBack * DoReject * TryRemove, TryCut * GetLast, TryCutLast * PartGen{,Back}, IteratePart{,Back} * EndsWith * Size{Is,AtMost}, Count{Is,AtMost,AtLeast} * MapDictPartial * SortGen{,By} - incremental sorting * Shuffle{,List,Gen} in module Random * FillInf, FillNewInf * Variable{Byte,Char}{Input,Output} - change the underlying stream at any time * Partial{Byte,Char}Output - shield lower layers from flush:#end and closing * WantGarbageCollect in module Threads * With{,Cancelable}AsyncExit in module Threads * Ignore - ?_... {} * Apply - ?f xs... {f xs...} * Unsafe{Min,Max}{,Int}, UnsafeVector in module Unsafe * EncodeInt, DecodeInt - conversion between ints and byte arrays * WriteStackTrace - Removed functions: * IsNull * JoinString * MapNext * MapPairs, MapPairsGen * Join * FinalizeWeakRef - use Close - Renamed functions: * Is -> HasType * IsTrue -> Identity * IsFalse -> '~' * Using_ -> Ensure * TouchObject -> Touch * JoinGen -> Join * ViewSequence -> PureIterator * Intersect -> Intersection * {BYTE,CHAR}_APPEND_INPUT -> APPEND_{BYTE,CHAR}_INPUT * {Byte,Char}AppendInput -> Append{Byte,Char}Input * {BYTE,CHAR}_COPY_INPUT -> PEEK_{BYTE,CHAR}_INPUT * {Byte,Char}CopyInput -> Peek{Byte,Char}Input * {BYTE,CHAR}_BUFFERED_{INPUT,OUTPUT} -> BUFFERED_{BYTE,CHAR}_{INPUT,OUTPUT} * {Byte,Char}Buffered{Input,Output} -> Buffered{Byte,Char}{Input,Output} * {BYTE,CHAR}_FILTERED_{INPUT,OUTPUT} -> FILTERED_{BYTE,CHAR}_{INPUT,OUTPUT} * {Byte,Char}Filtered{Input,Output} -> Filtered{Byte,Char}{Input,Output} * Escape -> WithExit * {,Cancelable}TimeLimit -> With{,Cancelable}TimeLimit * FileStatus -> TargetFileStatus * FileOrLinkStatus -> FileStatus * NamedFunction -> NamedCommand in module TextUI.ReadLine - Changed standard equality predicates: * 'Is' is strict equality. It compares the immutable structure of two objects. It differs from 'IsSame' only by not distinguishing the identity of value objects. An object is always equal to itself, even NaN, and objects with different types are always different. '0.0 ~%Is (-0.0)'. Dictionary lookup, 'IsIn', 'IsEqual', and literal patterns use 'Is'. A pattern 'is expr' is equivalent to 'when Is expr'. 'Is' is dispatched on the first argument only. The cases when both arguments are the same object or when they have different types are handled before dispatching. The default interpretation of 'Is' for types without a specialization is 'IsSame'. * '==' is arithmetic equality. It is defined for types which define '<', and for complex numbers. The definition should be compatible with '<' and '<='. - Removed 'when', 'where', and 'is' patterns. Changed the syntax of 'if', 'match', and 'define' patterns to include a pattern to match the subject against: * 'pat1->if cond' * 'pat1->match expr pat2' * 'pat1->define {def}' New patterns: * 'pat1->fun args...' (with an arbitrary predicate) - matches when the subject matches 'pat1', and 'subj->fun args...' is true. * 'pat1->fun args... = pat2' (with an arbitrary function) - matches when the subject matches 'pat1', and 'subj->fun args...' matches pat2. * 'pat1->with fields' - matches when the subject matches 'pat1', and each field matches. A field is either 'label:pat' which specifies the symbol and a pattern to match the field value against, or a name which specifies both at once. The argument before '%op' or '->fun' can be omitted, defaulting to '_'. This applies both to expressions (i.e. partial application) and patterns. If the keyword or function is negated with '~' in patterns with 'if', 'match', or a function, the negation doesn't apply to matching the subject against 'pat1'. For example 'x ~%Is 0' binds the subject to x if it's not 0, while '~(x %Is 0)' succeeds when x is not 0 but doesn't bind anything. - Using/4 and Ensure/4 have changed the argument order: obtain commit rollback body. - The first argument of Ensure is optional, defaulting to {}. - Using and Ensure: if the body fails, and releasing the resource fails too, the exception from the body has priority and is rethrown. CloseAll: if several actions fail, the earliest exception has priority. - Revised the rules of signal blocking: * The blocking state is still an (all, async) pair, but 'all' may no longer be infinite. * Explicit changes of the blocking state: BlockSignals increases 'all', UnblockSignals decreases 'all', BlockAsyncSignals sets 'async' to 1, UnblockSyncSignals sets 'async' to 1 and decreases 'all'. * Trying to unblock signals which are already unblocked fails. This is most likely an program error. * Signal handlers, exception handlers, and the 'commit' part of Using and Ensure increase 'all' by 1. The 'rollback' part of Using and Ensure increases 'all' by 2. In the case of an exception handler the state is changed wrt. the context of 'try'. * ThreadBlockSignals starts with all=1 and async=0. * Removed SignalsBlocked and AsyncSignalsBlocked. * When relocking a mutex or read-write lock with signals unblocked, signals are processed until a signal handler fails. - FindPart returns the haystack size on failure instead of Null. - Renamed 'start' to 'begin' when talking about index ranges, in particular in fields of 'ViewPart'. - EnvironmentVar{,s} renamed to EnvVar{,s}, moved from module System.Environment to Prelude, and environment variable names have been changed from strings to symbols. - Renamed Unix functions to be more consistent with Unix names or with one another: * CreateDirectory -> MakeDirectory * DeleteFile -> RemoveFile * SymLink -> MakeSymLink * ReadLink -> ReadSymLink * GetCurrentDirectory -> CurrentDirectory * SetCurrentDirectory -> ChangeDirectory * IsFile -> IsRegularFile * DirectoryContents{,Gen,Lazy} -> FileList{,Gen,Lazy} * contents of modules System.Directories and System.FileStatus moved to module System.Files - ICONV_ENCODING and IConvEncoding moved from Prelude to Encodings.IConv module (Core library). BlockingMode moved from Prelude to System.Files module (Unix library). - CopyStream function has been split into CopyByteStream and CopyCharStream. - New functions in module System.Files: * Make{Link,FIFO,Node} * Change{Mode,Owner,Time} - New library LZMA, providing LZMA compression scheme implemented by Igor Pavlov, used in 7zip. - When there is a name conflict between an implicitly imported definition and some other definition, the explicit definition is preferred and this is not an error. A note is shown though. - '|' patterns bind names which are common to subpatterns. - 'HasType' accepts multiple types and is true when the object matches any of the types. - Method definitions accept ','-separated sets of types. Multiple methods are defined, for each combination of the types. - 'private' and 'public' can be prefixes of definitions, e.g. 'private def F() {...};'. - Added macros inc and dec: 'inc v d' is 'v = v + d' (but v is computed only once), 'inc v' is 'inc v 1', and dec is analogous with '-'. - First-class modules are created by modules themselves instead of each time they are used. The format of module contents has been changed. - Expression of the form 'module Module.Name' evaluates to the module object of the given module. It can be used even before this module has been compiled. - Items of registered lists are unregistered only with 'Close' on the key returned by 'Register'. Applying the key is no longer supported. 'RegisterEvent' returns an event object; it can be unregistered with 'Close' instead of by applying it directly. - Old weak references has been split into two concepts: weak references and finalizers, with a changed semantics. 'WeakRef' - create a weak reference from a key object, a value object (which by default is the same as the key), and a close function. A weak reference by itself doesn't keep its key nor value alive. It enforces a relationship: as long as the weak reference and the key are alive, the value is kept alive too. 'weakRef absent present' returns 'present value' if the key is alive or 'absent()' otherwise. By default 'present' is 'Identity' and 'absent' is '{Null}'. If the key dies before its weak reference dies, the associated close function is scheduled for execution: it will be applied to the weak reference. It is meant to remove the weak reference from a data structure it is contained in. If the weak reference dies before the key dies, the close function is not executed. 'Finalizer' - create a finalizer from a key object and a close function. A finalizer doesn't keep its key alive; it is kept alive by the key. When the key dies, the close function is scheduled for execution: it will be applied to no arguments. The 'Close' method of a finalizer executes its close function immediately (or waits until it finishes if it's currently being executed), making the finalizer inactive. Collecting long chains of weak references is more efficient: O(N) rather than O(N^2). This change was inspired by http://www.haible.de/bruno/papers/cs/weak/WeakDatastructures-writeup.html but uses a different algorithm. Differences from the previous design: A weak reference is not kept alive by its key. A finalizer doesn't have an associated value and cannot be used to obtain a value. Explicit finalization is supported only for finalizers. Killing (deactivation without executing the close function) is not supported for either type. The close function of a weak reference gets the weak reference as the argument. A consequence of the change is that a value at a live key in a weak dictionary, if it refers to the weak dictionary itself, no longer keeps the dictionary alive. - Spelling "FileName" changed to "Filename" in ReadLine functions: FilenameCompletionFunction, FilenameQuoteCharacters, FilenameCompletionDesired, FilenameQuotingDesired, FilenameQuotingFunction, FilenameDequotingFunction. - Find and FindBack accept multiple sequences. The starting index is mandatory in this case. - Implemented Union, Difference, and Intersection. - Functions First, GetFirst, RemoveFirst, DoRemoveFirst, DoCutFirst, TryCutFirst are applicable to all collections, not just sequences, and in the case of sets and dictionaries operate on an arbitrarily picked element. - ReverseList can be applied to two collections: ReverseList x y == List (ReverseList x) y - Conversion of collections to strings and to character arrays flattens them recursively. - Generators and lazy lists are compared and hashed by elements. - Get, GetFirst, and TryCutFirst don't have separate specializations without explicit 'absent' argument. IsEqual doesn't have separate specializations without explicit element equality predicate. Sort doesn't have separate specializations without explicit element comparison predicate. List doesn't have separate specializations without explicit 'rest' argument. - Functions Min and Max are separately dispatched generic functions instead of always using '<'. When applied to numbers, they should be understood as arithmetic operations. When any argument is inexact, the result is inexact as well. They return the correct answer for 0.0 and -0.0. When any argument is NaN, the result is NaN. - GCD takes an arbitrary number of arguments. - The default Keys implementation is supposed to be the only one: it's now a constructor instead of a generic function defaulting to applying a private constructor. - IsOdd doesn't have a separate specialization but is defined as ~IsEven. - New syntax of expressions: 'match expression pattern' returns True if the expression matches the pattern. - Partial application is supported by '[]', '\', ',', 'if', 'case', 'match', and 'loop'. - New type of variable 'forward x', which may not be read until assigned, and which may be assigned only once. - A dynamic variable with no initial value has no global value at all instead of being initialized to Null. - Unused names and imports are noted. A note is like a warning but doesn't cause a non-zero exit code. - The 'Touch' function ignores its argument. It has three roles: * Silence the note about an unused name. * Ensure that the object is not garbage collected at this point yet (old 'TouchObject'). * Ensure that the name is available to C code. - When a multithreaded program is exiting, all other threads are canceled and waited for, and GC is performed. - When a thread is about to be garbage collected, it is kicked using ThreadKicked signal, so it has a chance to clean up. Threads with signals blocked are garbage collected silently. - ThreadBlockSignals blocks the signals once instead of absolutely, so BlockSignalsLess in the body unblocks them. - Event queues: EventReady can be called in two ways: * EventReady commit * EventReady commit rollback The commit function is executed when this event is taken, and the rollback function is executed when it is determined that it will not be taken. The rollback function may call EventNotReady if it determines that the event is no longer ready, i.e. that something has happened between it has notified that it's ready and it has been chosen as the first event to be taken. The argument of EventNotReady is a function which continues waiting. - The last argument of Flush{,To} is now optional, defaulting to #sync. - Supertypes of exceptions are used more consistently: * PROGRAM_ERROR - should not happen in a correct program, a library is used in a wrong way * EXTERNAL_ERROR - data given to the program is corrupted or the environment is uncooperative * RESOURCE_ERROR - an operation needs more resources than are currently available * EXIT - a non-local exit is in progress * SPECIAL_RESULT - this exception should be catched immediately by the context, it carries out-of-band information - Object ids are ordered. The order corresponds to the time of their creation. - ObjectId() creates an anonymous object id, not associated with any object, but hashable and ordered like other object ids. - Foreign threads are now more robust. Renamed KO_{UNBLOCK,BLOCK}_OTHER_THREADS* to KO_{ENTER,LEAVE}_FOREIGN*. Removed KO_{UNBLOCK,BLOCK}_OTHER_THREADS_OR_{BLOCK,UNBLOCK}_TIMER. KO_{ENTER,LEAVE}_FOREIGN* can be used instead, which now avoid physical creation of a new thread if there are no other threads which are running, waiting for I/O, sleeping, nor handling system signals. It is allowed to call KO_LEAVE_FOREIGN in an OS thread which has not been created by Kogut. The thread is adopted as a Kogut thread, with signals unblocked. The same Kogut thread is used to represent this thread until it finishes. New functions ko_{disallow,allow}_foreign_no_signals for disallowing other threads temporarily to just acquire access to the runtime, without running Kogut code. Mismatched KO_{ENTER,LEAVE}_FOREIGN* are detected at runtime when possible. - c_eval has an optional name of the result, written before inputs. If present, the result should be stored in this C variable, which is protected against GC. If absent, result should be stored into the unprotected KO_RESULT as before. - FileExists, IsRegularFile and IsDirectory use lstat instead of stat. - Unneeded symbols are freed from memory. - Local variables which become dead at a particular point of code are cleared, so the garbage collector doesn't unnecessarily retain objects they pointed to. - Renamed some environment variables controlling the runtime: * KO_HEAP_LIMIT_RATIO -> KO_GC_FACTOR (default: 1.5) * KO_MIN_HEAP_LIMIT -> KO_GC_SIZE (default: 4M) * KO_HEAP_SIZE_RATIO -> KO_GROW_FACTOR (default: 1.5) * KO_MIN_HEAP_SIZE -> KO_GROW_SIZE (default: 4M) * KO_TICK_FREQUENCY -> KO_TIMER (default: 100) - Improved detection of low memory condition: * If more than KO_MAX_MEMORY (default: RAM minus 64M, or half of RAM, whichever is larger) is used, OutOfMemory is signaled. This replaces KO_MAX_HEAP_SIZE which measured live heap size rather than allocated memory. * If more than KO_MIN_MEMORY (default: 1/4 of RAM) is used, and less than KO_MIN_FREE_MEMORY (default: 64M) is going to remain free (including buffers and swap), OutOfMemory is signaled. - KO_SHOW_STATS=1 and Kogut.Stats include GC time and maximum GC pause as both real and CPU time. - On Linux Kogut.Stats includes SystemStats which returns information from sysinfo(), mostly memory sizes. - If the 'inexact' option of Show is not Null, it means to show a point when the number is not shown exactly, rather than when it does not have an exact type. - 'precision' and 'fracPrecision' options for Show may be 'Inf'. - New show option showPrecision:True|False, default False - whether to add trailing zeros to suggest the precision of inexact numbers. - Conflicts in serialization unique names are reported. - When an object could not be serialized, a deserializer will fail in a predictable way. - Names of types, functions, and singletons are symbols, not strings. - Added functions which allow to locally extend or override serialization rules: * DefineSerializationRules {changes} - package serialization rules, expressed by executable code which defines them, into an object. * UseSerializationRules rules {body} - apply the packaged serialization rules during evaluation of body. - RecordSerialization accepts an optional argument: list of field names. If it's present, the type doesn't have to be a record type. - Changes in the Python binding: * RawPyCall, RawPyGetAttr, and RawPyGetItem are almost the same as the ordinary application and '@', but wrap the resulting Python object in Kogut without trying to convert it. * Removed RawPy{Set,Has,Del}Attr, RawPySetItem, PyDef{Class,Init,Method}. * Renamed PyIs -> PyHasType, PyExceptionIs -> PyExceptionHasType. * Python types are called PyNONE, PyINT, PyLONG, PyFLOAT, PyBOOL, PySTRING, PyUNICODE, PyTUPLE, PyLIST, PyDICT, PyTYPE, PyOBJECT, PySUPER, PyCLASS, PyINSTANCE, PyMETHOD, PyCLASS_METHOD, PySTATIC_METHOD, PyPROPERTY. * New types PySET, PyFROZEN_SET, PyENUMERATE, PyREVERSED, PySLICE. * Constructors of Python types have a Kogut-like interface. They are called PyInt, PyLong, PyFloat, PyString, PyUnicode, PyTuple, PyList, PyDict, PySet, PyFrozenSet, PyEnumerate, PyReversed, PyClass, PyMethod, PySlice. * New functions Py{Get,Set,Has,Del}SpecialAttr, PyCallSpecialMethod, PyType, PySymbol, ByteArrayToPyString, PyStringToByteArray. * Python exception types are available to Kogut: PyVALUE_ERROR etc. 'Fail (PyException type value)' raises a Python exception. * Attributes of Kogut objects as seen from Python don't necessarily correspond to Kogut fields, but can be defined per Kogut type with PyDef{Get,Set,Del}{,Special}Attr and PyDef{,Raw}{,Special}Method. * PyImport and PyAddModule take symbols, not strings. * Translation of collection interfaces between Python and Kogut is more consistent and more complete. In particular the semantics of iteration over dictionaries respects the convention of the language (key/value pairs in Kogut, keys only in Python). * Serialization / pickling works across languages. Detection of sharing works only within a subgraph of objects of one language though. - Python code can run concurrently with Kogut code. Python code may be called from multiple Kogut threads and vice versa. It is now safe to call Kogut from __del__ methods of Python objects. Switching between Python and Kogut is much slower however, and minimal supported version of Python is now 2.3. - Fixed a few issues with Python 2.5. - Perl code can run concurrently with Kogut code. Perl can only be called from a single thread. - KeyNotFound exception includes the key. - 'local', 'try', and 'handle' don't require braces in their bodies, and in the successful continuation of 'try'. These arguments may be arbitrary expressions, where braces denote a function and thus have the same meaning as before. - Names which are defined but not used are reported, unless they start with an underscore. - Stack traces include exceptions thrown during stack unwinding. - RunProgram passes the exception to the parent process if the program could not be executed. - Improved handling of the non-blocking mode of standard streams. If pthreads are used, the mode is queried at startup and is used to choose the method of ensuring that only the current thread is blocked by I/O. If pthreads are not used, standard streams are made non-blocking, and the original mode is restored before exiting or before ExecProgram (except when the current process is a child of ForkProcess{,CloneThreads,KillThreads}). - Unicode character database updated to Unicode 5.0.0. - New example: Connect Four game (koc4). - kocals supports variables, named functions, conditionals, comparisons. New kocalc constants: inff, nan, e. New functions: min, max, isdivisible, iseven, rationalize, isnumber, isnull, ispair, isbool, isexact, isfloat, isfinite, isinfinite, isnan, ratio, float. Running kocalc with arguments evaluates them and exits. - Small examples moved from directory tests to examples/small. - New small examples: KnightsTour, Sudoku. Sudoku was translated from a Lisp program by Frank Buß <http://www.frank-buss.de/lisp/sudoku.html> - ctags is a ctags config file to be put into ~/.ctags or included from it. - 'kokoc --depend' uses the real parser of the compiler instead of matching lines with regexps. - kokoc accepts C++ sources (extensions .cc, .cxx, .cpp, .c++). Kokogut headers are compatible with C++. Communication between Kogut and C++ must be done through extern "C" functions. - kogut-mode.el: M-q fills paragraphs in comments and strings. Automatic indentation reimplemented with slightly different rules. Local variables which look like standard macros but are not applied to arguments are highlighted as regular names. - Changed hash formula for integers. - Changed serialization format. - More efficient code is generated: lots of minor improvements, in particular related to loops (lambda-lifting, tighter tail recursion, reduced chained calls), mutable variables (they are not materialized as objects when possible), generic function dispatch (inlined code), objects with lots of fields (symbol lookup) etc. Option -O affects the generated C code by turning on some optimizations; the code may be longer. -O and -O1 through -O9 are equivalent (though they might be different for the C compiler), -O0 reverts to the default. - Fixed incorrect code generated by gcc-4.2.0 (gcc-4.1.0 was fine), by reducing pointer type punning and making one function not inline. Generated code still violates strict rules of pointer aliasing in C, but this would be hard to avoid, and it works now. - In version 0.6.1 the runtime was being compiled without optimization. - Bugs fixed: * Crash when a GC occurs during creating an IConv encoder or decoder. * Sorting was not stable. * Find for LIST with a non-zero starting index was wrong. * Declared supertypes are checked for being actually types. * Fixed QuotRem, Mod, and DivMod for mixed ints and floats. * Fixed IsDivisible when the second argument is a literal power for 2. * Fixed adding an already existing element to HASH_SET_BY. * Pipes created for redirected processes are set to blocking mode. * The timer is recreated after ForkProcess{,CloneThreads,KillThreads}. * CPU time statistics have overflowed after 36 minutes.