Recent changes to The CAN Interface

The CAN Interface modified by Peter Vranken

Peter Vranken — Wed, 18 May 2016 21:04:17 -0000

--- v9
+++ v10
@@ -1,594 +1 @@
-# CAN Interface #
-
-Table of contents
-
-[TOC]
-
-This open source project proposes a CAN interface for embedded
-applications. Although it meets a lot of demands of real applications,
-it's still considered a suggestion only; at many points it can be tailored
-or conceptually changed and a claim of this project is that its tools and
-code elements will still be beneficial even in these cases. It's not a
-fixed solution but a framework for modelling the best suitable solution in
-a particular environment.
-
-The offered solutions capture the parts of the CAN stack from the network
-layer till the application interface. An interface for sending and
-receiving CAN frames is required at the bottom and a race-condition free,
-signal based application code interface is offered at the top; the
-application interface can be data or function based and it offers not only
-the signal values but also a detailed transmission status, including bus
-status, timeout information and sequence counter and checksum validation
-results.
-
-## Concept ##
-
-The basic concept of the interface is the dispatcher engine. Such an
-engine has an event queue. Events are (usually but not necessarily)
-related to a frame and they are delivered to the handler of that event. In
-a typical environment the input of the queue will be connected to a system
-CAN interrupt, be it for notification of a received frame or a send
-acknowledge. The output of the queue is read by the main function of the
-dispatcher engine. The call of this function will usually be made from an
-application task, thus not in the context of the notifying interrupt. The
-main function delegates the events to the registered handlers and since
-this is all done in the context of the application task are there no
-further race-conditions. The (thread-safe implemented) queue decouples the
-interrupt context from the application task.
-
-Any number of dispatcher objects can be created. An addressed use case is
-an architecture with more than one (regular) application task and a
-dispatcher for each task. Each CAN frame is connected to the appropriate
-dispatcher. The association will be based on functional aspects and is
-made by the application code at initialization time. The frame will later
-be handled in the application task, which owns the associated dispatcher.
-It's also possible to associate dispatchers with physical CAN buses
-(normally CAN buses will share a dispatcher) or with kinds of events, e.g.
-frame reception events opposed to bus state change events opposed to frame
-send acknowledge events.
-
-The dispatcher engine including the necessary registration process for
-frames and events is available as hand coded C source code. It has a
-well-defined API to both sides (system and application code) and is
-considered a fixed code core. However, even for this core implementation,
-tailoring is possible. In principle, the other elements of this project can
-still be quite beneficial if the dispatcher core is not used at all. And if it
-is applied it could undergo some tailoring, most likely for sake of
-optimization. In the current release the code is designed for execution
-speed but with some simple, straightforward changes the trade off can be
-shifted towards less RAM consumption.
-
-The application interface of the dispatcher is callback based. A received
-event is notified by invoking the registered callback. From within this
-callback a set of API functions is available that permits to query all
-information about the event. In the first place the contents of a frame
-can be fetched in case of a reception event. The second element of this
-API is a couple of timer functions. Timer objects can be created, started,
-stopped, re-triggered, etc. and they will create another event at due
-time. Using timers the callback can easily implement timeouts for frame
-reception or different simple or complex send patterns for outbound
-frames.
-
-It's important to understand that the core implementation ends with the
-invocation of the callback. It's a matter of application design, what the
-callback does. This starts with the decision if there is one shared
-callback for all frames or for all kinds of events or if individual
-callbacks are used. In the former case large switch cases and/or nested
-data structures will be applied and in the latter case many different but
-similar functions will be needed.
-
-The intended use case of our open source project is that all the code or
-data structures, which is between the dispatcher's API and the interface
-with the functional application code and which dependents on the
-definitions of the frames in the network cluster is generated by the code
-generator, which is the core tool of this project. The code generator
-parses the network specification from a set of CAN bus related network
-database files (_*.dbc_) and can render this information in nearly any
-textual representation - in particular as C source code.
-
-We present two sample integrations of our CAN interface. Both let the code
-generator make the interface with the functional application code and all
-the required callback code, which is needed to serve this interface. The
-interface with functional application code is data oriented (there's a
-global struct per frame, whose fields hold the values of the frame's
-signals) and if desired it's a matter of only a few lines of additional
-template code to shape a functional interface instead.
-
-The samples know different frame characteristics (strictly regular,
-triggered by data change or data-change triggered plus periodic fallback
-behavior) and all needed data flow code is generated in a fully automated
-way; input are the network databases, which evidently have to accurately
-specify all relevant frame and signal properties.
-
-The samples support sequence counter and checksum validation. These
-elements are integrated into the callbacks and the complete control code
-is generated (whereas e.g. the actual checksum computation is considered
-an external function, which is just invoked by the generated code). Again,
-this only holds if having or not having a checksum and/or sequence counter
-and the kind of checksum and the counter's range, etc. is all properly
-specified in the network databases.
-
-The sample integrations are intended to demonstrate the capabilities of
-the concept. They won't be usable out of the box in any other environment
-but they will be a good starting point for customizing the offered
-solution to the needs of that environment.
-
-## What is sample code, what is intended production code, what to reuse ##
-
-The main folder canInterface is split into the following components, found
-as sub-folders:
-
-- *canInterface*. This folder contains the hand coded dispatcher engine.
-  It consists only of a few C source files. There's no makefile; the
-  intention is to integrate this code into the aimed project as source
-  files. This part is considered fixed, no modifications on this code
-  need to be done.
-    There's the template of a configuration header file, see
-  *canInterface/code/cde_canDispatcherEngine.config.h.template*. This file
-  is copied to the target project, renamed to
-  *cde_canDispatcherEngine.config.h* and the contained \#define's are set
-  according to the needs of the target platform (e.g. alignment) and the
-  project (e.g. API tailoring)
-  
-- *winTest*. This is a complete program, compilable with GNU make 3.81 and
-  GCC under Windows. It is used for testing the dispatcher engine
-  implementation and out of scope for most people
-  
-- *winSampleIntegration*. This is the first sample integration. It is a
-  complete program, compilable with GNU make 3.81 and GCC under Windows.
-  The needed effort to port it to Linux or Mac OS will be insignificant
-  since no operation system specific services are used. Particularly, no
-  multitasking is involved. The different "application tasks" are executed
-  strictly sequential, there's no preemption.

-    The sample itself is structured in components. These components again
-  can be distinguished according to their usefulness and re-usability for
-  true applications of the CAN interface. See below
-  
-- *arduinoSampleIntegration*. This is the second sample integration. The
-  functionality is nearly the same but here we use a real hardware
-  platform and the application builds on an existing preemptive real time
-  operating system. You will see that the code generation templates and
-  the integration code are modified but only to a minor extend
-  
-The main purpose of the sample integrations is to demonstrate how the
-combination of the given implementation of the dispatcher engine with the
-network dependent, generated parts of the code can be managed. And how
-this fits into a given platform environment and how it communicates with
-the functional application software.
-
-Summarizing, the files in component *canInterface* can be reused out of
-the box. Just edit the configuration header file and compile the sources
-with your project. The files from one of the two sample integrations can
-be reused only as starting point for your project. Particularly the code
-generation templates will be an asset.
-
-
-### The files of the Windows sample integration ###
-
-The source code of the this sample is separated in the following folders:
-
-- *osSimulation*. We present a running Windows console application. The
-  connection of the CAN interface to this application should resemble the
-  integration into a real embedded platform as much as possible. To meet
-  this demand this module defines a hypothetic Basic Software (BSW)
-  interface with regularly called tasks and CAN related interrupts.

-    The CAN interface can send and receive CAN frames: Send frames are
-  written into a log file and received frames are purely simulated.

-    The simulation has a number of configurable random elements in order
-  to simulate irregular timing and other errors like checksum or sequence
-  errors.

-    If you consider to integrate the CAN interface into your embedded
-  platform then you would probably compare the structure of the hypothetic
-  BSW interface with your actual BSW in order to get a feeling to which
-  extend the sample will fit to your specific demands
-
-- *integration*. This folder contains those files, which are typically
-  needed in every environment to connect the CAN interface to the given
-  platform.

-    For this specific sample code integration means integration with the
-  Windows environment, too. Therefore, you will also find some trivial
-  code needed to setup a console application, beginning with the C main
-  function
-
-- *codeGen*. This folder contains all generated C code and the script
-  *generateCode.cmd* to reproduce the code generation. (Porting this
-  Windows script to Linux or Mac OS is trivial.) The sample uses a single
-  network database file only, located in the sub-folder *dbcFiles*.
-  First have a look at this network database then browse through the
-  generated C code to see how the network information was translated into
-  program code.

-    The templates that control the code generation are stored in sub-folder
-  *templates*. Once you understood the structure of the generated code you
-  will find it easy and straightforward to modify the templates according
-  to your needed modifications of the generated code. Likely, there won't
-  be many modifications and very likely no structural changes. The
-  templates are widely platform independent. Only the initialization, when
-  frames and buses are registered with the operating system will differ
-  
-- *APSW*. This folder contains the functional code. The work assumption is
-  that this code is self-contained in that it builds on a data based API
-  not dealing with the mechanism which update the information in this API
-  or which connect this information with the outer world. This work
-  assumption relates to typical work environments, where the functional
-  code is made with tools like The Mathworks MATLAB/Simulink or ETAS
-  ASCET.

-    The functional code is held as simple as possible, the functionality
-  is trivial. The only aim of its specification was to involve all the
-  different kinds of CAN frames, which should be supported, being regular,
-  data-change triggered or a mixture of both.

-    The only value of this code is that it demonstrates how application
-  code can link to the CAN transmitted data
-
-- *logger*. Out of interest. The simple logging mechanism the user interface
-  of the sample application builds on
-
-- *environment*. Out of interest. Some basic types are defined and some
-  GCC specific issues are tackled. Just needed to make the code compilable
-
-
-### The files of the Arduino sample integration ###
-
-The Arduino sample integration barely differs from the Windows
-integration. The main difference is that it is the better test case in
-that it uses real tasks with preemption. Moreover, an existing real time
-operating system was applied so that the code generation process needed to
-undergo the prove that it can be configured for a given BSW interface (as
-far as no hand-written integration code is anyway in between).
-
-The Arduino sample distributes the functional code across two regular
-tasks of different period times. Above it has been said, that the concept
-of the dispatcher engines shapes a race condition free programming
-environment. This statement does of course no longer hold if different
-application tasks share the data delivered by one and the same dispatcher
-object. The Arduino code points to this fact and applies critical sections
-where needed to sort this out.
-
-Beyond this the Arduino sample will not have an added value for you in
-comparison to the Windows sample integration. Or vice versa.
-
-## Code architecture ##
-
-### The architecture chosen for the sample integrations ###
-
-The samples let the code generator produce individual handler functions
-for all the frames. Each frame is registered with its own dedicated
-handler. A number of six different kinds of transmission patterns has been
-specified (regular, data-change triggered and mixed, both for in- and
-outbound frames) and each frame will get a dedicated instance of one of
-the related six handlers. The data needed by the handlers is defined as
-far as possible local to the functions. We don't see large data structures
-but only a lot of similar functions.
-
-Pros and cons: This architecture means fast code since all data is
-directly addressable. It consumes a lot of ROM due to the repetition of
-the handler code. The main advantage of this architecture is the ease of
-maintenance: As a matter of fact the handlers are developed not in C but
-in template code. (This is C plus the constructs of the template engine.)
-Naturally, only a single instance of each handler needs to be developed
-and this is most easy if the handler implementation doesn't deal with
-complex, nested (external) data structures but can make use of simple
-local function variables and some flat static variables.
-
-### A more common architecture ###
-
-A more typical architecture would have only a single instance of the
-handler code. The only handler would be made reentrant and it would
-operate on one or more data tables, which have many individual entries,
-which are dedicated to the different frames. An element from such a data
-table would be a struct with many fields that describe all the properties
-of a frame: as simple as the CAN ID and as complex as signal layout
-patterns of contained checksums or sequence counters. The handler code
-would first fetch the right, frame related entry from the table and then
-have a lot of conditional code, which is controlled by the contents of
-that entry.
-
-The trade-off between ROM consumption and execution speed differs. Much
-less ROM and an about 40% higher execution time due to the now needed
-conditional code and more complex data addressing. The RAM consumption
-will not differ as the chosen algorithms and hence the needed state
-information are not affected by the choice of the architecture.
-
-This more conventional architecture is fully supported by our interface
-and code generation concept. The dispatcher engines doesn't care at all
-how many callbacks are present; whether it are different callbacks or a
-few or always the only one. The handler code is similar in structure but
-will now need array indexing and/or pointer operations. More important, it
-becomes one-time hand-coded C code; since it now is frame independent
-there's no longer any reason to generate it from a network database.
-Instead, you will let the code generator generate the large data table
-holding all the details about buses, frames and signals. The data is
-constant by nature and the complete table can be placed in ROM.
-
-The handler code can be derived from the samples. Take an arbitrary
-instance of the different kinds of handlers, copy them to the hand-coded
-part of your project and replace the access to local data elements
-(referenced by name only) with indexed access of the data table. The root
-of all data access will be an index into the data table. All frames get a
-linear, null based index during the registration process and this index
-can be fetched from within the handler by means of the available API.
-
-If you have distinct handlers for all the supported transmission patterns
-then you can still let the code generator generate the code which
-registers the frames with the appropriate handler. New templates for the
-generation of the data table will have to be written from scratch. The
-effort for this is little.
-
-### More alternatives ###
-
-#### Additional transmission patterns ####
-
-If the specification of the network databases introduces more transmission
-pattern then the templates will need according modification. One could
-imagine bursts of frames (a due frame is send a configurable number of
-times for sake of redundancy) or sending on demand (by invocation of a
-dedicated trigger function), etc. Most of this will be straightforward
-refinement of the existing template code.
-
-#### Not using the dispatcher concept #####
-
-Particularly for small systems (Arduino!) it's not necessarily the best
-choice to integrate the dispatcher engine code. A shared buffer and simple
-global suspend/resume interrupt functions are often sufficient to safely handle
-the CAN data flow. What remains in every environment is the need for the
-pack and unpack functions to link the signal based functional code to the
-frame based hardware drivers.
-
-In such an environment the code generator can be applied to generate the
-needed shared buffers, the pack and unpack functions and the API. The API
-can be modelled as global data structures as in our sample integrations or
-as functional API.
-
-### Data table based pack/unpack functions ###
-
-The trade-off between ROM consumption and CPU load can be shifted further
-in the direction of less ROM and higher execution time by not generating
-individual pack and unpack functions for each frame. Instead, a data table
-can be designed, which holds an entry per frame. Such an entry will hold a
-list of signal descriptors with byte order and bit layout and scaling
-information. A one-time hand-coded pack and unpack function pair would now
-operate on this table.
-
-A data based API becomes unlikely in such an architecture as the pack and
-unpack functions will probably be signal related without the possibility of
-using the best fitting data type for a signal; they could take frame and
-signal index as input arguments. The code generator would generate
-meaningful enumerations for frame and signal indexes or it would hide all
-of this in signal related preprocessor macros.
-
-## Support of model based design ##
-
-If the application code is made with a graphical data flow model based
-system like The Mathworks Simulink (with dSPACE TargetLink or The
-Mathworks Embedded Coder) or ETAS ASCET then a machine readable
-specification of the API is required for import into these application
-code generators. Our code generator can generate such a specification
-(typically an XML or MATLAB M script file). This is just the same for any
-imaginable architecture and according templates are provided with our
-sample integrations.
-
-The code generator is controlled by command line. If this is combined with
-the code generation scripts of such a model based design tool and a
-makefile based compilation script then even a low budget platform like
-Arduino can get much of the characteristics of a rapid prototyping
-system.
-
-## Multi-tasking ##
-
-The implementation of the dispatcher engine is made thread-safe but this
-only holds under some preconditions:
-
-- Multi-core platforms are not supported with the trivial exception of
-  running all affected code on a single core of such a platform
-- A priority controlled operating system is assumed. Each activatable CPU
-  context has a priority and the running context will never be interrupted
-  by another context of same or lower priority
-- The dispatcher queue can connect only two competing CPU contexts. This
-  excludes a software architecture, where different event sources, which
-  can preempt each other, try to post into the same dispatcher object. A
-  typical example would be a system with two interrupts of different
-  priority, one to notify bus state changes and the other one notifying
-  frame receptions. Here, one would either apply two different dispatcher
-  objects or implement some mutual exclusion code between the interrupts
-- The event sending context needs to have a higher priority than the event
-  receiving context. This is normally given; we have an interrupt as event
-  source and the lower prior application task as receiver
-- The initialization of the code must be possible in a race-condition free
-  environment; initialization is not designed thread-safe. No events must
-  be posted until all frame and bus registration has been done and the
-  registration calls must not be done from competing CPU contexts. Most
-  platforms offer an initialization task for this purpose
-
-If these prerequisites are fulfilled then the complete event receiving code
-can be designed and implemented race condition free and data integrity is
-granted.
-
-## Memory management concept ##
-
-The required data tables and the timer objects of the CAN interface are
-allocated on a simple heap. For good reasons, dynamic memory management is
-unwanted in embedded platforms. Therefore a safe and most simple heap
-design has been chosen, which is compliant with usual requirements for
-embedded software.
-
-The heap is a fixed-size chunk of RAM, which is statically defined by the
-client code (the embedding integration code). It'll probably use a static
-array of the desired number of bytes for this purpose. Memory allocation
-starts at the lowest address of the chunk. Each memory request from the
-CAN interface is served by taking the next number of bytes from the heap
-area. An error message is returned if the heap is exhausted and there's no
-method to return allocated memory to the heap. Fragmentation is excluded
-by principle.
-
-The heap offers a diagnostic API, which can return the number of used and
-still unused bytes of RAM. Any reasonable application of the CAN
-interface has an upper bounds of heap memory consumption; if the
-diagnostic interface is queried for the used amount of RAM then the heap
-can be set to exactly this number of bytes (i.e. in the next compilation
-of the code) and there's no risk that the heap will ever be exhausted.
-
-The core idea is the upper bounds of heap memory consumption. This relates
-to the fact that there must be no dependency of RAM consumption on
-run-time data or events. This is given for all usual applications and easy
-to prove. The RAM consumption depends on the number of registered frames,
-the number of dispatcher engines, the size of the events, the number of
-timer objects and a few configuration details at compile time. All of this
-is determined in a static way by the source code but not by any run-time
-effect. If the network database changes, if code is regenerated the memory
-consumption will change but only then. Our concept is to use the heap
-diagnostics in DEBUG compilation to report the actual consumption and to
-make the heap size accurately fitting in the production compilation. Of
-course, the accurate settings needs to undergo an adaptation after each
-code regeneration. (Very sophistical elaborated code generation
-templates could even anticipate the required code size as they have all
-the knowledge about number of frames etc.) One has to understand that
-memory allocation errors can be definitely excluded for all the future if
-the compiled software passed the very first run-time test after code
-regeneration and build.
-
-One exception has to be mentioned. The timer API knows destruction of
-timer objects. As said, there's no concept of freeing memory for reuse.
-Instead, the timer implementation has a list of destroyed objects. This
-list is inspected first, when a(nother) timer object is created. If there
-a (destroyed) object in the list, then it is now resuscitated and reused
-rather then allocating new memory on the heap. This is a bit of dynamic
-memory allocation. However, even if event handlers make use of dynamic
-creation and destruction of timer objects then there's in far the most
-cases still an upper bounds for the memory consumption but it now becomes
-data and event dependent when this limit will be reached. The matter is:
-When can we safely query the diagnostic API to get the maximum memory
-consumption. If testing is too short then we will not get the absolute
-maximum yet.
-
-To avoid this problem, the timer API has been designed such that dynamic
-timer creation and deletion can be completely avoided. Instead, a timer
-can be suspended and re-triggered by the application code; the object as
-such is permanently existing. All ever needed timer objects are created in
-the initialization phase but at run-time no single byte will be allocated
-on the heap any more. The diagnostic API can be safely queried for the
-memory consumption already at the end of the code initialization.
-
-### Machine alignment ###
-
-The implementation of the heap depends on the common machine alignment.
-This has to be configured at compile time in the configuration header
-file of the CAN interface.
-
-### Critical section code needed? ###
-
-All dispatcher engines share the same heap. If dynamic timer creation and
-deletion is used (opposed to our recommendation, see above) then a
-critical section is required by the implementation. No such code is
-required otherwise. This has to be configured at compile time in the
-configuration header file of the CAN interface.
-
-## Error handling concept ###
-
-The error handling concept make the distinction between run-time problems
-and implementation/compile-time problems. The former are handled by return
-value of the affected function, whereas the latter are mainly caught by
-assertions.
-
-Run-time problems are those, which might appear because of external data
-or events. Only few possible error states in the dispatcher engine are of
-this kind. The most prominent example is the queue overrun. This event is
-reported by return value to the event posting context and is available to
-the functional application code by a (naturally asynchronous) diagnostic
-API. Data repair is impossible. This failure is prototypic in that
-
-- it is reported by return value
-- data repair is not possible
-- the software remains in stable, functional state
-
-Far the most error states are of kind implementation/compile time error.
-These errors appear only because of the configuration of the source code
-but do not depend on outer world data or events. They will unconditionally
-occur as soon as the according program path is taken. The most prominent
-example is the memory allocation error. As explained above this error will
-not occur because of data or timing effects, it's either in the software
-at compile time or not. Errors of this characteristics can be caught most
-efficiently by assertions; there must be no CPU consuming dynamic test
-code in the production code for these kind of errors.
-
-Using assertions makes it unavoidable to have an initial testing phase in
-DEBUG compilation. If the software design doesn't make use of dynamic
-timer creation and deletion (see section Memory management concept) then
-this test phase could even be quite short; nearly all of these errors and
-surely the memory allocation errors would be reported immediately in the
-system initialization phase.
-
-## Use of dispatcher engine for application purpose ##
-
-Nobody cares what to use callbacks and timers for. If our samples use them
-to implement timeout recognition for inbound CAN frames and send time patterns
-for outbound frames then this only documents the intended use case of
-these elements.
-
-Many embedded applications use state machines as fundamental design
-pattern. The functionality split in many distinct, small operations, which
-are scheduled dependent on the current state and external data and events.
-Timing events typically play an important role in embedded applications.
-The implementation of an application as a single-threaded state machine
-shares some characteristics with applications designed for preemptive
-multi-tasking. In the first place can they show a pseudo-parallel behavior
-like preemptive and non-preemptive operating systems.
-
-It's well possible to use our dispatcher engine to not only control the
-upper layers of the CAN communication but to host such an application at
-the same time. A portion of callbacks and timers would be used just as
-shown in our samples but more callbacks and timers can control the "tasks"
-of the functional application code. Furthermore, the dispatched events
-must not necessarily be CAN related but may origin from other hardware
-devices. All of this running in a single-threaded, RTOS free platform can
-become a cheap, easy, less error-prone (entirely race-condition free) and
-sufficient solution particularly on small platforms.
-
-## Optimizations ##
-
-The code of the dispatcher engine as it is today is not particularly
-intended for small systems; the chosen data structure support fast
-execution rather than little RAM consumption. We provide an Arduino
-integration but the 8k of RAM of an Arduino board will soon become a
-limiting factor.
-
-A suitable customization decision for small systems with little RAM can
-be to not use the dispatcher engine and to only benefit from the code
-generator. This has been discussed above.
-
-Another idea is to optimize the dispatcher engine code for less RAM
-consumption. Some obvious things can be done easily.
-
-The implementation makes use of the C concept of a somewhat undetermined
-integer. It uses *int* and *unsigned int* where the actual range of the
-integers doesn't really matter so that the compiler will chose the native
-word size of the underlaying machine. This yield normally a good execution
-time performance. However, in most cases where *int* is used the use of
-even a one Byte integer will suffice - it's often about the number of
-buses and CAN frames and serving 250 CAN frames is a lot. By carefully
-replacing all appearances of *int* with a more specific type like
-*uint8_t* from *stdint.h* a significant amount of RAM can be saved.
-Additional conditional preprocessor code and assertions can be applied to
-avoid unrecognized overruns of the limited integers.
-
-The other obvious potential for saving RAM on cost of execution speed is
-the substitution of pointers with indexes. Needed cross references between
-objects are implemented by fast, direct pointing. Most of these code
-locations can be changed just like that towards the use of indexes; frames
-and buses are already held in arrays.
-
-Timer objects are on the heap; here a structural change would be required
-to introduce indexes. One would allocate a fixed table of timers. The
-application code could state the table size as it already is for frames
-and buses. This change would shape the demand for a diagnostic API to get
-the maximum table utilization in order to make the table size manageable.
-
-In most cases a one Byte integer will suffice for the indexes, which again
-leads to a significant RAM saving, although on cost of execution speed.
-
-Our intention is to offer both concepts in a later refinement of the
-dispatcher implementation; optimization for speed or for RAM consumption
-would become a compile-time configuration parameter. The biggest issue in
-doing so is the strongly increased testing effort.
-
-Saving ROM on cost of execution speed and maintainability can be achieved
-by alternative designs of the callback code, this has been discussed
-above.
+[[include repo=code path=doc/wiki/canInterface.md]]

The CAN Interface modified by Peter Vranken

Peter Vranken — Sun, 18 Oct 2015 20:44:14 -0000

--- v8
+++ v9
@@ -29,7 +29,7 @@
 a typical environment the input of the queue will be connected to a system
 CAN interrupt, be it for notification of a received frame or a send
 acknowledge. The output of the queue is read by the main function of the
-dispatcher engine. This call of this function will usually be made from an
+dispatcher engine. The call of this function will usually be made from an
 application task, thus not in the context of the notifying interrupt. The
 main function delegates the events to the registered handlers and since
 this is all done in the context of the application task are there no
@@ -79,7 +79,7 @@

 The intended use case of our open source project is that all the code or
 data structures, which is between the dispatcher's API and the interface
-with the functional application code and that dependents on the
+with the functional application code and which dependents on the
 definitions of the frames in the network cluster is generated by the code
 generator, which is the core tool of this project. The code generator
 parses the network specification from a set of CAN bus related network
@@ -105,7 +105,7 @@
 is generated (whereas e.g. the actual checksum computation is considered
 an external function, which is just invoked by the generated code). Again,
 this only holds if having or not having a checksum and/or sequence counter
-and the kind of checksum and the counter's range, etc. is properly
+and the kind of checksum and the counter's range, etc. is all properly
 specified in the network databases.

 The sample integrations are intended to demonstrate the capabilities of
@@ -148,7 +148,7 @@
   functionality is nearly the same but here we use a real hardware
   platform and the application builds on an existing preemptive real time
   operating system. You will see that the code generation templates and
-  the integration code is modified but only to a minor extend
+  the integration code are modified but only to a minor extend

 The main purpose of the sample integrations is to demonstrate how the
 combination of the given implementation of the dispatcher engine with the

The CAN Interface modified by Peter Vranken

Peter Vranken — Fri, 11 Sep 2015 21:04:25 -0000

--- v7
+++ v8
@@ -392,6 +392,11 @@
 - The event sending context needs to have a higher priority than the event
   receiving context. This is normally given; we have an interrupt as event
   source and the lower prior application task as receiver
+- The initialization of the code must be possible in a race-condition free
+  environment; initialization is not designed thread-safe. No events must
+  be posted until all frame and bus registration has been done and the
+  registration calls must not be done from competing CPU contexts. Most
+  platforms offer an initialization task for this purpose

 If these prerequisites are fulfilled then the complete event receiving code
 can be designed and implemented race condition free and data integrity is
@@ -506,8 +511,8 @@

 Using assertions makes it unavoidable to have an initial testing phase in
 DEBUG compilation. If the software design doesn't make use of dynamic
-timer creation and deletion (see [The CAN Interface#memory-management-concept]) then this
-test phase could even be quite short; nearly all of these errors and
+timer creation and deletion (see section Memory management concept) then
+this test phase could even be quite short; nearly all of these errors and
 surely the memory allocation errors would be reported immediately in the
 system initialization phase.

The CAN Interface modified by Peter Vranken

Peter Vranken — Fri, 11 Sep 2015 20:54:58 -0000

--- v6
+++ v7
@@ -506,7 +506,7 @@

 Using assertions makes it unavoidable to have an initial testing phase in
 DEBUG compilation. If the software design doesn't make use of dynamic
-timer creation and deletion (see [CAN Interface:#Memory management concept]) then this
+timer creation and deletion (see [The CAN Interface#memory-management-concept]) then this
 test phase could even be quite short; nearly all of these errors and
 surely the memory allocation errors would be reported immediately in the
 system initialization phase.

The CAN Interface modified by Peter Vranken

Peter Vranken — Fri, 11 Sep 2015 20:51:43 -0000

--- v5
+++ v6
@@ -475,3 +475,115 @@
 critical section is required by the implementation. No such code is
 required otherwise. This has to be configured at compile time in the
 configuration header file of the CAN interface.
+
+## Error handling concept ###
+
+The error handling concept make the distinction between run-time problems
+and implementation/compile-time problems. The former are handled by return
+value of the affected function, whereas the latter are mainly caught by
+assertions.
+
+Run-time problems are those, which might appear because of external data
+or events. Only few possible error states in the dispatcher engine are of
+this kind. The most prominent example is the queue overrun. This event is
+reported by return value to the event posting context and is available to
+the functional application code by a (naturally asynchronous) diagnostic
+API. Data repair is impossible. This failure is prototypic in that
+
+- it is reported by return value
+- data repair is not possible
+- the software remains in stable, functional state
+
+Far the most error states are of kind implementation/compile time error.
+These errors appear only because of the configuration of the source code
+but do not depend on outer world data or events. They will unconditionally
+occur as soon as the according program path is taken. The most prominent
+example is the memory allocation error. As explained above this error will
+not occur because of data or timing effects, it's either in the software
+at compile time or not. Errors of this characteristics can be caught most
+efficiently by assertions; there must be no CPU consuming dynamic test
+code in the production code for these kind of errors.
+
+Using assertions makes it unavoidable to have an initial testing phase in
+DEBUG compilation. If the software design doesn't make use of dynamic
+timer creation and deletion (see [CAN Interface:#Memory management concept]) then this
+test phase could even be quite short; nearly all of these errors and
+surely the memory allocation errors would be reported immediately in the
+system initialization phase.
+
+## Use of dispatcher engine for application purpose ##
+
+Nobody cares what to use callbacks and timers for. If our samples use them
+to implement timeout recognition for inbound CAN frames and send time patterns
+for outbound frames then this only documents the intended use case of
+these elements.
+
+Many embedded applications use state machines as fundamental design
+pattern. The functionality split in many distinct, small operations, which
+are scheduled dependent on the current state and external data and events.
+Timing events typically play an important role in embedded applications.
+The implementation of an application as a single-threaded state machine
+shares some characteristics with applications designed for preemptive
+multi-tasking. In the first place can they show a pseudo-parallel behavior
+like preemptive and non-preemptive operating systems.
+
+It's well possible to use our dispatcher engine to not only control the
+upper layers of the CAN communication but to host such an application at
+the same time. A portion of callbacks and timers would be used just as
+shown in our samples but more callbacks and timers can control the "tasks"
+of the functional application code. Furthermore, the dispatched events
+must not necessarily be CAN related but may origin from other hardware
+devices. All of this running in a single-threaded, RTOS free platform can
+become a cheap, easy, less error-prone (entirely race-condition free) and
+sufficient solution particularly on small platforms.
+
+## Optimizations ##
+
+The code of the dispatcher engine as it is today is not particularly
+intended for small systems; the chosen data structure support fast
+execution rather than little RAM consumption. We provide an Arduino
+integration but the 8k of RAM of an Arduino board will soon become a
+limiting factor.
+
+A suitable customization decision for small systems with little RAM can
+be to not use the dispatcher engine and to only benefit from the code
+generator. This has been discussed above.
+
+Another idea is to optimize the dispatcher engine code for less RAM
+consumption. Some obvious things can be done easily.
+
+The implementation makes use of the C concept of a somewhat undetermined
+integer. It uses *int* and *unsigned int* where the actual range of the
+integers doesn't really matter so that the compiler will chose the native
+word size of the underlaying machine. This yield normally a good execution
+time performance. However, in most cases where *int* is used the use of
+even a one Byte integer will suffice - it's often about the number of
+buses and CAN frames and serving 250 CAN frames is a lot. By carefully
+replacing all appearances of *int* with a more specific type like
+*uint8_t* from *stdint.h* a significant amount of RAM can be saved.
+Additional conditional preprocessor code and assertions can be applied to
+avoid unrecognized overruns of the limited integers.
+
+The other obvious potential for saving RAM on cost of execution speed is
+the substitution of pointers with indexes. Needed cross references between
+objects are implemented by fast, direct pointing. Most of these code
+locations can be changed just like that towards the use of indexes; frames
+and buses are already held in arrays.
+
+Timer objects are on the heap; here a structural change would be required
+to introduce indexes. One would allocate a fixed table of timers. The
+application code could state the table size as it already is for frames
+and buses. This change would shape the demand for a diagnostic API to get
+the maximum table utilization in order to make the table size manageable.
+
+In most cases a one Byte integer will suffice for the indexes, which again
+leads to a significant RAM saving, although on cost of execution speed.
+
+Our intention is to offer both concepts in a later refinement of the
+dispatcher implementation; optimization for speed or for RAM consumption
+would become a compile-time configuration parameter. The biggest issue in
+doing so is the strongly increased testing effort.
+
+Saving ROM on cost of execution speed and maintainability can be achieved
+by alternative designs of the callback code, this has been discussed
+above.

The CAN Interface modified by Peter Vranken

Peter Vranken — Thu, 10 Sep 2015 20:45:36 -0000

--- v4
+++ v5
@@ -269,7 +269,7 @@
 complex, nested (external) data structures but can make use of simple
 local function variables and some flat static variables.

-### Common architecture ###
+### A more common architecture ###

 A more typical architecture would have only a single instance of the
 handler code. The only handler would be made reentrant and it would
@@ -357,8 +357,8 @@

 ## Support of model based design ##

-If the application code is made with a graphical data flow model model
-based system like The Mathworks Simulink (with dSPACE TargetLink or The
+If the application code is made with a graphical data flow model based
+system like The Mathworks Simulink (with dSPACE TargetLink or The
 Mathworks Embedded Coder) or ETAS ASCET then a machine readable
 specification of the API is required for import into these application
 code generators. Our code generator can generate such a specification
@@ -379,22 +379,23 @@

 - Multi-core platforms are not supported with the trivial exception of
   running all affected code on a single core of such a platform
-- A priority controlled operating system is assumed. Each CPU context has
-  a priority and the running context will never be interrupted by another
-  context of same or lower priority
-- The dispatcher queue can connect only two CPU competing contexts. This
-  excludes a software architecture, where different event sources try to
-  post into the same dispatcher object if they can preempt each other. A
+- A priority controlled operating system is assumed. Each activatable CPU
+  context has a priority and the running context will never be interrupted
+  by another context of same or lower priority
+- The dispatcher queue can connect only two competing CPU contexts. This
+  excludes a software architecture, where different event sources, which
+  can preempt each other, try to post into the same dispatcher object. A
   typical example would be a system with two interrupts of different
   priority, one to notify bus state changes and the other one notifying
   frame receptions. Here, one would either apply two different dispatcher
   objects or implement some mutual exclusion code between the interrupts
-- The event sending context needs to be of higher priority than the event
+- The event sending context needs to have a higher priority than the event
   receiving context. This is normally given; we have an interrupt as event
   source and the lower prior application task as receiver

 If these prerequisites are fulfilled then the complete event receiving code
-can be designed and implemented race condition free.
+can be designed and implemented race condition free and data integrity is
+granted.

 ## Memory management concept ##

@@ -404,7 +405,7 @@
 design has been chosen, which is compliant with usual requirements for
 embedded software.

-The heap is a fixed size chunk of RAM, which is statically defined by the
+The heap is a fixed-size chunk of RAM, which is statically defined by the
 client code (the embedding integration code). It'll probably use a static
 array of the desired number of bytes for this purpose. Memory allocation
 starts at the lowest address of the chunk. Each memory request from the
@@ -414,7 +415,7 @@
 by principle.

 The heap offers a diagnostic API, which can return the number of used and
-remaining, unused bytes of RAM. Any reasonable application of the CAN
+still unused bytes of RAM. Any reasonable application of the CAN
 interface has an upper bounds of heap memory consumption; if the
 diagnostic interface is queried for the used amount of RAM then the heap
 can be set to exactly this number of bytes (i.e. in the next compilation
@@ -470,7 +471,7 @@
 ### Critical section code needed? ###

 All dispatcher engines share the same heap. If dynamic timer creation and
-deletion is used (opposed to our recommendation, see above) than a
+deletion is used (opposed to our recommendation, see above) then a
 critical section is required by the implementation. No such code is
 required otherwise. This has to be configured at compile time in the
 configuration header file of the CAN interface.

The CAN Interface modified by Peter Vranken

Peter Vranken — Thu, 10 Sep 2015 20:26:15 -0000

--- v3
+++ v4
@@ -1,4 +1,6 @@
 # CAN Interface #
+
+Table of contents

 [TOC]

The CAN Interface modified by Peter Vranken

Peter Vranken — Thu, 10 Sep 2015 20:24:54 -0000

--- v2
+++ v3
@@ -1,4 +1,6 @@
 # CAN Interface #
+
+[TOC]

 This open source project proposes a CAN interface for embedded
 applications. Although it meets a lot of demands of real applications,
@@ -109,4 +111,364 @@
 but they will be a good starting point for customizing the offered
 solution to the needs of that environment.

-(to be continued)
+## What is sample code, what is intended production code, what to reuse ##
+
+The main folder canInterface is split into the following components, found
+as sub-folders:
+
+- *canInterface*. This folder contains the hand coded dispatcher engine.
+  It consists only of a few C source files. There's no makefile; the
+  intention is to integrate this code into the aimed project as source
+  files. This part is considered fixed, no modifications on this code
+  need to be done.
+    There's the template of a configuration header file, see
+  *canInterface/code/cde_canDispatcherEngine.config.h.template*. This file
+  is copied to the target project, renamed to
+  *cde_canDispatcherEngine.config.h* and the contained \#define's are set
+  according to the needs of the target platform (e.g. alignment) and the
+  project (e.g. API tailoring)
+  
+- *winTest*. This is a complete program, compilable with GNU make 3.81 and
+  GCC under Windows. It is used for testing the dispatcher engine
+  implementation and out of scope for most people
+  
+- *winSampleIntegration*. This is the first sample integration. It is a
+  complete program, compilable with GNU make 3.81 and GCC under Windows.
+  The needed effort to port it to Linux or Mac OS will be insignificant
+  since no operation system specific services are used. Particularly, no
+  multitasking is involved. The different "application tasks" are executed
+  strictly sequential, there's no preemption.

+    The sample itself is structured in components. These components again
+  can be distinguished according to their usefulness and re-usability for
+  true applications of the CAN interface. See below
+  
+- *arduinoSampleIntegration*. This is the second sample integration. The
+  functionality is nearly the same but here we use a real hardware
+  platform and the application builds on an existing preemptive real time
+  operating system. You will see that the code generation templates and
+  the integration code is modified but only to a minor extend
+  
+The main purpose of the sample integrations is to demonstrate how the
+combination of the given implementation of the dispatcher engine with the
+network dependent, generated parts of the code can be managed. And how
+this fits into a given platform environment and how it communicates with
+the functional application software.
+
+Summarizing, the files in component *canInterface* can be reused out of
+the box. Just edit the configuration header file and compile the sources
+with your project. The files from one of the two sample integrations can
+be reused only as starting point for your project. Particularly the code
+generation templates will be an asset.
+
+
+### The files of the Windows sample integration ###
+
+The source code of the this sample is separated in the following folders:
+
+- *osSimulation*. We present a running Windows console application. The
+  connection of the CAN interface to this application should resemble the
+  integration into a real embedded platform as much as possible. To meet
+  this demand this module defines a hypothetic Basic Software (BSW)
+  interface with regularly called tasks and CAN related interrupts.

+    The CAN interface can send and receive CAN frames: Send frames are
+  written into a log file and received frames are purely simulated.

+    The simulation has a number of configurable random elements in order
+  to simulate irregular timing and other errors like checksum or sequence
+  errors.

+    If you consider to integrate the CAN interface into your embedded
+  platform then you would probably compare the structure of the hypothetic
+  BSW interface with your actual BSW in order to get a feeling to which
+  extend the sample will fit to your specific demands
+
+- *integration*. This folder contains those files, which are typically
+  needed in every environment to connect the CAN interface to the given
+  platform.

+    For this specific sample code integration means integration with the
+  Windows environment, too. Therefore, you will also find some trivial
+  code needed to setup a console application, beginning with the C main
+  function
+
+- *codeGen*. This folder contains all generated C code and the script
+  *generateCode.cmd* to reproduce the code generation. (Porting this
+  Windows script to Linux or Mac OS is trivial.) The sample uses a single
+  network database file only, located in the sub-folder *dbcFiles*.
+  First have a look at this network database then browse through the
+  generated C code to see how the network information was translated into
+  program code.

+    The templates that control the code generation are stored in sub-folder
+  *templates*. Once you understood the structure of the generated code you
+  will find it easy and straightforward to modify the templates according
+  to your needed modifications of the generated code. Likely, there won't
+  be many modifications and very likely no structural changes. The
+  templates are widely platform independent. Only the initialization, when
+  frames and buses are registered with the operating system will differ
+  
+- *APSW*. This folder contains the functional code. The work assumption is
+  that this code is self-contained in that it builds on a data based API
+  not dealing with the mechanism which update the information in this API
+  or which connect this information with the outer world. This work
+  assumption relates to typical work environments, where the functional
+  code is made with tools like The Mathworks MATLAB/Simulink or ETAS
+  ASCET.

+    The functional code is held as simple as possible, the functionality
+  is trivial. The only aim of its specification was to involve all the
+  different kinds of CAN frames, which should be supported, being regular,
+  data-change triggered or a mixture of both.

+    The only value of this code is that it demonstrates how application
+  code can link to the CAN transmitted data
+
+- *logger*. Out of interest. The simple logging mechanism the user interface
+  of the sample application builds on
+
+- *environment*. Out of interest. Some basic types are defined and some
+  GCC specific issues are tackled. Just needed to make the code compilable
+
+
+### The files of the Arduino sample integration ###
+
+The Arduino sample integration barely differs from the Windows
+integration. The main difference is that it is the better test case in
+that it uses real tasks with preemption. Moreover, an existing real time
+operating system was applied so that the code generation process needed to
+undergo the prove that it can be configured for a given BSW interface (as
+far as no hand-written integration code is anyway in between).
+
+The Arduino sample distributes the functional code across two regular
+tasks of different period times. Above it has been said, that the concept
+of the dispatcher engines shapes a race condition free programming
+environment. This statement does of course no longer hold if different
+application tasks share the data delivered by one and the same dispatcher
+object. The Arduino code points to this fact and applies critical sections
+where needed to sort this out.
+
+Beyond this the Arduino sample will not have an added value for you in
+comparison to the Windows sample integration. Or vice versa.
+
+## Code architecture ##
+
+### The architecture chosen for the sample integrations ###
+
+The samples let the code generator produce individual handler functions
+for all the frames. Each frame is registered with its own dedicated
+handler. A number of six different kinds of transmission patterns has been
+specified (regular, data-change triggered and mixed, both for in- and
+outbound frames) and each frame will get a dedicated instance of one of
+the related six handlers. The data needed by the handlers is defined as
+far as possible local to the functions. We don't see large data structures
+but only a lot of similar functions.
+
+Pros and cons: This architecture means fast code since all data is
+directly addressable. It consumes a lot of ROM due to the repetition of
+the handler code. The main advantage of this architecture is the ease of
+maintenance: As a matter of fact the handlers are developed not in C but
+in template code. (This is C plus the constructs of the template engine.)
+Naturally, only a single instance of each handler needs to be developed
+and this is most easy if the handler implementation doesn't deal with
+complex, nested (external) data structures but can make use of simple
+local function variables and some flat static variables.
+
+### Common architecture ###
+
+A more typical architecture would have only a single instance of the
+handler code. The only handler would be made reentrant and it would
+operate on one or more data tables, which have many individual entries,
+which are dedicated to the different frames. An element from such a data
+table would be a struct with many fields that describe all the properties
+of a frame: as simple as the CAN ID and as complex as signal layout
+patterns of contained checksums or sequence counters. The handler code
+would first fetch the right, frame related entry from the table and then
+have a lot of conditional code, which is controlled by the contents of
+that entry.
+
+The trade-off between ROM consumption and execution speed differs. Much
+less ROM and an about 40% higher execution time due to the now needed
+conditional code and more complex data addressing. The RAM consumption
+will not differ as the chosen algorithms and hence the needed state
+information are not affected by the choice of the architecture.
+
+This more conventional architecture is fully supported by our interface
+and code generation concept. The dispatcher engines doesn't care at all
+how many callbacks are present; whether it are different callbacks or a
+few or always the only one. The handler code is similar in structure but
+will now need array indexing and/or pointer operations. More important, it
+becomes one-time hand-coded C code; since it now is frame independent
+there's no longer any reason to generate it from a network database.
+Instead, you will let the code generator generate the large data table
+holding all the details about buses, frames and signals. The data is
+constant by nature and the complete table can be placed in ROM.
+
+The handler code can be derived from the samples. Take an arbitrary
+instance of the different kinds of handlers, copy them to the hand-coded
+part of your project and replace the access to local data elements
+(referenced by name only) with indexed access of the data table. The root
+of all data access will be an index into the data table. All frames get a
+linear, null based index during the registration process and this index
+can be fetched from within the handler by means of the available API.
+
+If you have distinct handlers for all the supported transmission patterns
+then you can still let the code generator generate the code which
+registers the frames with the appropriate handler. New templates for the
+generation of the data table will have to be written from scratch. The
+effort for this is little.
+
+### More alternatives ###
+
+#### Additional transmission patterns ####
+
+If the specification of the network databases introduces more transmission
+pattern then the templates will need according modification. One could
+imagine bursts of frames (a due frame is send a configurable number of
+times for sake of redundancy) or sending on demand (by invocation of a
+dedicated trigger function), etc. Most of this will be straightforward
+refinement of the existing template code.
+
+#### Not using the dispatcher concept #####
+
+Particularly for small systems (Arduino!) it's not necessarily the best
+choice to integrate the dispatcher engine code. A shared buffer and simple
+global suspend/resume interrupt functions are often sufficient to safely handle
+the CAN data flow. What remains in every environment is the need for the
+pack and unpack functions to link the signal based functional code to the
+frame based hardware drivers.
+
+In such an environment the code generator can be applied to generate the
+needed shared buffers, the pack and unpack functions and the API. The API
+can be modelled as global data structures as in our sample integrations or
+as functional API.
+
+### Data table based pack/unpack functions ###
+
+The trade-off between ROM consumption and CPU load can be shifted further
+in the direction of less ROM and higher execution time by not generating
+individual pack and unpack functions for each frame. Instead, a data table
+can be designed, which holds an entry per frame. Such an entry will hold a
+list of signal descriptors with byte order and bit layout and scaling
+information. A one-time hand-coded pack and unpack function pair would now
+operate on this table.
+
+A data based API becomes unlikely in such an architecture as the pack and
+unpack functions will probably be signal related without the possibility of
+using the best fitting data type for a signal; they could take frame and
+signal index as input arguments. The code generator would generate
+meaningful enumerations for frame and signal indexes or it would hide all
+of this in signal related preprocessor macros.
+
+## Support of model based design ##
+
+If the application code is made with a graphical data flow model model
+based system like The Mathworks Simulink (with dSPACE TargetLink or The
+Mathworks Embedded Coder) or ETAS ASCET then a machine readable
+specification of the API is required for import into these application
+code generators. Our code generator can generate such a specification
+(typically an XML or MATLAB M script file). This is just the same for any
+imaginable architecture and according templates are provided with our
+sample integrations.
+
+The code generator is controlled by command line. If this is combined with
+the code generation scripts of such a model based design tool and a
+makefile based compilation script then even a low budget platform like
+Arduino can get much of the characteristics of a rapid prototyping
+system.
+
+## Multi-tasking ##
+
+The implementation of the dispatcher engine is made thread-safe but this
+only holds under some preconditions:
+
+- Multi-core platforms are not supported with the trivial exception of
+  running all affected code on a single core of such a platform
+- A priority controlled operating system is assumed. Each CPU context has
+  a priority and the running context will never be interrupted by another
+  context of same or lower priority
+- The dispatcher queue can connect only two CPU competing contexts. This
+  excludes a software architecture, where different event sources try to
+  post into the same dispatcher object if they can preempt each other. A
+  typical example would be a system with two interrupts of different
+  priority, one to notify bus state changes and the other one notifying
+  frame receptions. Here, one would either apply two different dispatcher
+  objects or implement some mutual exclusion code between the interrupts
+- The event sending context needs to be of higher priority than the event
+  receiving context. This is normally given; we have an interrupt as event
+  source and the lower prior application task as receiver
+
+If these prerequisites are fulfilled then the complete event receiving code
+can be designed and implemented race condition free.
+
+## Memory management concept ##
+
+The required data tables and the timer objects of the CAN interface are
+allocated on a simple heap. For good reasons, dynamic memory management is
+unwanted in embedded platforms. Therefore a safe and most simple heap
+design has been chosen, which is compliant with usual requirements for
+embedded software.
+
+The heap is a fixed size chunk of RAM, which is statically defined by the
+client code (the embedding integration code). It'll probably use a static
+array of the desired number of bytes for this purpose. Memory allocation
+starts at the lowest address of the chunk. Each memory request from the
+CAN interface is served by taking the next number of bytes from the heap
+area. An error message is returned if the heap is exhausted and there's no
+method to return allocated memory to the heap. Fragmentation is excluded
+by principle.
+
+The heap offers a diagnostic API, which can return the number of used and
+remaining, unused bytes of RAM. Any reasonable application of the CAN
+interface has an upper bounds of heap memory consumption; if the
+diagnostic interface is queried for the used amount of RAM then the heap
+can be set to exactly this number of bytes (i.e. in the next compilation
+of the code) and there's no risk that the heap will ever be exhausted.
+
+The core idea is the upper bounds of heap memory consumption. This relates
+to the fact that there must be no dependency of RAM consumption on
+run-time data or events. This is given for all usual applications and easy
+to prove. The RAM consumption depends on the number of registered frames,
+the number of dispatcher engines, the size of the events, the number of
+timer objects and a few configuration details at compile time. All of this
+is determined in a static way by the source code but not by any run-time
+effect. If the network database changes, if code is regenerated the memory
+consumption will change but only then. Our concept is to use the heap
+diagnostics in DEBUG compilation to report the actual consumption and to
+make the heap size accurately fitting in the production compilation. Of
+course, the accurate settings needs to undergo an adaptation after each
+code regeneration. (Very sophistical elaborated code generation
+templates could even anticipate the required code size as they have all
+the knowledge about number of frames etc.) One has to understand that
+memory allocation errors can be definitely excluded for all the future if
+the compiled software passed the very first run-time test after code
+regeneration and build.
+
+One exception has to be mentioned. The timer API knows destruction of
+timer objects. As said, there's no concept of freeing memory for reuse.
+Instead, the timer implementation has a list of destroyed objects. This
+list is inspected first, when a(nother) timer object is created. If there
+a (destroyed) object in the list, then it is now resuscitated and reused
+rather then allocating new memory on the heap. This is a bit of dynamic
+memory allocation. However, even if event handlers make use of dynamic
+creation and destruction of timer objects then there's in far the most
+cases still an upper bounds for the memory consumption but it now becomes
+data and event dependent when this limit will be reached. The matter is:
+When can we safely query the diagnostic API to get the maximum memory
+consumption. If testing is too short then we will not get the absolute
+maximum yet.
+
+To avoid this problem, the timer API has been designed such that dynamic
+timer creation and deletion can be completely avoided. Instead, a timer
+can be suspended and re-triggered by the application code; the object as
+such is permanently existing. All ever needed timer objects are created in
+the initialization phase but at run-time no single byte will be allocated
+on the heap any more. The diagnostic API can be safely queried for the
+memory consumption already at the end of the code initialization.
+
+### Machine alignment ###
+
+The implementation of the heap depends on the common machine alignment.
+This has to be configured at compile time in the configuration header
+file of the CAN interface.
+
+### Critical section code needed? ###
+
+All dispatcher engines share the same heap. If dynamic timer creation and
+deletion is used (opposed to our recommendation, see above) than a
+critical section is required by the implementation. No such code is
+required otherwise. This has to be configured at compile time in the
+configuration header file of the CAN interface.

The CAN Interface modified by Peter Vranken

Peter Vranken — Wed, 09 Sep 2015 20:44:48 -0000

--- v1
+++ v2
@@ -1,4 +1,4 @@
-# CAN interface #
+# CAN Interface #

 This open source project proposes a CAN interface for embedded
 applications. Although it meets a lot of demands of real applications,

The CAN Interface modified by Peter Vranken

Peter Vranken — Wed, 09 Sep 2015 20:23:53 -0000

CAN interface

This open source project proposes a CAN interface for embedded
applications. Although it meets a lot of demands of real applications,
it's still considered a suggestion only; at many points it can be tailored
or conceptually changed and a claim of this project is that its tools and
code elements will still be beneficial even in these cases. It's not a
fixed solution but a framework for modelling the best suitable solution in
a particular environment.

The offered solutions capture the parts of the CAN stack from the network
layer till the application interface. An interface for sending and
receiving CAN frames is required at the bottom and a race-condition free,
signal based application code interface is offered at the top; the
application interface can be data or function based and it offers not only
the signal values but also a detailed transmission status, including bus
status, timeout information and sequence counter and checksum validation
results.

Concept

The basic concept of the interface is the dispatcher engine. Such an
engine has an event queue. Events are (usually but not necessarily)
related to a frame and they are delivered to the handler of that event. In
a typical environment the input of the queue will be connected to a system
CAN interrupt, be it for notification of a received frame or a send
acknowledge. The output of the queue is read by the main function of the
dispatcher engine. This call of this function will usually be made from an
application task, thus not in the context of the notifying interrupt. The
main function delegates the events to the registered handlers and since
this is all done in the context of the application task are there no
further race-conditions. The (thread-safe implemented) queue decouples the
interrupt context from the application task.

Any number of dispatcher objects can be created. An addressed use case is
an architecture with more than one (regular) application task and a
dispatcher for each task. Each CAN frame is connected to the appropriate
dispatcher. The association will be based on functional aspects and is
made by the application code at initialization time. The frame will later
be handled in the application task, which owns the associated dispatcher.
It's also possible to associate dispatchers with physical CAN buses
(normally CAN buses will share a dispatcher) or with kinds of events, e.g.
frame reception events opposed to bus state change events opposed to frame
send acknowledge events.

The dispatcher engine including the necessary registration process for
frames and events is available as hand coded C source code. It has a
well-defined API to both sides (system and application code) and is
considered a fixed code core. However, even for this core implementation,
tailoring is possible. In principle, the other elements of this project can
still be quite beneficial if the dispatcher core is not used at all. And if it
is applied it could undergo some tailoring, most likely for sake of
optimization. In the current release the code is designed for execution
speed but with some simple, straightforward changes the trade off can be
shifted towards less RAM consumption.

The application interface of the dispatcher is callback based. A received
event is notified by invoking the registered callback. From within this
callback a set of API functions is available that permits to query all
information about the event. In the first place the contents of a frame
can be fetched in case of a reception event. The second element of this
API is a couple of timer functions. Timer objects can be created, started,
stopped, re-triggered, etc. and they will create another event at due
time. Using timers the callback can easily implement timeouts for frame
reception or different simple or complex send patterns for outbound
frames.

It's important to understand that the core implementation ends with the
invocation of the callback. It's a matter of application design, what the
callback does. This starts with the decision if there is one shared
callback for all frames or for all kinds of events or if individual
callbacks are used. In the former case large switch cases and/or nested
data structures will be applied and in the latter case many different but
similar functions will be needed.

The intended use case of our open source project is that all the code or
data structures, which is between the dispatcher's API and the interface
with the functional application code and that dependents on the
definitions of the frames in the network cluster is generated by the code
generator, which is the core tool of this project. The code generator
parses the network specification from a set of CAN bus related network
database files (*.dbc) and can render this information in nearly any
textual representation - in particular as C source code.

We present two sample integrations of our CAN interface. Both let the code
generator make the interface with the functional application code and all
the required callback code, which is needed to serve this interface. The
interface with functional application code is data oriented (there's a
global struct per frame, whose fields hold the values of the frame's
signals) and if desired it's a matter of only a few lines of additional
template code to shape a functional interface instead.

The samples know different frame characteristics (strictly regular,
triggered by data change or data-change triggered plus periodic fallback
behavior) and all needed data flow code is generated in a fully automated
way; input are the network databases, which evidently have to accurately
specify all relevant frame and signal properties.

The samples support sequence counter and checksum validation. These
elements are integrated into the callbacks and the complete control code
is generated (whereas e.g. the actual checksum computation is considered
an external function, which is just invoked by the generated code). Again,
this only holds if having or not having a checksum and/or sequence counter
and the kind of checksum and the counter's range, etc. is properly
specified in the network databases.

The sample integrations are intended to demonstrate the capabilities of
the concept. They won't be usable out of the box in any other environment
but they will be a good starting point for customizing the offered
solution to the needs of that environment.

(to be continued)