#1093 [PATCH] Lexer for Intel HEX file format

[Markus Heidelberg]

These patches adapt LexHex.cxx to support Intel HEX in addition to Motorola S-Record. Patches 1-3 fix/extend some S-Record stuff, patch 4 prepares some helper functions for reusing in Intel HEX, patch 5 adds the new lexer. In the example file, the first section is valid and the second section contains errors (unknown type, wrong byte count, wrong checksum).

I wonder whether it would be better to use the same lexical states for both lexers because they are mostly equal with few states only occuring in one of the lexers. I.e. SCE_HEX_ instead of SCE_SREC_ and SCE_IHEX_.

Another tought is about reordering the states to keep related states side by side. And also keeping gaps between groups for extensibility, is this usual?

I know, the states must not be changed for compatibility reasons, but maybe this is not true as long as the states/lexers have not been released yet?

[Markus Heidelberg]

Patches for Scintilla.

[Markus Heidelberg]

Patches for SciTE.

[Neil Hodgson]

The states are similar enough that sharing would be reasonable. Sharing may lead to better support in projects using these lexers.

If you want to renumber states then do it before this is in a release. The next release is likely in 2 to 4 weeks.

Current lexers mostly avoid gaps so downstream projects do not expect them and may behave in unexpected ways.

State ordering is not generally important although it may lead to ordering in UI elements.

[Markus Heidelberg]

Patches v2 for Scintilla.
0001..0004 are unchanged.

[Neil Hodgson]

Coverity is warning of overflow:

static int CalcChecksum(unsigned int startPos, int cnt, bool twosCompl, Accessor &styler)
214{
215        unsigned int cs = 0;
216
    1. Condition pos < startPos + cnt, taking false branch
217        for (unsigned int pos = startPos; pos < startPos + cnt; pos += 2) {
218                int val = GetHexaChar(pos, styler);
219
220                if (val < 0) {
221                        return val;
222                }
223
224                // overflow does not matter
225                cs += val;
226        }
227
    2. Condition twosCompl, taking true branch
228        if (twosCompl) {
229                // low byte of two's complement
    3. overflow: Add operation overflows on operands ~cs and 1U.

CID 1260663 (#1 of 1): Overflowed return value (INTEGER_OVERFLOW)
4. overflow_sink: Overflowed or truncated value (or a value computed from an overflowed or truncated value) ~cs + 1U & 0xffU used as return value.
230                return (~cs + 1) & 0xFF;
231        } else {
232                // low byte of one's complement
233                return (~cs) & 0xFF;
234        }
235}

[Markus Heidelberg]

Patch 0004 v3 for Scintilla.
Variable "cs" is now signed int and I have changed the return line to:
return -cs & 0xFF;

[Markus Heidelberg]

Patches v2 for SciTE.
0001 is unchanged.

[Neil Hodgson]

Committed as [50344d], [8ace46], [f4f5cb], [bde286], [058f4a], [dc22b7], [db73bf], [84d1ad], [315e7c], [25548c], [e85d2a], [f952d1].

Since I don't have a paid Coverity license, it can only be run once a week so I don't know yet whether it will OK the updated checksum calculation.