#1441 Avoid dynamic resize LineState

[Zufu Liu]

Unlike LineLevels, which is allocated once inside LineLevels::SetLevel(), LineState is dynamic resized inside LineState::SetLineState(). for lexer that uses line state, every line would have to be set, allocate once would improve the performance.

draft code:

int LineState::SetLineState(Sci::Line line, int state, Sci::Line lines) {
    assert(line >= 0 && line < lines);
    lineStates.EnsureLength(lines + 1);
    const int stateOld = lineStates[line];
    if (stateOld != state) {
        lineStates[line] = state;
    }
    return stateOld;
}

[Neil Hodgson]

Its similar to std::vector with exponential growth so its unlikely to be an issue particularly with the other costs occurring during a lex. Profiling a 400 MB Lua file, LineState::SetLineState() is 2.4% of the lexing time; 1.6% of lex+fold.

If you want to prepare a full patch, it can go in.

Checking for a change in value looks like unnecessary complexity to me, replacing a branch inside operator[] with one outside.

[Zufu Liu]

Full changes for LineState::SetLineState() (copied from LineLevels::SetLevel()).
Removed level comparison inside LineLevels::SetLevel() as almost every lexer only call SetLevel() when fold level diff.

As for value change check, maybe a special method bool UpdateValueAt() can be added to reduce branches in CellBuffer::SetStyleAt() and CellBuffer::SetStyleFor().

    const char curVal = style.ValueAt(position);
    if (curVal != styleValue) {
        style.SetValueAt(position, styleValue);

[Neil Hodgson]

That doesn't work because stateOld is shadowed.

[Zufu Liu]

Ouch, updated patch. in SetLineState3.diff, prev and stateOld is set to the parameter to prevent NotifyModified() in case of out bound set.

[Zufu Liu]

The change will change the behavior ofGetMaxLineState, not sure what it's used for.

[Neil Hodgson]

Its very early. I think the intention here was that line state wouldn't need to be allocated for the whole document (or at all) when it is used for a rare case like nested comments. Application styling code could check for the maximum length allocated before get/set. GetMaxLineState wasn't included in IDocument so it isn't used in Lexilla.

[Neil Hodgson]

Committed as [a61a35] with additional changes to unit tests and documenting the changed behaviour of SCI_GETMAXLINESTATE.