#2130 Inefficient rectangular selection in SciTE

[Piotr Fusik]

I use SciTE 4.2.0 64-bit on Windows 10 1903.
I have a two megabyte, 40-thousand-lines text file with no syntax highlighting, no special characters and no excessively long lines. Making a rectangular (using Shift+Alt+click) spanning all the lines takes about a minute on an i7, with one core utilized fully and a "not responding"message in the title bar.
Why does this take so long? Could it be improved?

[Neil Hodgson]

Its laying out the text on each line to work out where the left and right side are as document positions. Much of the time is in setting up a graphics context which is done twice per line.

Retaining a graphics context over the whole operation could speed it up but that breaks through the current layering. That would leave MeasureWidths as the main cost and that can be reduced by increasing the level of caching with cache.layout=3.

Here's part of a profile:

Function Name   Total CPU [unit, %] Self CPU [unit, %]
Editor::SetRectangularRange 2915 (93.85%)   1 (0.03%)
Editor::SPositionFromLineX  2912 (93.75%)   0 (0.00%)
AutoSurface::AutoSurface    1659 (53.41%)   1 (0.03%)
Editor::ButtonUpWithModifiers   1505 (48.45%)   0 (0.00%)
Editor::ButtonDownWithModifiers 1420 (45.72%)   0 (0.00%)
EditView::SPositionFromLineX    1246 (40.12%)   7 (0.23%)
EditView::LayoutLine    1170 (37.67%)   12 (0.39%)
PositionCache::MeasureWidths    1125 (36.22%)   2 (0.06%)
SurfaceD2D::MeasureWidths   1118 (35.99%)   9 (0.29%)
SurfaceD2D::SetScale    846 (27.24%)    3 (0.10%)
Editor::Paint   150 (4.83%) 0 (0.00%)
ScintillaWin::WndPaint  150 (4.83%) 0 (0.00%)
EditView::PaintText 141 (4.54%) 0 (0.00%)
EditView::DrawLine  121 (3.90%) 0 (0.00%)
EditView::RetrieveLineLayout    69 (2.22%)  3 (0.10%)
DrawTranslucentSelection    50 (1.61%)  13 (0.42%)
SelectionRange::Intersect   37 (1.19%)  37 (1.19%)
LineLayoutCache::Retrieve   34 (1.09%)  3 (0.10%)
Selection::CharacterInSelection 33 (1.06%)  33 (1.06%)

[Piotr Fusik]

Thank you for your analysis!

I was editing the result of "dir /b /s", removing the common directory prefix. Since the selection in every line is identical, would it be feasible to reuse the previously measured width?

[Neil Hodgson]

Performing the detection would also take time for what I expect would be a rare case. Long columns of differing values from a database dump or similar would be the more common case.

Detecting that the selection is the same on each line requires resolving the selection to positions which is the costly operation. So, what you probably want to do is determine the positions on the first line then check if each line has that much of a common prefix. I don't think this approach is worth the additional complexity.

[Piotr Fusik]

If there is no easy solution, I'm okay with closing this as "won't fix".
Search&replace is an obvious alternative for my task.

[Neil Hodgson]

It is quite likely that performance can be improved in some way even if I can't see a good direction currently. Caching tends to get slathered on every performance problem so there's likely some way to use it here.

Someone else may want to work on this issue so I'll leave it open.

[Neil Hodgson]

Performance improved by reusing surface with [703e30].

For a 100 MB file containing repeated copies of SciTEBase.cxx, selecting first 80 pixels of each of the first 1,000,000 lines took 64 seconds and now takes 17 seconds on a 3.4 GHz i7-6700.

Syntax highlighting improves speed as the widths of lexemes are cached so there will be less improvement to files without syntax highlighting.

Here's the most significant part of a profile after the change:

Function Name   Total CPU [unit, %] Self CPU [unit, %]
Editor::SetRectangularRange 16041 (92.22%)  50 (0.29%)
EditView::SPositionFromLineX    15947 (91.68%)  115 (0.66%)
EditView::LayoutLine    14499 (83.36%)  275 (1.58%)
PositionCache::MeasureWidths    13534 (77.81%)  409 (2.35%)
SurfaceD2D::MeasureWidths   12904 (74.19%)  97 (0.56%)
[External Call] DWriteTextLayout::GetClusterMetrics 11474 (65.97%)  11474 (65.97%)
Editor::ButtonUpWithModifiers   8427 (48.45%)   0 (0.00%)
Editor::ButtonDownWithModifiers 7835 (45.04%)   0 (0.00%)
EditView::RetrieveLineLayout    1301 (7.48%)    26 (0.15%)
Editor::Paint   1103 (6.34%)    0 (0.00%)
ScintillaWin::WndPaint  1103 (6.34%)    0 (0.00%)
EditView::PaintText 1098 (6.31%)    0 (0.00%)
EditView::DrawLine  902 (5.19%) 0 (0.00%)

Performance can always be seen as a problem, no matter how fast. I am marking this issue as fixed.

[Neil Hodgson]

Performance improved by reusing surface with [703e30].

For a 100 MB file containing repeated copies of SciTEBase.cxx, selecting first 80 pixels of each of the first 1,000,000 lines took 64 seconds and now takes 17 seconds on a 3.4 GHz i7-6700.

Syntax highlighting improves speed as the widths of lexemes are cached so there will be less improvement to files without syntax highlighting.

Here's the most significant part of a profile after the change:

Function Name   Total CPU [unit, %] Self CPU [unit, %]
Editor::SetRectangularRange 16041 (92.22%)  50 (0.29%)
EditView::SPositionFromLineX    15947 (91.68%)  115 (0.66%)
EditView::LayoutLine    14499 (83.36%)  275 (1.58%)
PositionCache::MeasureWidths    13534 (77.81%)  409 (2.35%)
SurfaceD2D::MeasureWidths   12904 (74.19%)  97 (0.56%)
[External Call] DWriteTextLayout::GetClusterMetrics 11474 (65.97%)  11474 (65.97%)
Editor::ButtonUpWithModifiers   8427 (48.45%)   0 (0.00%)
Editor::ButtonDownWithModifiers 7835 (45.04%)   0 (0.00%)
EditView::RetrieveLineLayout    1301 (7.48%)    26 (0.15%)
Editor::Paint   1103 (6.34%)    0 (0.00%)
ScintillaWin::WndPaint  1103 (6.34%)    0 (0.00%)
EditView::PaintText 1098 (6.31%)    0 (0.00%)
EditView::DrawLine  902 (5.19%) 0 (0.00%)

Performance can always be seen as a problem, no matter how fast. I am marking this issue as fixed.