#1316 Transpose multiple selections

[Dave Keenan]

I first posted this as a Notepad++ feature request here:
https://github.com/notepad-plus-plus/notepad-plus-plus/issues/6226
but was told it would be better implemented at the level of Scintilla. So I respectfully request that changes be made to Scintilla that will enable, or simplify the implementation of, the following features in Notepad++.

When multi-editing (Ctrl+select) is enabled, and there are 2 selections, then Ctrl+T (Transpose) should swap the contents of those 2 selections. This should not affect the clipboard, and it should not affect which selection-position is the main selection. If there is only 1 selection, Ctrl+T should do as it does now (transpose lines). If there are 3 or more selections it should rotate their contents.

The same should apply to the rows of a rectangular or column selection (Alt-select) irrespective of whether multi-editing is enabled.

The Edit menu should contain an item "Transpose Ctrl+T". This might change from "Transpose Lines" to "Transpose Selections" to "Transpose Rows" depending on the nature of the selection(s).

A subset of these features exist in Sublime Text:
https://stackoverflow.com/questions/16801313/swapping-text-selections-in-sublime-text-2
and in Visual Studio (via a plug-in):
https://marketplace.visualstudio.com/items?itemName=SSDiver2112.SwapSelection

[Neil Hodgson]

There is currently a SCI_LINEREVERSE method to reverse lines and I expect a "reverse selections" method would be useful for columns. Its simple to understand how reverse generalises to different numbers of selections. If there was a "rotate" then symmetry could lead to a "rotate backwards".

The Edit menu is not controlled by Scintilla.

[Dave Keenan]

Hi Neil. Thanks for responding. Yes, it might be useful to have "reverse selections" and "rotate selections backwards" as well as "rotate selections". Since my primary desire is to have Ctrl+T swap 2 selections, and since any one of those methods would have that effect on 2 selections, the question for me is, which one should be bound to Ctrl+T with its mnemonic of "transpose". In general usage "transpose" can refer to any kind of permutation including rotations and reversals, but FWIW anyone familiar with its musical meaning would expect a rotation, not a reversal.

My (admittedly weak) scenario for the usefulness of column rotation is having accidentally omitted a value from a table. By typing the missing value at the bottom and making the appropriate rectangular selection, it could be rotated into the correct position with a single Ctrl+T.

Another consideration is that repeated rotations of n selections produce n different results (including those equivalent to backwards rotations) whereas repeated reversals only ever produce 2 results.

A backwards rotation might be achieved by selecting the column from bottom to top and using Ctrl+T. Or if the direction of Alt-selection has no effect on the order of the multiple selections, then rotate backwards could be Ctrl+Shift+T. Reverse could be Ctrl+Alt+T.

[Neil Hodgson]

The menu item name and key binding are application-level issues. There is a default keybinding for Ctrl+T but applications commonly override the default keybindings.

The ambiguity of 'transpose' argues for distinct API names based on "rotate" and/or "reverse".

[Dave Keenan]

I agree. "SCI_ROTATESELECTIONS", "SCI_BACKROTATESELECTIONS" and "SCI_REVERSESELECTIONS" (all plural) make sense to me. Before posting this feature request, I briefly got excited when I saw SCI_ROTATESELECTION (singular). It might have been better called SCI_ROTATEMAINSELECTION, or when considered alongside SCI_SETMAINSELECTION and SCI_GETMAINSELECTION, it could have been called SCI_INCMAINSELECTION (understood to increment modulo the number of selections).

I think the forward direction of rotation should correspond to the following pseudocode.

n = numSelections;
temp = selection[n];
for i = n downto 2
    selection[i] = selection[i-1];
selection[1] = temp;

Notepad++ presently relies entirely on the default keybinding of Ctrl+T for SCI_LINETRANSPOSE. It does not appear in any menu.

Would you provide default keybindings for SCI_ROTATESELECTIONS, SCI_BACKROTATESELECTIONS and SCI_REVERSESELECTIONS? If so, could they be Ctrl+T, Ctrl+Shift+T and Ctrl+Alt+T respectively?

Would it make sense to have a method, say "SCI_TRANSPOSE", which would do an SCI_LINETRANSPOSE if there is a single selection, and do an SCI_ROTATESELECTIONS if there are multiple selections? SCI_TRANSPOSE could then have the default keybinding of Ctrl+T. Applications can always override this, and need not use SCI_TRANSPOSE at all, but those that do not presently override Scintilla's default Ctrl+T binding would automatically benefit while retaining the expected line-transpose behaviour.

[Neil Hodgson]

In the API, selection generally refers to the positions, not the content so that is why its called SCI_ROTATESELECTION. SCI_GETSELECTIONS doesn't retrieve the selected text which is done with SCI_GETSELTEXT nor does SCI_CLEARSELECTIONS remove text.

It is unlikely I will work on an implementation of this in the near future. The complexities bought up in the Notepad++ thread make it look non-trivial. I will accept a basic implementation that just handles the simpler cases and document it as provisional.

It would be safer for applications to try any new API before making it a default keybinding. Separate APIs for different actions allow applications to compose functions - for example, transposing two characters may be preferred for an empty selection, or a single selection may be "swap with next word". If some combination is popular enough then that might become another API.

[Dave Keenan]

In the related Notepad++ thread, sasumner has raised a good question about how Scintilla should behave when rotating rectangular selections, when virtual space is enabled for rectangular selections (as it is in Notepad++). See https://github.com/notepad-plus-plus/notepad-plus-plus/issues/6226#issuecomment-545483213

[Dave Keenan]

Thanks Neil (@nyamatongwe). That all makes perfect sense. I'm happy with the behaviour provided by the Lua script given here:
https://community.notepad-plus-plus.org/topic/18415/transpose-multiple-selections/1

However, the issues raised by rotation of rectangular selections containing virtual space have a counterpart in the copying and pasting of such selections. It seems like a bug to me, that when I copy and paste such a selection into a position with text to the right of it, virtual spaces are not turned into real spaces.

[Dave Keenan]

Hi Neil (@nyamatongwe). We now have a Lua script that demonstrates the desired behaviour when rotating rectangular selections containing virtual space. It also implements transposing two characters for an empty selection not at the start or end of a line.
See https://community.notepad-plus-plus.org/topic/18415/transpose-multiple-selections/11

[Neil Hodgson]

getSelectionNTextVirtual does not behave sensibly with a multi-line selection where the selection start has some virtual space. Say the text is (with \n line ends):

0
1

So line 0 starts at position 0 and line 1 at position 2. The user drags from 2 characters to the right of line 0 to just after "1", producing a selection (1+2v .. 3). Since the virtual space is before the selection, it should not be part of the selection text. However, getSelectionNTextVirtual adds some spaces to the text.

[Dave Keenan]

Ah. Good point. Notepad++ only enables virtual space for rectangular selections, so this script didn't have to deal with multi-line selections with virtual space. I'll give it some thought.

[Neil Hodgson]

The position and virtual space should be kept together and a SelectionNStartVirtualSpace and ...End... APIs could help here.

A SelectionNText API to retrieve the text of one selection may also help although, to stay compatible with other APIs which ignore virtual space, this wouldn't add the spaces like the getSelectionNTextVirtual Lua function. The case for SelectionNTextIncludingVirtualSpace doesn't appear that strong to me.

setSelectionNTextVirtual is complex and I don't fully understand the implementation. Maybe the Lua code would be better off treating the real text and virtual spaces as separate elements in a tuple instead of joining them together.

[Dave Keenan]

For readers that might come later, I note that this feature request is closely related to the Option to treat some virtual space associated with selections as real spaces feature request:
https://sourceforge.net/p/scintilla/feature-requests/1322/

getSelectionNTextVirtual does not behave sensibly with a multi-line selection where the selection start has some virtual space.

… SelectionNStartVirtualSpace and ...End... APIs could help here.

Indeed. I have implemented the equivalent for my fix below.

-- Get the number of positions the virtual start is past the real start, for the selection with index idx
local function getSelectionNStartVirtualSpace(idx)
    local anchor = editor.SelectionNAnchor[idx]
    local caret = editor.SelectionNCaret[idx]
    local anchorVirtual = editor.SelectionNAnchorVirtualSpace[idx]
    local caretVirtual = editor.SelectionNCaretVirtualSpace[idx]
    if anchor < caret or (anchor == caret and anchorVirtual < caretVirtual) then
        return anchorVirtual
    else
        return caretVirtual
    end
end


-- Get the number of positions the virtual end is past the real end, for the selection with index idx
local function getSelectionNEndVirtualSpace(idx)
    local anchor = editor.SelectionNAnchor[idx]
    local caret = editor.SelectionNCaret[idx]
    local anchorVirtual = editor.SelectionNAnchorVirtualSpace[idx]
    local caretVirtual = editor.SelectionNCaretVirtualSpace[idx]
    if anchor < caret or (anchor == caret and anchorVirtual < caretVirtual) then
        return caretVirtual
    else
        return anchorVirtual
    end
end


-- Get text of selection with index idx, converting any trailing virtual space to real spaces
local function getSelectionNTextVirtual(idx)
    local selStart = editor.SelectionNStart[idx]
    local selEnd = editor.SelectionNEnd[idx]
    local startVirtual = getSelectionNStartVirtualSpace(idx) -- Number of positions virtual start is past real start 
    local endVirtual = getSelectionNEndVirtualSpace(idx) -- Number of positions virtual end is past real end

    local str = editor:textrange(selStart, selEnd) or ""

    -- The following section was added to give the behaviour I expect when transposing rectangular selections
    -- that contain virtual space. Dave Keenan, 2019-11-01
    if selStart == selEnd then -- If the real part of the selection is empty
        endVirtual = endVirtual - startVirtual -- The virtual part that comes after, is the difference between end and start virtual
    -- else any start virtual is on a different line and so should not be subtracted from end virtual
    end
    str = str .. string.rep(" ", endVirtual) -- Convert virtual space at the end of the selection into real spaces at the end of the string

    return str
end

[Dave Keenan]

setSelectionNTextVirtual is complex and I don't fully understand the implementation.

I agree the code is hard to follow, but I hope you agree that the visible behaviour it produces, when rotating rectangular selections, is completely straightforward.

I now realise it is attempting to reproduce behaviour that you have already implemented, for dragging and dropping, or pasting, into a selection that may have virtual space at its start. Is there some way I can use the API to access that existing behaviour (without affecting the clipboard)?

[Neil Hodgson]

There doesn't appear to be an API for inserting at a position with virtual space and the methods that convert virtual space to real space RealizeVirtualSpace are called in situations that can be complex. The simplest could be in InsertPaste for SC_MULTIPASTE_ONCE where its followed immediately by inserting text at that position.

The target could be extended with virtual space SCI_SETTARGETSTARTVIRTUALSPACE, but that could leave stale virtual space influencing future operations so it should decay to 0 quite easily, such as when SCI_SETTARGETSTART or SCI_SETTARGETRANGE is called.

[Dave Keenan]

I have added Rotate Backwards and Reverse, of multiple selections and of characters within a single selection, in the Lua script here:
https://community.notepad-plus-plus.org/topic/18415/transpose-multiple-selections/15

They handle multi-line virtual selections now. But they still have a problem when there is more than one virtual selection on the same line.

[Neil Hodgson]

Initial versions of patches to allow the target to have virtual space which may be converted to real space when replacing the target with text and to retrieve the virtual space at start and end of multiple selections.

[Dave Keenan]

Thanks Neil. Good stuff.

[Neil Hodgson]

Updated patches to work on Cocoa and GTK and added documentation.

The full functionality requested in this issue has multiple choices (if the selection is empty / rectangular, etc.) that could be made differently. For example, a single empty caret could mean transpose lines or transpose characters. Its unlikely everyone will be pleased by the particular choices made (policy) and may want extended functionality (like transposing words if a single word is selected). Therefore its worthwhile providing the basic building blocks so that applications can define actions that suit their users and fit with their other features. I'd like to make sure that the additional APIs are sufficient for this.

The next step could be to define some 'middle-ground' APIs such as one for strictly rotating the selections, leaving the choice of whether to do this or transpose lines or characters (depending on selection shape) up to the application. That is, I do not see a do-everything Scintilla API in the near future. Such an API may be implemented after experience is gained with how people use the feature.

[Dave Keenan]

I totally agree with the approach you describe above. Yes, APIs for
• rotate selections forward, 0 1 2 -> 2 0 1
• rotate selections backward, 0 1 2 -> 1 2 0
• reverse selections, 0 1 2 -> 2 1 0
would be good.

I would hope their behaviour, in the cases where there are 2 or more non-empty selections, would be very similar to the behaviour of the three corresponding items near the end of my script in those cases.
https://community.notepad-plus-plus.org/post/48237

That includes the treatment of virtual space. e.g. a rectangular selection should remain rectangular (the same rectangle) when rotated or reversed, even when it includes some rows with virtual space before (no real text) and some rows with virtual space after.

But your version should work correctly when there are two or more virtual selections on the same line, unlike mine. :-)

[Neil Hodgson]

New APIs added with [80102f] and [5d73aa].

[Neil Hodgson]

There are some problems with the implementation of this with adjacent selections. Select two adjacent characters (say "ab") as two separate selections in this order: swipe over 'b' forwards; ctrl swipe over 'a' forwards; perform transpose operation. Only 'a' remains with 'b' deleted. This is sensitive to the ordering of operations and can also lead t overlapping selections. It is caused by the automatic movement of the other selections when one selection is being changed. Its possible to chase this away by performing operations in different orders but that just moves it to another pattern of operations.

Adjacent selections appear quite difficult to fix with complete generality. It may be uncommon to do this but it should not destroy data.

[Dave Keenan]

I see what you mean. I don't know what to do about that. Could it be considered a bug in the code that does the automatic movement of the other selections when one selection is being changed?

[Neil Hodgson]

Wrote up the bug with a potential solution as [bugs:#2140].

[Neil Hodgson]

[bugs:#2140] fix committed.

Considering adding a TargetFromSelectionN method but its just replacing 4 obvious calls with 1 so not really a strong benefit, partitularly if the calling code is also going to be examining virtual space and selection direction.

[Dave Keenan]

I don't have an opinion on that. I'll be happy with whatever you decide.