Git Repository Merge Request #24: WIP - Kitty terminal drivers (merged)

This patch adds 'kittycairo' and 'kittygd' terminals supporting the Kitty Graphics Protocol (https://sw.kovidgoyal.net/kitty/graphics-protocol/) which is a more modern, more compact, truecolor alternative to sixel, supported by the Kitty and Konsole terminal emulators, among others.

It's built atop the existing PNG renderers (cairo and gd), and the existing base64-embedding code (which gets a bit of an overhaul to allow it to output bytes through a callback instead of just outputting to a FILE*, so that we can chunk and wrap the data properly).

This is a WIP — some error handling is missing, the documentation mostly hasn't been written, and I'm sure I've gotten some code-style stuff wrong — but I've tested it locally and it works great. If the patch is interesting then please get back to me and I'll clean it up. I'm also on IRC (hobbs) for more interactive chat.

[Andrew Rodland]

In action: https://vimeo.com/823613499/aab4928b77

[Andrew Rodland]

The animate flag could theoretically be put to better use, because kitty graphics protocol has actual animation support, but I haven't done that yet.

[Ethan Merritt]

Quick review.

In general it looks very promising.

1. minor issue - it doesn't make sense to #include kitty.c both from term.h and from gd.trm. Either include it once from term.h to satisfy all possible users, or include it in parallel from the terminals that want it (cairo.trm, gd.trm)
2. What is the significance of the size of kitty_buf? Does it relate to the size of the terminal window?
3. The following sequence sometimes results in locking up the kitty terminal output. Not 100% reproducible; often it works without obvious error.
   set term kittygd
   load 'animate.dem'
   <cr> to start animation
   ^C to interrupt after a few frames.
   At this point output to the kitty terminal [sometimes] becomes jammed. You can switch to another terminal and plot normally, but if you switch back to kittygd it remains jammed. I imagine some sort of cleanup/reinitialization of the callback output routine is neeeded but I don't know exactly what or where to put it.</cr>
4. I would appreciate a concise statement of what advantages the kitty terminal(s) bring. I gather there is a kitty terminal emulator that supports this protocol and not sixel, but there are about a bajillion terminal emulators out there and more of them support sixel than kitty so I don't think that's a very strong argument by itself. I am thinking that maybe the biggest plus is that sixel+truecolor necessarily requires libgd, whereas you could build kittycairo even if for some reason you don't want to add a libgd dependence.
5. I am intrigued by your mention of possible animation support. What exactly would that mean? I had previously wanted to make 3D plots in the sixel terminals responsive to mouse or arrow events to rotate the view angles but coudn't figure out how to do that since it seemed to be more a manipulation of the terminal input layer (bypassing readline) than anything to do with the output protocol. Is there something in the kitty protocol that can forcibly catch keystrokes that would otherwise go to stdin?

[Andrew Rodland]

1. Yes, that's a simple mistake, I meant to have it in term.h only.
2. No, it's a protocol constraint, because the terminal itself will need to have a buffer of that size while parsing escape codes. The kitty graphics doc says that everyone should support up to 4096.
3. Interesting. I'll look into it, you're probably on the right track. Or it could be related to the lack of error handling, if something is in a bad state and throwing an error, and it's currently just getting swallowed.
4. Indeed, most terminals that support any kind of graphics support sixel. The advantages of kitty graphics are that they're truecolor rather than indexed-color, so they look better. sixelgd's "truecolor" is still indexed, just with a non-fixed palette, so you can only have 256 distinct colors in an image or else you start to see dithering crud. Kittycairo also looks prettier in general because it's cairo.
5. Kitty graphics has two modes: there's a mode where you send an image and have it displayed immediately (which is what we're doing now), and there's a mode where you send an image to a numbered surface, and then request that surface to be displayed at a given position. In the later mode you can also draw a new image to the surface, or overdraw any rectangular subset of it, and the display will update in-place.
   5b. About mouse support: kitty has the same mouse support that most newer terminals do (actually, I'm a Konsole user, not a kitty user). It should be possible to handle it and have interactivity at the terminal. But yes, it's just a bunch of escape codes on stdin. I think that instead of bypassing readline, it's possible to work with readline and add the mouse escape codes as "keybindings", so that all you have to do is send the magic "yes, I want mouse clicks" code to the terminal, and then you will receive those interactions as readline events. It's totally not my field, and it seems sparsely documented, but I think it's possible.

[Ethan Merritt]

Now merged, and integrated with the new pseudo-mousing routines for interactive response to arrow keys during "pause mouse".

Thank you for your contribution to gnuplot!

[Andrew Rodland]

Thanks! I did intend to come back to this and do more cleanup, I just got quite busy. So thank you for taking it to the finish. Let me know if there's anything else I can lend a hand with.