Ark - 2015-02-14

Thinking about the lens, I consider a new rendering pipeline (if you want to call it this way) to improve memory usage, speed and responsiveness of MComix, especially when dealing with large images. The general idea looks like this:

  • Always use PIL Image objects for any operation, i.e. only use gtk.Image if there is no other way.
  • Create working copies only if necessary. These copies should be as small as possible.
  • Apply all affine transforms at once, including scaling.
  • Apply all color ops (contrast, brightness etc.) at once. This reduces noise and should improve both memory usage and speed (as opposed to the current implementation of "Enhance image" where there are new images for each color op with accumulated rounding errors, at least if these ops are not lazy). This might need some testing to check whether it really works as intended.

There are some pitfalls:

  • The resampling method used for the affine transforms might need some more pixels from the environment due to the support the respective interpolation function might have. This also applies to sharpening and blurring.
  • Pillow's affine transform function does not accept ANTIALIAS as interpolation filter. Pillow's resize function, however, does. This issue has already been reported to the Pillow developers. In the meantime, we might need to work around this issue which means that we cannot apply all affine transforms at once but have to introduce an additional step.
  • Pillow's functions are not lazy in general. In order to achieve good results, we must apply sharpening/blurring before rescaling. However, if there is a large image and you zoom out, sharpening/blurring might take a very long time and a lot of memory. I consider cheating in this case. That is, if upscaling is necessary, everything is performed as usual. However, if downscaling is necessary (due to "zoom to fit", for example) sharpening/blurring is applied to the downscaled image rather than the original image. In order to compensate the relatively large radius in this case, I consider simply weakening the effect of the convolution. I don't know whether this is a valid approximation but a few tests in GIMP suggest that it might work quite well.
  • Since some resampling methods and filters might produce pixel values below 0 or above max, it might be a good idea to use floating point values rather than integers for the temporary images. However, MComix is a simple viewer and not a scientific image restauration application, so integers should work just fine.

Thus, there are at least two cases to consider:

  • Case 1 (scaling factor > 1):
    • Create a pre-cropped working copy of the region of interest (including border pixels for the support).
    • Apply sharpening/blurring to the working copy. (This might create a new working copy.)
    • Apply the resulting affine transform to the working copy. (This will create a new working copy.)
    • Apply all color ops to the working copy.
    • Create a converted (and possibly cropped) image so PyGTK can render the result.
  • Case 2 (scaling factor <= 1):
    • Apply the resulting affine transform to the original image. (This will create a new pre-cropped working copy.) Make sure you keep enough border pixels for the convolution.
    • Apply sharpening/blurring to the working copy. Reduce this effect to compensate the new radius.
    • Apply all color ops to the working copy.
    • Create a converted (and possibly cropped) image so PyGTK can render the result.

This way, we only need to create working copies that are as big as the screen (and some more pixels for the border) at most. Furthermore, this approach might work for the main area as well as the lens or the thumbnails, eliminating redundancies in the code.

I attached a first draft. The only things it can do right now are "flip" and "rotate". Simply run it in a working directory that contains a "test_input.png" and it will create a "test_output.png". (The demo code shows a quite complicated way to do nothing so the result will be a simple copy as the matrix implies.)

The bad thing is that I don't have that much time right now so any help would be greatly appreciated.

EDIT: Some more details added, scaling factor = 1 pushed to case 2.

 
new_rendering_draft.py