Border Pixel Buffer (Feature request)

[oneilmw]

This program does not work quite the way I would like it to.
Specifically, I don't like the way it handles extrapolating the border beyond the boundaries of the given image.
The file "Diamond Sword" is the default texture in Minecraft that I will be using as an example.
From what I can tell, the xBRZ algorithm extrapolates the out-of-bounds border pixels to be based on the in-bounds border pixels, so I am guessing it treats the borders such that they have the same color, as seen in the "Theory" file I made. When I upscale the "Diamond Sword" file using 6xBRZ, I get my "6x Default" image.
As you can see, the sword does not look terribly sharp; it looks blocky and unweildy.

My clunky, roundabout solution is to tell xBRZ that the out-of-bounds pixels sould be treated as transparent, and I did so by editing the image to give it a buffer border of transparent pixels. After running this modified file through 6xBRZ and removing 6 pixels from the 4 sides, I got what I was looking for, "diamond_sword Border Big." As you can see, the the blade's tip looks more like a sword tip, and is now visually consistent with the rest of the image.
This solution, from what I can tell, actually makes more sense than the current extrapolation implementation because it upscales the borders in a way that is consistant with the way it upscales everything else.

Looking at this, though, this is a solution specific to the result I am looking for. I am trying to upscale an image that A) has a clear subject with a transparent background and B) will never tile. This solution works for this case because I'm telling it to treat the borders like background, which the GUI kind of functions as, and the image's background (mask) IS transparent. If it were a mask color, I might want to tell xBRZ the color, so it could treat the buffer pixels as that color.
What if I was using an image that would tile, like a block in Minecraft? Neither of these implementations are ideal. The correct way for a tiling image would be to:

Copy each of the four outermost rows/columns
Switch the two row copies and switch the two column copies
Extend the canvas by one pixel in all four directions
Paste those copies on the corresponding (now side-swapped) border in the space you made
Run the filter
Reduce the canvas size by (the upscaling number of pixels) in all 4 directions.

This technique emulates having the same tile on all four sides.
So, depending on how the image is intended to be used may change how the algorithm should extrapolate the out-of-bounds pixels.
Can you implement these options and document them?

[Zenju]

I don't understand what you mean with "out-of-bounds border". Perhaps you're having an image that has both an alpha channel and a mask color? I don't see anything in Theory.png that would indicate that the outer pixels are not part of the image.

[oneilmw]

In "Theory.png," I took it upon myself to extend the border of the default texture by one pixel-showing how the algorithm seems to treat the outside border pixels as merely an extension of the inside border pixels.
I COPYPASTED THE BORDER OUT ONE PIXEL, because that's what the algorithm thinks is there. It's a demonstration of what the algorithm uses, then crops out.
Out-of-bounds border = outside border pixels = the information that a scaling algorithm needs to extrapolate past the practical canvas size to determine how the interior border (the outermost pixels that ARE inside the practical canvas size) would interact with the canvas' edge.
Border = the rectangle inside which contains the image; defined by the image's dimensions.
aka the rectangle that defines the canvas size.
If you still don't get it so far, I'll whip up a proper infographic with this terminology.
Also, for the purpose of this discussion, look up the difference between interpolation and extrapolation. xBRZ is an excellent interpolator and a very dull extrapolator when it comes to in-between pixels (interpolation) and out-of-bounds pixels (extrapolation) accordingly.

[oneilmw]

diamond_sword.png is the default texture.

x6 Default.png is diamond_sword.png through 6xBRZ. It's blocky near the border, and the border pixels are unnaturaly glued to the sides instead of forming smooth curves.

Theory.png is an image based on diamond_sword.png, modified with an extra line of pixels around the border which represent how I believe xBRZ treats filtering the border. These added pixels I made are the out-of-bounds border pixels, and are a duplicate of its in-bounds border pixels.

diamond_sword Border Big.png is the result of taking diamond_sword.png, adding a 1 pixel transparent border around it, passing it through 6xBRZ, then removing 6 pixels from all 4 sides.
As you can see, this contrasts from x6 Default.png by having the pommel and tip having smooth curves where they touch the border, which is much more consistent with the rest of the image.

Adjacent Extrapolation (default).png is an infographic showing how xBRZ extrapolates the out-of-bounds border. It's basically Theory.png with labels on the border, the in-bounds border pixels which are inside the desired canvas size, and the out-of-bounds border pixels which xBRZ must somehow extrapolate. This image shows the extrapolation technique currently implemented.

Mask Extrapolation (Requested).png shows the proper way to extrapolate the border pixels: Just use a flat color (in this case, transparent.)

Now, time for tiling. It makes no sense to use the following tiling system for the diamond sword, as seen in Tiling Extrapolation (Requested).png, because that image isn't supposed to tile at all. However, what about an image that IS supposed to tile?

stone.png is the default stone texture, and it is meant to tile.

stone 6x Default.png is stone.png through 6xBRZ. At first glance, it looks fine, and if it was a simple graphic, it would be so, but it is meant to tile.

Tile (verb):
To repeat an image in the vertical and/or horizontal direction.

stone 6x Default Tiled.png shows that the default extrapolation technique is inadequate for this image because the seams between two adjacent tiles are visible. You could play a game of tic-tac-toe on it. Not good.

The solution is to simulate what the input image would look like if it were tiled.

Tiling Extrapolation (Requested) - Stone.png shows the correct way to extrapolate the out-of-bounds border pixels: Copy each edge, paste it past the opposite edge, copy each corner, paste it past the opposite corner. This simulates where the other tiles would be. I don't exactly know if the corners are necessary, but it is a safeguard.

stone18.png is that extrapolation, ready to be passed through the current 6xBRZ. It's one pixel size increase on all 4 sides increases its resolution from 16x16 to 18x18, hence the name.

stone 6x Fixed.png is stone18.png passed through 6xBRZ, then with 6 pixels removed from all 4 sides. It is what stone.png SHOULD result in but doesn't. I had to do a lot of tedious work that could be automated.

Here's the value, though:

stone 6x Fixed Tiled.png is stone 6x Default.png, just tiled three times in both directions.
Compare stone 6x Default Tiled.png and stone 6x Fixed Tiled.png. The seams are no longer visible in stone 6x Fixed Tiled! My alogrithm worked, but it is entirely too much work to do manually for at least 487 images!

That is why I would like to be able to choose which extrapolation technique xBRZ uses.

Did you understand all that? I know it was a lot.

[Zenju]

Indeed for pixels outside the input image, the xBRZ algorithm just assumes duplicates of the border, while looking as far as 2 pixels beyond.
For images having an alpha channel a better solution is to just treat these outside pixels as transparent. A full-blown set of new parameters just to handle the border behavior seems a bit overkill. In the RGB case (= emulator use), the current behavior (duplication) is fine, for ARGB images, transparent outside pixels are a better default. I've implemented these changes in xBRZ 1.8.