#384 GM not handling Unicode file names

[Daniel Cheung]

GM is not recognizing files name with Chinese characters

Platform: Win10
Version: GraphicsMagick 1.3.20
Code: gm -identify 獅藝學會.jpg
Returned:

>gm -identify 獅藝學會.jpg
gm identify: Unable to open file (????.jpg) [Invalid argument].
gm identify: Request did not return an image.

I saw the -encoding parameter on the site's doc, but it's nowhere to be found on a Windows.

[Daniel Cheung]

This supposed bug is referenced on StackOverflow, http://stackoverflow.com/questions/36907644/graphicsmagick-unable-to-process-unicode-filenames

[Bob Friesenhahn]

I think that it would be necessary to use a file opening function supporting wide characters (UTF-16). There is still the issue of how a simple character string (command line argument) can be passed to such a function. If the argument string is assumed to be encoded in UTF-8 then it can be translated to UTF-16 before being passed to Windows file opening function. This should become the normal case for Windows. We would also need special handling for functions which test for the existence of files or produce lists of file names from the operating system.

Some other OSs store filenames as UTF-8 (or a native 8-bit character set not containing 0) and the file paths/names are simply preserved so that GM is able to open such names on an OS that behaves in this way.

[mrrhq]

On GNU/Linux, you'd think that GNU Unifont would be able to handle something like this. Yet another bigger problem is that GraphicsMagick still uses Xorg Fonts on Linux. I know this is a Windows-related question, but just something to consider: What about full TrueType and ClearType support entirely from top to bottom? On Linux, it should use the newer Fontconfig.

My system has no X Fonts installed, since I'm slowly transitioning to Wayland. Every time I open up GM, it always comes up with something like gm animate: Unable to load font (-*-helvetica-medium-r-normal--12-*-*-*-*-*-iso8859-1) [Resource temporarily unavailable].

Maybe UTF-8 is good enough? Most fonts and glyphs can be supported with base UTF-8 with Basic Multilingual Plane (BMP).

[Bob Friesenhahn]

On Mon, 30 May 2016, mrrhq wrote:

On GNU/Linux, you'd think that GNU Unifont would be able to handle
something like this. Yet another bigger problem is that
GraphicsMagick still uses Xorg Fonts on Linux. I know this is a
Windows-related question, but just something to consider: What about
full TrueType and ClearType support entirely from top to bottom?
On Linux, it should use the newer Fontconfig.

I agree that GraphicsMagick should be using/supporting Fontconfig.

My system has no X Fonts installed, since I'm slowly transitioning
to Wayland. Every time I open up GM, it always comes up with
something like gm animate: Unable to load font (-*-helvetica-medium-r-normal--12-*-*-*-*-*-iso8859-1) [Resource temporarily unavailable].

This is a situation that I was unaware of since I have not used a
system with this limitation.

Maybe UTF-8 is good enough? Most fonts and glyphs can be supported with base UTF-8.

I think that the font selection/rendering issues are distinct from the
file opening problem under Windows. The file opening problem under
Windows is due to needing to use the wide-character version of
fopen(). It is necessary to convert the filename from its current
encoding (which may be UTF-8, or could be an encoding based on current
locale) into the UTF-16 that Windows requires.

Regardless, the fix for these issues will (unfortunately) not be in
the forthcoming release cycle.