Hi Alan
I don't think that's entirely correct regarding new Harfbuzz. It seems that it can use uniscribe or one of a few other backends, but these are optional and it can definitely be built without any of these options, because I've done so on Windows. I think it's a bit like Plplot -> if you want svg output then there is built in code to do that, however you could also do it via a wxWindows backend and probably via a Qt or Cairo backend. But to be honest I'm guessing a lot of this as the documentation is non-existant and I've only found one example which I can't fully build because it uses X to display the results.
The reason why I mentioned it was that if we start parsing text in plcore then I thought we might get to a stage where these text shaping issues spring up, But perhaps the higher level libraries will do this for us anyway. As far as the freetype backends are concerned you are right that they give poor results for non-western unicode scripts. All freetype can do is load the glyphs it is told to load from the font file it is told to load them from. It has no concept of font fallback (which is fontconfig's job  - no equivalent on Windows) or ligiatures or text direction (which is uniscribe or harbuzz's job). Regarding using Pango, both this and FontConfig are near impossible to build on Windows, I know because I've tried. Even with cygwin I think it's a challenge. I did once start and nearly finish writing something to do the font shaping for freetype, but it used uniscribe so was windows based only. The one (maybe only) advantage that freetype has is that it's antialiased text does look very nice. Even using the antialiased wxGraphicsContext backend, rotated text lookes quite jagged, at least on Windows. Although ironically on Linux there is a good chance wxGraphicsContext text is rendered using freetype!

From: Alan W. Irwin <irwin@beluga.phys.uvic.ca>
To: phil rosenberg <philip_rosenberg@yahoo.com>
Cc: "plplot-devel@lists.sourceforge.net" <plplot-devel@lists.sourceforge.net>
Sent: Tuesday, 8 October 2013, 20:32
Subject: Re: [Plplot-devel] [Plplot-general] overline underline not working

On 2013-10-08 03:02-0700 phil rosenberg wrote:

> Yes Harfbuzz is moderately low level. It is the equivalent level to
uniscribe on Windows, so it deals with things like converting a series
of text characters into a series of glyphs and probably things like
text direction. [...] Sorry it
was a bit off topic, but I thought it might be worth mentioning.

Hi Phil:

Your comment is right on topic since we do need to discuss from time
to time the libraries that PLplot depends on now or which it might
depend on in the future.

The discussion of HarfBuzz can be confusing because there are two
versions (see the wikipedia article) The old version may well have had
the features you state, but the new version is limited to text shaping
only.  Thus, with the new version a higher level library (Qt or Pango)
handles text layout issues, a separate low-level library (e.g., one of
UniScribe (on Windows), FreeType (on all platforms), or AGG (on all
platforms) handles rendering issues, and fontconfig (another low-level
library) handles font choosing issues.

[out of order]
> [HarfBuzz] is at a similar level to FreeType, so if we ever
wanted the FreeType backend to parse unicode correctly (something we
discussed some time ago) then this would be the tool to use.

Perhaps (see all remaining paragraphs except the last two), but in any
case only indirectly via another library such as Pango (see last
two paragraphs below).

The use of plfreetype may just die a natural death. The list of device
drivers that use plfreetype is limited to just wingcc, wxwidgets, and
the unmaintained and deprecated gd.  But at some stage we will likely
disable gd, the plfreetype use is optional for both wingcc and
wxwidgets, and only the wxDC and agg wxwidgets backends potentially
can even use plfreetype. Furthermore, If you compare the default text
handling with plfreetype text handling for the wxDC backend, e.g., by
using the commands

examples/c/x24c -dev wxwidgets -drvopt backend=0,freetype=0
examples/c/x24c -dev wxwidgets -drvopt backend=0,freetype=1

the first of those commands (which uses default text handling for the
wxDC case) produces essentially perfect results while the second
version (which uses plfreetype) has missing glyphs for Hindi, Korean,
and Mandarin, and the glyphs in the peace word in the Arabic and
Hebrew languages are layed out in incorrect (left-to-right) order. So
that comparison (if freetype is available on your platform at all)
nicely illustrates the issues with the plfreetype approach and
why it should be allowed to die a natural death or be replaced
completely.

What happens when you use default (non plfreetype) text handling
for the wxDC case, i.e., what happens when your run

examples/c/x24c -dev wxwidgets -drvopt backend=0,freetype=0

on Windows?  Do you get essentially perfect results (like
http://plplot.sourceforge.net/examples-data/demo24/x24.01.png) with no
missing glyphs and all glyphs rendered in the correct order?  If so,
that would prove that something better than plfreetype was available
on all platforms for the wxDC backend, and in that case we should simply
drop the plfreetype possibility from the wxDC backend.

That step would only leave the plfreetype possibility for gd (if
somebody wanted to maintain it), wingcc, and the agg backend of
wxwidgets. In those cases instead of waiting for a natural death a
possible way forward might be to eliminate the use of plfreetype
altogether by replacing calls to plfreetype by the equivalent calls to
Pango instead (i.e., indirectly use HarfBuzz, fontconfig, and freetype
rather than using freetype directly).  We already know how to use
Pango for text handling in the case of the cairo device driver so it
might be straightforward to use essentially the same code for the
device drivers mentioned above rather than using calls to plfreetype.
And, of course, that use of pango should be optional to cater to the
needs of those Windows users who prefer to limit the PLplot
dependencies at the expense of having less powerful PLplot
capabilities.

Are there any volunteers for looking further at the possibility of
modernizing either/both wingcc and the agg backend of the wxwidgets
device by replacing calls to plfreetype with calls to Pango along the
lines I have suggested?

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________