paps-discuss

[Paps-discuss] Combining characters not rendered properly

[<Tho...@si...>]

There are still problems with positioning and spacing of combining 
characters which are demonstrated with the attached example file; 
user paps --font "monospace 20" for a nice demo output to check 
on screen.

While combining characters seem to be handled well on the Thai line, 
they are not on the Latin letters:
* spacing is wrong; each combining character is followed by an extra space
* the position of six specific Latin combining accents (starting with 
  the grave accent U+0300) is wrong, it's not on top but rather on the 
  next line position! (Dov said in a mail last year this was a pango 
  problem - since you know the pango interface being used, it's 
  probably better the paps maintainers report the problem there than 
  if I would do it)

Also, the combining accents look somehow unmatched to the letter style, 
they seem to be too small and too thin, and poorly placed inside the 
base characters, rather than attached to them (on top or bottom, 
respectively).

Kind regards,
Thomas

Re: [Paps-discuss] Combining characters not rendered properly

[Jan W. S. <jst...@pl...>]

Tho...@si... wrote:
> There are still problems with positioning and spacing of
> combining characters which are demonstrated with the attached
> example file; user paps --font "monospace 20" for a nice demo
> output to check on screen.

This is interesting. It seems to depend on the font being used.

The sequence 0x61 0xCC 0x80 (a followed by combining `) should
produce something looking like =C3=A0. It does not, when the font is
"monospace" on my system. In my case, "monospace" is actually
Bitstream Vera Sans Mono. Also, when I say

=2E/paps --font "Courier New 20" testpaps >testpaps.ps

the accents are wrongly placed.

However, when I say

=2E/paps --font "freemono 20" testpaps >testpaps.ps

the accents are placed OK! (Perhaps apart from U+323; but I do not
know where it ought to be placed).

This suggests that several fonts have bugs (fonts are basically
"interpreted programs" nowadays, so they can have bugs!):

-- either both Bitstream Vera Sans Mono and Courier New have bugs;
-- or Freemono has bugs, and somehow Pango uses a work-around
   which is itself a bug, causing it to fail with the other
   fonts. I'll investigate this using fontforge; in the meantime,
   could you confirm the different behaviour of the fonts?

> since you know the pango interface being used, it's probably
> better the paps maintainers report the problem there than if I
> would do it)

By "you" I suppose you mean the collective readership of this
list. I am sure the developers read this list. Hello, hello, Dov,
Akira, can you confirm this?

Regards, Jan

Re: [Paps-discuss] Combining characters not rendered properly

[Dov G. <dov...@gm...>]

Oops. I answered last time only to Jan instead of to the list. My
fault. Here is a forward, Jan's answer and my answer.

On 12/7/06, Jan Willem Stumpel <jst...@pl...> wrote:
> Dov Grobgeld schreef:
> > Hi Jan. :-)
> >
> > Yes, I read the list, though I don't always have the time to answer.
> >
> > Indeed it seems to be a font problem as may be seen by loading the
> > text in gedit. The rendering of paps is the same as in gedit, which is
> > exactly what it is supposed to be.
>
> Yes.. I'll do that. Means I have to install gedit first. I'll do
> it for the good cause, but I don't like it because gedit is
> terribly unstable, and it is an "imperialistic" program, which
> demands I install lots of other stuff as well:
>
> vega:~# apt-get install gedit
> Reading package lists... Done
> Building dependency tree... Done
> The following extra packages will be installed:
>   gnome-media-common libgnome-media0 libgtop2-7 libgtop2-common
>   libmetacity0 libnautilus-burn3 libpanel-applet2-0
>   libtotem-plparser1 libwnck18 metacity-common python-glade2
>   python-gnome2-desktop python-pyorbit
>
> Suggested packages:
>   python-gnome2-desktop-doc
>
> Ah well.. I can always wipe all this stuff afterwards.

You can always compile /usr/share/gtk-2.0/demo/textview.c or use
gtk-demo and change the font in ~/.gtkrc-2.0 instead.

Or use inkscape, or gimp, or even my program guci
(http://imagic.weizmann.ac.il/~dov/freesw/gtk/guci/ ) or any other gtk
program.

Regards,
Dov

Re: [Paps-discuss] Combining characters not rendered properly

[Thomas W. <mi...@to...>]

Hello, for the record (and maybe in support of a pango bug report),=20
I'd like to share my further observations about misplaced combining=20
characters with various fonts:

> > There are still problems with positioning and spacing of
> > combining characters which are demonstrated with the attached
> > example file; user paps --font "monospace 20" for a nice demo
> > output to check on screen.

> This is interesting. It seems to depend on the font being used.

> The sequence 0x61 0xCC 0x80 (a followed by combining `) should
> produce something looking like =C3=A0. It does not, when the font is
> "monospace" on my system. In my case, "monospace" is actually
> Bitstream Vera Sans Mono. Also, when I say

> ./paps --font "Courier New 20" testpaps >testpaps.ps

> the accents are wrongly placed.

> However, when I say

> ./paps --font "freemono 20" testpaps >testpaps.ps

> the accents are placed OK! (Perhaps apart from U+323; but I do not
> know where it ought to be placed).

> This suggests that several fonts have bugs (fonts are basically
> "interpreted programs" nowadays, so they can have bugs!):

> -- either both Bitstream Vera Sans Mono and Courier New have bugs;
> -- or Freemono has bugs, and somehow Pango uses a work-around
>    which is itself a bug, causing it to fail with the other
>    fonts. I'll investigate this using fontforge; in the meantime,
>    could you confirm the different behaviour of the fonts?

I can confirm that it depends on the fonts and here are some=20
test results, not quite the same you had, also exposing additional=20
strange effects:

paps --font "monospace 20" testpaps > testpaps-monospace.ps
=3D> all 6 accents are placed wrong (to the right), extra space added

paps --font "freemono 20" testpaps > testpaps-freemono.ps
=3D> same output as "monospace" although not configured as such (see belo=
w)

paps --font "lucida console 20" testpaps > testpaps-lucida-console.ps
=3D> all 6 accents are placed correctly! However, white space (before the=
=20
   top and bottom borders of the STARG=CE=9B=CC=8ATE box) is expanded too=
 much,=20
   and line spacing is expanded
=3D> the strangest thing about this is that Lucida Console is my=20
   configured monospace font, yet they look different!

paps --font "monofur 20" testpaps > testpaps-monofur.ps
=3D> like with Lucida Console, all accents look fine;
   subsequent spacing is only slightly mangled,=20
   anyway the box doesn't align

paps --font "Courier 20" testpaps > testpaps-courier.ps
=3D> all 6 accents are placed wrong (to the right), extra space added
   (like with monospace) -
   this is probably what you describe for "Courier New" while here=20
   "Courier New" gives a different view:

paps --font "Courier New 20" testpaps > testpaps-courier-new.ps
=3D> only U+326 is placed wrong, all others are placed correctly, also=20
   white space after those other 5 is fine, there is, however, too much=20
   white space after some of the other accents (in the line before=20
   the STARG=CE=9B=CC=8ATE box, which is again completely mangled)

paps --font "Lucida Typewriter 20" testpaps > testpaps-lucidatypewriter.p=
s
paps --font "Vera Sans Mono 20" testpaps > testpaps-vera-sans-mono.ps
paps --font "Arial Mono 20" testpaps > testpaps-arialmono.ps
=3D> These three produce the same output - the fonts are apparently not=20
   found (with no message indicating this!) but the result has the=20
   interesting effect of placing the accents LEFT of the base characters.

paps --font "Code2000 20" testpaps > testpaps-code2000.ps
=3D> Like with the previous result, accent are placed LEFT

paps --font "Bitstream Vera Sans Mono 20" testpaps > testpaps-bitstream-v=
era-sans-mono.ps
=3D> all 6 accents wrong + extra space
   (with the full name, it's found)

paps --font "Andale Mono 20" testpaps > testpaps-andale-mono.ps
=3D> all 6 accents wrong + extra space

paps --font "Comic Sans MS 20" testpaps > testpaps-comic.ps
paps --font "Bodoni 20" testpaps > testpaps-bodoni.ps
=3D> all 6 accents placed well, subsequent space only very slightly affec=
ted

If you're interested to view my results, I'll place a zip file=20
(including my $HOME/.fonts.conf) to http://towo.net/towo/testpaps.zip=20
later this evening.

Kind regards,
Thomas

Re: [Paps-discuss] Combining characters not rendered properly

[Jan W. S. <jst...@pl...>]

Thomas Wolff schreef:
> Hello, for the record (and maybe in support of a pango bug report), 
> I'd like to share my further observations about misplaced combining 
> characters with various fonts:

> I can confirm that it depends on the fonts and here are some 
> test results, not quite the same you had, also exposing additional 
> strange effects: [..]

This is exceedingly weird. Especially the accents being displayed
*before* the base letter. I am now investigating this more
systematically (haven't finished, will let the list know as soon
as possible), but I haven't found a clear pattern yet. In
particular, there does not seem to be a clear correlation between
"treating combining accents correctly" and "having the combining
accents included in the font" as shown by Fontforge. Also, while
some fonts behave OK and others do not in pango-using programs, in
non-pango using programs like Openoffice (?) some fonts also
behave OK and others do not, but they are not the same fonts.

This begins to look like one of these logic puzzles of the type
"in one street live an Englishman, a German, a Frenchman, and a
Greek. The owner of the dog drinks whisky. The tea drinker lives
next to the Frenchman. The German has a cat. The Greek's
neighbour's house is red. [etc., etc.] Which pet is owned by the
man in the green house?"

Can our particular puzzle be solved, or do we need more data? The
object is, of course, to find out where the bugs are! I will
certainly study your fonts.conf, but it may take some time.

Regards, Jan

Re: [Paps-discuss] Combining characters not rendered properly

[Dov G. <dov...@gm...>]

Whether combined precomposed characters exist is not the only
indication of whether the font is "correct". A OpenType may describe
combinations either through  GSUB table that indicate what character
combinations should be rendered by one or more other glyphs, or
through a GPOS table that declare the relative offset of the
characters relative to one another. But of course we expect the
various layout engines (Qt, Gtk/Pango, and the IBM engine used in
Mozilla and OpenType if I'm not mistaken) to treat and render the
fonts the same way...

Regards,
Dov


On 12/7/06, Jan Willem Stumpel <jst...@pl...> wrote:
> Thomas Wolff schreef:
> > Hello, for the record (and maybe in support of a pango bug report),
> > I'd like to share my further observations about misplaced combining
> > characters with various fonts:
>
> > I can confirm that it depends on the fonts and here are some
> > test results, not quite the same you had, also exposing additional
> > strange effects: [..]
>
> This is exceedingly weird. Especially the accents being displayed
> *before* the base letter. I am now investigating this more
> systematically (haven't finished, will let the list know as soon
> as possible), but I haven't found a clear pattern yet. In
> particular, there does not seem to be a clear correlation between
> "treating combining accents correctly" and "having the combining
> accents included in the font" as shown by Fontforge. Also, while
> some fonts behave OK and others do not in pango-using programs, in
> non-pango using programs like Openoffice (?) some fonts also
> behave OK and others do not, but they are not the same fonts.
>
> This begins to look like one of these logic puzzles of the type
> "in one street live an Englishman, a German, a Frenchman, and a
> Greek. The owner of the dog drinks whisky. The tea drinker lives
> next to the Frenchman. The German has a cat. The Greek's
> neighbour's house is red. [etc., etc.] Which pet is owned by the
> man in the green house?"
>
> Can our particular puzzle be solved, or do we need more data? The
> object is, of course, to find out where the bugs are! I will
> certainly study your fonts.conf, but it may take some time.
>
> Regards, Jan
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Paps-discuss mailing list
> Pap...@li...
> https://lists.sourceforge.net/lists/listinfo/paps-discuss
>

Re: [Paps-discuss] Combining characters not rendered properly

[Jan W. S. <jst...@pl...>]

[copied to lin...@nl... from a discussion which started
at pap...@li...]

The "combining accents" problem seems really complicated.
Depending on the font and the rendering engine, combining accents
are sometimes displayed/printed correctly, or sometimes not.

Here are the results of some experiments, trying to display the
following combining (not pre-combined) accents:

=C3=A0 made by a + U+300
=C3=A1 made by a + U+301
=C3=A3 made by a + U+303
=E1=BA=A3 made by a + U+309
=E1=BA=A1 made by a + U+323
a with comma below made by a + U+326

Now sometimes this works (the accents are displayed above or below
the base letters) and sometimes it does not (the accents are
displayed after the base letters).

Legend:

OO : rendering of combining accents in Openoffice
GE : rendering of combining accents in Pango (gedit/paps)
TW : rendering in paps by Thomas Wolff's experiments
CA : "combining accents" present in font file
CL : "OT Glyph Class" (whatever that is) of combining accents
     (as shown by fontforge)
GS : font file has a GSUB table

-  : not at all
-- : font does not have code position for combining accents
     (Type1 font file)
+  : some accents (or "yes")
++ : all accents (especially including U+323, U+326)
A  : OT Glyph Class is "Automatic"
M  : OT Glyph Class is "Mark"


Font                       OO    GE   TW   CA   CL   GS
=3D=3D=3D=3D                       =3D=3D    =3D=3D   =3D=3D   =3D=
=3D   =3D=3D   =3D=3D

Andale Mono                 -     -         -    A    -

Arial                      ++    ++         +    A    +

AR PL KaitiM GB            ++    ++         -    A

Baekmuk Dotum              ++    ++         -    A

Bitstream Vera Serif       ++    ++         -    A

Bitstream Vera Sans Mono    -     -    -    -    A

Code2000                   ++    ++        ++    M

Comic Sans MS              ++    ++         -    A    -

Courier (10 pitch)          -    ++    -   --

Courier New                 -     -    +    +    A    +

FreeMono                    +     +        ++    M

FreeSans                   ++    ++        ++    M

Free Serif                 ++    ++        ++    M

Lucida Bright               +     +         -    A

Luxi Mono                   -     -         -    A

Luxi Sans                   +     +         -    A

Luxi Serif                  +     +         -    A

Times New Roman            ++    ++         +    A    +

Trebuchet MS               ++    ++         -    A    -

URW Bookman L              ++    ++        --

Verdana                     -     -         +    A    -


Well, I cannot make heads or tails of this. In general, the
behaviour of pango seems to be the same as that of Openoffice,
apart from the case Courier 10 pitch (which is a type 1 font). But
why the combining accents work in some fonts and not in others, I
have no clue. Are there bugs in the rendering engines? Or (more
likely) in the fonts? But what are these bugs exactly?

Thomas Wolff also showed paps results with "vera sans mono" which
shows the accents *before* the base letters. Which font is this
exactly? It it obviously not the same as "Bitstream vera Sans Mono".

I include (for members of the linux-utf-8 list) a test file made
by Thomas Wolff, containing "combining accents".

Regards, Jan

Re: [Paps-discuss] Combining characters not rendered properly

[Arne ) <ar...@li...>]

On Friday 08 December 2006 20:03, Jan Willem Stumpel wrote:
> The "combining accents" problem seems really complicated.
> Depending on the font and the rendering engine, combining accents
> are sometimes displayed/printed correctly, or sometimes not.

Correct.

> Here are the results of some experiments, trying to display the
> following combining (not pre-combined) accents:

[...]

> Font                       OO    GE   TW   CA   CL   GS
> =3D=3D=3D=3D                       =3D=3D    =3D=3D   =3D=3D   =3D=3D   =
=3D=3D   =3D=3D
>
> Andale Mono                 -     -         -    A    -
>
> Arial                      ++    ++         +    A    +
>
> AR PL KaitiM GB            ++    ++         -    A

I highly doubt, that the Arphic fonts can do combining accents. The=20
accents are not present in those fonts. Which simply means, that your=20
font rendering engine replaced the glyphs with some from other fonts.

> Well, I cannot make heads or tails of this. In general, the
> behaviour of pango seems to be the same as that of Openoffice,
> apart from the case Courier 10 pitch (which is a type 1 font). But
> why the combining accents work in some fonts and not in others, I
> have no clue. Are there bugs in the rendering engines? Or (more
> likely) in the fonts? But what are these bugs exactly?

Both.

The fonts need to contain
a) the base glyphs
b) the combining accents
c) "anchors" in the GPOS table to tell the rendering engine where=20
exactly to place the accents. In fontforge this is done easily. :)

The rendering engine needs to support the GPOS table and render the=20
glyphs according to it. AFAIK only pango can do this currently.

M$ has cheated in its Times New Roman font. They use the normal accents,=20
not the combining ones and give them a negative position. Some=20
rendering engines will then appear to render them "correctly", although=20
they actually don't. This might also be the answer for the next=20
question...

> Thomas Wolff also showed paps results with "vera sans mono" which
> shows the accents *before* the base letters. Which font is this
> exactly? It it obviously not the same as "Bitstream vera Sans Mono".
>

HTH
Arne
=2D-=20
Arne G=C3=B6tje (=E9=AB=98=E7=9B=9B=E8=8F=AF) <ar...@li...>
PGP/GnuPG key: 1024D/685D1E8C
=46ingerprint: 2056 F6B7 DEA8 B478 311F  1C34 6E9F D06E 685D 1E8C
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [Paps-discuss] Combining characters not rendered properly

[Jan W. S. <jst...@pl...>]

Arne G=C3=B6tje (=E9=AB=98=E7=9B=9B=E8=8F=AF) wrote:

> I highly doubt that the Arphic fonts can do combining accents.
> The accents are not present in those fonts. Which simply
> means, that your font rendering engine replaced the glyphs with
> some from other fonts.

Your explanation is enlightening. Thanks! I haven't yet found out
the exact mechanism by which some fonts which do not contain
combining accents, yet manage to display them. It must have
something to do with the order of preference in fontconfig's
config files.

>> But what are these bugs exactly?
>=20
> Both.
>=20
> The fonts need to contain a) the base glyphs b) the combining=20
> accents c) "anchors" in the GPOS table to tell the rendering=20
> engine where exactly to place the accents. In fontforge this is
> done easily. :)

I am sure I detect some irony here.. Anyway it is true that in all
cases when a), b), c) are fulfilled (but there are not many of
those), combining accents work (at least for *single* accents).
And they work (more or less by luck) also in some other cases.

I suppose for the Unicode "*multiple* combining accents" mechanism
to work, more complicated anchor classed ought to be defined
(allowing, e.g., "accent on top of accent"). But it seems no
actual font does this at the moment. In the case of the multiple
accents of Classical Greek, Openoffice seems to do OK. But I think
Openoffice uses another mechanism.

So the advice to users has to remain: use pre-combined accents
whenever possible. Don't count on the "combining accents"
mechanism to work. It won't, except from some lucky cases.

Thanks again for your explanation.

Regards, Jan

Re: [Paps-discuss] Combining characters not rendered properly

[<ar...@li...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jan Willem Stumpel wrote:
> Arne Götje (高盛華) wrote:
>> The fonts need to contain a) the base glyphs b) the combining 
>> accents c) "anchors" in the GPOS table to tell the rendering 
>> engine where exactly to place the accents. In fontforge this is
>> done easily. :)
> 
> I am sure I detect some irony here.. Anyway it is true that in all
> cases when a), b), c) are fulfilled (but there are not many of
> those), combining accents work (at least for *single* accents).
> And they work (more or less by luck) also in some other cases.
> 
> I suppose for the Unicode "*multiple* combining accents" mechanism
> to work, more complicated anchor classed ought to be defined
> (allowing, e.g., "accent on top of accent"). But it seems no
> actual font does this at the moment. In the case of the multiple
> accents of Classical Greek, Openoffice seems to do OK. But I think
> Openoffice uses another mechanism.
> 
> So the advice to users has to remain: use pre-combined accents
> whenever possible. Don't count on the "combining accents"
> mechanism to work. It won't, except from some lucky cases.

yes, it depends totally on the font to define the *position* of the
accents (and weather or not they can be stacked). But it depends on the
rendering engine to *interpret* the information the font gives about the
accents.

BTW: there was no irony in my statement. In Fontforge it is really easy
to define the "anchors". For all pre-composed Latin based combinations,
you can get it done with around 10 anchor classes, including the stacked
ones for Vietnamese.

The difficulty is to decide how the combinations should be displayed,
i.e. which "standard" to follow and which scripts to support.
You need to define the anchor points, both for the base glyphs and the
combining accents, separately and for each possible combination. This is
quite some work...

In my CJK-Unifont (http://www.cjkunifonts.info) project, I have used
"anchors" to display compositions, which are used in the Minnan language
(or better it's romanization) and which are not available as
pre-composed glyphs in Unicode.

Example: o <U+0301> <U+0358> -> ó͘

However, I made no attempts yet to implement this for the already
existing pre-composed glyphs, due to lack of time and low priority for
my project.

> 
> Thanks again for your explanation.

You are welcome. :)

Cheers
Arne
- --
Arne Götje (高盛華) <ar...@li...>
PGP/GnuPG key: 1024D/685D1E8C
Fingerprint: 2056 F6B7 DEA8 B478 311F  1C34 6E9F D06E 685D 1E8C
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFfLkQbp/QbmhdHowRAk84AJ9e/ej9k6v8Tjz1x73aYJLxVtDpZACfZAjX
yDwPgn1wlUC4Xvit84xGdrA=
=nWG8
-----END PGP SIGNATURE-----

[Paps-discuss] (no subject)

[Thomas W. <to...@to...>]

Arne Götje (高盛華) wrote:
> The fonts need to contain a) the base glyphs b) the combining 
> accents c) "anchors" in the GPOS table to tell the rendering 
> engine where exactly to place the accents. In fontforge this is
> done easily. :)

Jan Willem Stumpel wrote:
> ...
> So the advice to users has to remain: use pre-combined accents
> whenever possible. Don't count on the "combining accents"
> mechanism to work. It won't, except from some lucky cases.

Arne Götje (高盛華) wrote:
> yes, it depends totally on the font to define the *position* of the
> accents (and weather or not they can be stacked). But it depends on the
> rendering engine to *interpret* the information the font gives about the
> accents.
> 
> BTW: there was no irony in my statement. In Fontforge it is really easy
> to define the "anchors". For all pre-composed Latin based combinations,
> you can get it done with around 10 anchor classes, including the stacked
> ones for Vietnamese.
> 
> The difficulty is to decide how the combinations should be displayed,
> i.e. which "standard" to follow and which scripts to support.
> You need to define the anchor points, both for the base glyphs and the
> combining accents, separately and for each possible combination. This is
> quite some work...

Rich Felker wrote:
> Well there are two flip sides to this situation. On the one hand
> you're right, but on the other hand if no one tries to use the
> combining characters, applications and fonts will remain broken. :(
> Since I need to use scripts where combining is essential, I'm somewhat
> inclined to hilight the brokenness in apps (by using combining chars
> in more situations) in hopes that the authors will fix them...

Arne Götje (高盛華) wrote:
> In this case, *you* need to *define* which combinations you need and
> *how* they should be displayed. If they exist already as pre-composed
> glyphs in Unicode, then it's no problem to implement them... but if not,
> it's up to you to do the definition first and then either to modify
> existing fonts by yourself (e.g. by using fontforge), or ask the
> upstream author of the fonts to do it for you.
> 
> So, I suggest, you make a list of which combinations you need (and how
> they should look like) and ask the upstream maintainer of the font of
> your choice to implement them for you. ;)
>
> If the application of your choice then still cannot display the
> combinations correctly, prod the maintainers of that application to
> either use a rendering engine which can interpret the GPOS table of TTF
> fonts, or hack the support in by themselves, or (the best solution)
> forward the request to the upstream maintainer of the rendering engine
> they use...

Rich Felker wrote:
> *nod* these are all good suggestions.
Not in practice, I'm afraid.

Jan Willem Stumpel wrote:
> In the meantime I found that most fonts on my system do *not* have
> this table (including the Bitstream Vera fonts and the MS "core"
> fonts). It seems that including such a table is one of the things
> that we must "badger" upstream font developers about.

It was interesting that Arne pointed out that the problem could in theory 
be solved by fixing fonts but I don't think it's appropriate 
to shift the task of fixing the issue to fonts and their providers, 
because that approach is not reliable and not practical given the 
huge number of existing fonts and the small number of rendering engines.

Rather a font rendering engine should do the following:
* If anchor points are defined in the font, they should of course 
  be used.
* In absence of appropriate font information, the rendering engine 
  should determine the bounding boxes of base character and accent(s) 
  which should easily enable it to apply some kind of best-guess 
  approach to place the accent to a proper location.
  The engine should maintain a list of which accents are (normally) 
  placed on top or below, and whether the standard position is 
  with a gap or not (sometimes below), centered or attached to an edge.
  For italic or oblique fonts, the slanting angle should be considered 
  for best possible placement.
* If the font does not contain the appropriate accent and the 
  rendering engine provides the feature of font substitution, the 
  accent should be taken from a different font. For maximal 
  consistency of style, neither the base character nor a precomposed 
  character should be taken from that other font (unless the base 
  character itself is missing in the current font).

I actually implemented an automatic accent placement algorithm myself 
quite a while ago for fixing PostScript fonts; I wrote a PostScript 
program that would compose the Latin-1 set of combined characters from 
base letters and accents automatically and save an updated PostScript 
font. This way I could make use of the many free PostScript fonts that 
used to be made by amateurs, often just containing the Adobe standard 
encoding characters.

Kind regards,
Thomas

Re: [Paps-discuss] (no subject)

[Dov G. <dov...@gm...>]

QW5vdGhlciBzaW1pbGFyIHNvbHV0aW9uIHRoYXQgSSBzdGFydGVkIHdvcmtpbmcgb24gZm9yIEhl
YnJldyBmb250cyBpcwp0byBhcHBseSBzdWNoIGhldXJpc3RpY3MgYnV0IHRoZW4gc2F2ZSB0aGUg
cmVzdWx0IGluIHRoZSByZWxldmFudApvcGVudHlwZSB0YWJsZXMuCgpOb3RlIHRoYXQgUGFuZ28g
ZG9lcyBzdXBwb3J0IHRoZSBpbXBsZW1lbnRhdGlvbiBvZiBzdWNoIGhldXJpc3RpY3MgaW4KdGhl
IHZhcmlvdXMgbW9kdWxlcy4gSSB3cm90ZSBzdWNoIGEgbW9kdWxlIGZvciBIZWJyZXcgd2hpY2gg
aXMgdXNlZCBpbgp0aGUgYWJzZW5zZSBvZiB0aGUgT3BlblR5cGUgdGFibGVzLgoKUmVnYXJkcywK
RG92CgpPbiAxMi8xMi8wNiwgVGhvbWFzIFdvbGZmIDx0b3dvQHRvd28ubmV0PiB3cm90ZToKPiBB
cm5lIEfDtnRqZSAo6auY55ub6I+vKSB3cm90ZToKPiA+IFRoZSBmb250cyBuZWVkIHRvIGNvbnRh
aW4gYSkgdGhlIGJhc2UgZ2x5cGhzIGIpIHRoZSBjb21iaW5pbmcKPiA+IGFjY2VudHMgYykgImFu
Y2hvcnMiIGluIHRoZSBHUE9TIHRhYmxlIHRvIHRlbGwgdGhlIHJlbmRlcmluZwo+ID4gZW5naW5l
IHdoZXJlIGV4YWN0bHkgdG8gcGxhY2UgdGhlIGFjY2VudHMuIEluIGZvbnRmb3JnZSB0aGlzIGlz
Cj4gPiBkb25lIGVhc2lseS4gOikKPgo+IEphbiBXaWxsZW0gU3R1bXBlbCB3cm90ZToKPiA+IC4u
Lgo+ID4gU28gdGhlIGFkdmljZSB0byB1c2VycyBoYXMgdG8gcmVtYWluOiB1c2UgcHJlLWNvbWJp
bmVkIGFjY2VudHMKPiA+IHdoZW5ldmVyIHBvc3NpYmxlLiBEb24ndCBjb3VudCBvbiB0aGUgImNv
bWJpbmluZyBhY2NlbnRzIgo+ID4gbWVjaGFuaXNtIHRvIHdvcmsuIEl0IHdvbid0LCBleGNlcHQg
ZnJvbSBzb21lIGx1Y2t5IGNhc2VzLgo+Cj4gQXJuZSBHw7Z0amUgKOmrmOebm+iPrykgd3JvdGU6
Cj4gPiB5ZXMsIGl0IGRlcGVuZHMgdG90YWxseSBvbiB0aGUgZm9udCB0byBkZWZpbmUgdGhlICpw
b3NpdGlvbiogb2YgdGhlCj4gPiBhY2NlbnRzIChhbmQgd2VhdGhlciBvciBub3QgdGhleSBjYW4g
YmUgc3RhY2tlZCkuIEJ1dCBpdCBkZXBlbmRzIG9uIHRoZQo+ID4gcmVuZGVyaW5nIGVuZ2luZSB0
byAqaW50ZXJwcmV0KiB0aGUgaW5mb3JtYXRpb24gdGhlIGZvbnQgZ2l2ZXMgYWJvdXQgdGhlCj4g
PiBhY2NlbnRzLgo+ID4KPiA+IEJUVzogdGhlcmUgd2FzIG5vIGlyb255IGluIG15IHN0YXRlbWVu
dC4gSW4gRm9udGZvcmdlIGl0IGlzIHJlYWxseSBlYXN5Cj4gPiB0byBkZWZpbmUgdGhlICJhbmNo
b3JzIi4gRm9yIGFsbCBwcmUtY29tcG9zZWQgTGF0aW4gYmFzZWQgY29tYmluYXRpb25zLAo+ID4g
eW91IGNhbiBnZXQgaXQgZG9uZSB3aXRoIGFyb3VuZCAxMCBhbmNob3IgY2xhc3NlcywgaW5jbHVk
aW5nIHRoZSBzdGFja2VkCj4gPiBvbmVzIGZvciBWaWV0bmFtZXNlLgo+ID4KPiA+IFRoZSBkaWZm
aWN1bHR5IGlzIHRvIGRlY2lkZSBob3cgdGhlIGNvbWJpbmF0aW9ucyBzaG91bGQgYmUgZGlzcGxh
eWVkLAo+ID4gaS5lLiB3aGljaCAic3RhbmRhcmQiIHRvIGZvbGxvdyBhbmQgd2hpY2ggc2NyaXB0
cyB0byBzdXBwb3J0Lgo+ID4gWW91IG5lZWQgdG8gZGVmaW5lIHRoZSBhbmNob3IgcG9pbnRzLCBi
b3RoIGZvciB0aGUgYmFzZSBnbHlwaHMgYW5kIHRoZQo+ID4gY29tYmluaW5nIGFjY2VudHMsIHNl
cGFyYXRlbHkgYW5kIGZvciBlYWNoIHBvc3NpYmxlIGNvbWJpbmF0aW9uLiBUaGlzIGlzCj4gPiBx
dWl0ZSBzb21lIHdvcmsuLi4KPgo+IFJpY2ggRmVsa2VyIHdyb3RlOgo+ID4gV2VsbCB0aGVyZSBh
cmUgdHdvIGZsaXAgc2lkZXMgdG8gdGhpcyBzaXR1YXRpb24uIE9uIHRoZSBvbmUgaGFuZAo+ID4g
eW91J3JlIHJpZ2h0LCBidXQgb24gdGhlIG90aGVyIGhhbmQgaWYgbm8gb25lIHRyaWVzIHRvIHVz
ZSB0aGUKPiA+IGNvbWJpbmluZyBjaGFyYWN0ZXJzLCBhcHBsaWNhdGlvbnMgYW5kIGZvbnRzIHdp
bGwgcmVtYWluIGJyb2tlbi4gOigKPiA+IFNpbmNlIEkgbmVlZCB0byB1c2Ugc2NyaXB0cyB3aGVy
ZSBjb21iaW5pbmcgaXMgZXNzZW50aWFsLCBJJ20gc29tZXdoYXQKPiA+IGluY2xpbmVkIHRvIGhp
bGlnaHQgdGhlIGJyb2tlbm5lc3MgaW4gYXBwcyAoYnkgdXNpbmcgY29tYmluaW5nIGNoYXJzCj4g
PiBpbiBtb3JlIHNpdHVhdGlvbnMpIGluIGhvcGVzIHRoYXQgdGhlIGF1dGhvcnMgd2lsbCBmaXgg
dGhlbS4uLgo+Cj4gQXJuZSBHw7Z0amUgKOmrmOebm+iPrykgd3JvdGU6Cj4gPiBJbiB0aGlzIGNh
c2UsICp5b3UqIG5lZWQgdG8gKmRlZmluZSogd2hpY2ggY29tYmluYXRpb25zIHlvdSBuZWVkIGFu
ZAo+ID4gKmhvdyogdGhleSBzaG91bGQgYmUgZGlzcGxheWVkLiBJZiB0aGV5IGV4aXN0IGFscmVh
ZHkgYXMgcHJlLWNvbXBvc2VkCj4gPiBnbHlwaHMgaW4gVW5pY29kZSwgdGhlbiBpdCdzIG5vIHBy
b2JsZW0gdG8gaW1wbGVtZW50IHRoZW0uLi4gYnV0IGlmIG5vdCwKPiA+IGl0J3MgdXAgdG8geW91
IHRvIGRvIHRoZSBkZWZpbml0aW9uIGZpcnN0IGFuZCB0aGVuIGVpdGhlciB0byBtb2RpZnkKPiA+
IGV4aXN0aW5nIGZvbnRzIGJ5IHlvdXJzZWxmIChlLmcuIGJ5IHVzaW5nIGZvbnRmb3JnZSksIG9y
IGFzayB0aGUKPiA+IHVwc3RyZWFtIGF1dGhvciBvZiB0aGUgZm9udHMgdG8gZG8gaXQgZm9yIHlv
dS4KPiA+Cj4gPiBTbywgSSBzdWdnZXN0LCB5b3UgbWFrZSBhIGxpc3Qgb2Ygd2hpY2ggY29tYmlu
YXRpb25zIHlvdSBuZWVkIChhbmQgaG93Cj4gPiB0aGV5IHNob3VsZCBsb29rIGxpa2UpIGFuZCBh
c2sgdGhlIHVwc3RyZWFtIG1haW50YWluZXIgb2YgdGhlIGZvbnQgb2YKPiA+IHlvdXIgY2hvaWNl
IHRvIGltcGxlbWVudCB0aGVtIGZvciB5b3UuIDspCj4gPgo+ID4gSWYgdGhlIGFwcGxpY2F0aW9u
IG9mIHlvdXIgY2hvaWNlIHRoZW4gc3RpbGwgY2Fubm90IGRpc3BsYXkgdGhlCj4gPiBjb21iaW5h
dGlvbnMgY29ycmVjdGx5LCBwcm9kIHRoZSBtYWludGFpbmVycyBvZiB0aGF0IGFwcGxpY2F0aW9u
IHRvCj4gPiBlaXRoZXIgdXNlIGEgcmVuZGVyaW5nIGVuZ2luZSB3aGljaCBjYW4gaW50ZXJwcmV0
IHRoZSBHUE9TIHRhYmxlIG9mIFRURgo+ID4gZm9udHMsIG9yIGhhY2sgdGhlIHN1cHBvcnQgaW4g
YnkgdGhlbXNlbHZlcywgb3IgKHRoZSBiZXN0IHNvbHV0aW9uKQo+ID4gZm9yd2FyZCB0aGUgcmVx
dWVzdCB0byB0aGUgdXBzdHJlYW0gbWFpbnRhaW5lciBvZiB0aGUgcmVuZGVyaW5nIGVuZ2luZQo+
ID4gdGhleSB1c2UuLi4KPgo+IFJpY2ggRmVsa2VyIHdyb3RlOgo+ID4gKm5vZCogdGhlc2UgYXJl
IGFsbCBnb29kIHN1Z2dlc3Rpb25zLgo+IE5vdCBpbiBwcmFjdGljZSwgSSdtIGFmcmFpZC4KPgo+
IEphbiBXaWxsZW0gU3R1bXBlbCB3cm90ZToKPiA+IEluIHRoZSBtZWFudGltZSBJIGZvdW5kIHRo
YXQgbW9zdCBmb250cyBvbiBteSBzeXN0ZW0gZG8gKm5vdCogaGF2ZQo+ID4gdGhpcyB0YWJsZSAo
aW5jbHVkaW5nIHRoZSBCaXRzdHJlYW0gVmVyYSBmb250cyBhbmQgdGhlIE1TICJjb3JlIgo+ID4g
Zm9udHMpLiBJdCBzZWVtcyB0aGF0IGluY2x1ZGluZyBzdWNoIGEgdGFibGUgaXMgb25lIG9mIHRo
ZSB0aGluZ3MKPiA+IHRoYXQgd2UgbXVzdCAiYmFkZ2VyIiB1cHN0cmVhbSBmb250IGRldmVsb3Bl
cnMgYWJvdXQuCj4KPiBJdCB3YXMgaW50ZXJlc3RpbmcgdGhhdCBBcm5lIHBvaW50ZWQgb3V0IHRo
YXQgdGhlIHByb2JsZW0gY291bGQgaW4gdGhlb3J5Cj4gYmUgc29sdmVkIGJ5IGZpeGluZyBmb250
cyBidXQgSSBkb24ndCB0aGluayBpdCdzIGFwcHJvcHJpYXRlCj4gdG8gc2hpZnQgdGhlIHRhc2sg
b2YgZml4aW5nIHRoZSBpc3N1ZSB0byBmb250cyBhbmQgdGhlaXIgcHJvdmlkZXJzLAo+IGJlY2F1
c2UgdGhhdCBhcHByb2FjaCBpcyBub3QgcmVsaWFibGUgYW5kIG5vdCBwcmFjdGljYWwgZ2l2ZW4g
dGhlCj4gaHVnZSBudW1iZXIgb2YgZXhpc3RpbmcgZm9udHMgYW5kIHRoZSBzbWFsbCBudW1iZXIg
b2YgcmVuZGVyaW5nIGVuZ2luZXMuCj4KPiBSYXRoZXIgYSBmb250IHJlbmRlcmluZyBlbmdpbmUg
c2hvdWxkIGRvIHRoZSBmb2xsb3dpbmc6Cj4gKiBJZiBhbmNob3IgcG9pbnRzIGFyZSBkZWZpbmVk
IGluIHRoZSBmb250LCB0aGV5IHNob3VsZCBvZiBjb3Vyc2UKPiAgIGJlIHVzZWQuCj4gKiBJbiBh
YnNlbmNlIG9mIGFwcHJvcHJpYXRlIGZvbnQgaW5mb3JtYXRpb24sIHRoZSByZW5kZXJpbmcgZW5n
aW5lCj4gICBzaG91bGQgZGV0ZXJtaW5lIHRoZSBib3VuZGluZyBib3hlcyBvZiBiYXNlIGNoYXJh
Y3RlciBhbmQgYWNjZW50KHMpCj4gICB3aGljaCBzaG91bGQgZWFzaWx5IGVuYWJsZSBpdCB0byBh
cHBseSBzb21lIGtpbmQgb2YgYmVzdC1ndWVzcwo+ICAgYXBwcm9hY2ggdG8gcGxhY2UgdGhlIGFj
Y2VudCB0byBhIHByb3BlciBsb2NhdGlvbi4KPiAgIFRoZSBlbmdpbmUgc2hvdWxkIG1haW50YWlu
IGEgbGlzdCBvZiB3aGljaCBhY2NlbnRzIGFyZSAobm9ybWFsbHkpCj4gICBwbGFjZWQgb24gdG9w
IG9yIGJlbG93LCBhbmQgd2hldGhlciB0aGUgc3RhbmRhcmQgcG9zaXRpb24gaXMKPiAgIHdpdGgg
YSBnYXAgb3Igbm90IChzb21ldGltZXMgYmVsb3cpLCBjZW50ZXJlZCBvciBhdHRhY2hlZCB0byBh
biBlZGdlLgo+ICAgRm9yIGl0YWxpYyBvciBvYmxpcXVlIGZvbnRzLCB0aGUgc2xhbnRpbmcgYW5n
bGUgc2hvdWxkIGJlIGNvbnNpZGVyZWQKPiAgIGZvciBiZXN0IHBvc3NpYmxlIHBsYWNlbWVudC4K
PiAqIElmIHRoZSBmb250IGRvZXMgbm90IGNvbnRhaW4gdGhlIGFwcHJvcHJpYXRlIGFjY2VudCBh
bmQgdGhlCj4gICByZW5kZXJpbmcgZW5naW5lIHByb3ZpZGVzIHRoZSBmZWF0dXJlIG9mIGZvbnQg
c3Vic3RpdHV0aW9uLCB0aGUKPiAgIGFjY2VudCBzaG91bGQgYmUgdGFrZW4gZnJvbSBhIGRpZmZl
cmVudCBmb250LiBGb3IgbWF4aW1hbAo+ICAgY29uc2lzdGVuY3kgb2Ygc3R5bGUsIG5laXRoZXIg
dGhlIGJhc2UgY2hhcmFjdGVyIG5vciBhIHByZWNvbXBvc2VkCj4gICBjaGFyYWN0ZXIgc2hvdWxk
IGJlIHRha2VuIGZyb20gdGhhdCBvdGhlciBmb250ICh1bmxlc3MgdGhlIGJhc2UKPiAgIGNoYXJh
Y3RlciBpdHNlbGYgaXMgbWlzc2luZyBpbiB0aGUgY3VycmVudCBmb250KS4KPgo+IEkgYWN0dWFs
bHkgaW1wbGVtZW50ZWQgYW4gYXV0b21hdGljIGFjY2VudCBwbGFjZW1lbnQgYWxnb3JpdGhtIG15
c2VsZgo+IHF1aXRlIGEgd2hpbGUgYWdvIGZvciBmaXhpbmcgUG9zdFNjcmlwdCBmb250czsgSSB3
cm90ZSBhIFBvc3RTY3JpcHQKPiBwcm9ncmFtIHRoYXQgd291bGQgY29tcG9zZSB0aGUgTGF0aW4t
MSBzZXQgb2YgY29tYmluZWQgY2hhcmFjdGVycyBmcm9tCj4gYmFzZSBsZXR0ZXJzIGFuZCBhY2Nl
bnRzIGF1dG9tYXRpY2FsbHkgYW5kIHNhdmUgYW4gdXBkYXRlZCBQb3N0U2NyaXB0Cj4gZm9udC4g
VGhpcyB3YXkgSSBjb3VsZCBtYWtlIHVzZSBvZiB0aGUgbWFueSBmcmVlIFBvc3RTY3JpcHQgZm9u
dHMgdGhhdAo+IHVzZWQgdG8gYmUgbWFkZSBieSBhbWF0ZXVycywgb2Z0ZW4ganVzdCBjb250YWlu
aW5nIHRoZSBBZG9iZSBzdGFuZGFyZAo+IGVuY29kaW5nIGNoYXJhY3RlcnMuCj4KPiBLaW5kIHJl
Z2FyZHMsCj4gVGhvbWFzCj4KPgo+Cj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQo+IFRha2UgU3VydmV5cy4g
RWFybiBDYXNoLiBJbmZsdWVuY2UgdGhlIEZ1dHVyZSBvZiBJVAo+IEpvaW4gU291cmNlRm9yZ2Uu
bmV0J3MgVGVjaHNheSBwYW5lbCBhbmQgeW91J2xsIGdldCB0aGUgY2hhbmNlIHRvIHNoYXJlIHlv
dXIKPiBvcGluaW9ucyBvbiBJVCAmIGJ1c2luZXNzIHRvcGljcyB0aHJvdWdoIGJyaWVmIHN1cnZl
eXMgLSBhbmQgZWFybiBjYXNoCj4gaHR0cDovL3d3dy50ZWNoc2F5LmNvbS9kZWZhdWx0LnBocD9w
YWdlPWpvaW4ucGhwJnA9c291cmNlZm9yZ2UmQ0lEPURFVkRFVgo+Cj4gX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KPiBQYXBzLWRpc2N1c3MgbWFpbGluZyBs
aXN0Cj4gUGFwcy1kaXNjdXNzQGxpc3RzLnNvdXJjZWZvcmdlLm5ldAo+IGh0dHBzOi8vbGlzdHMu
c291cmNlZm9yZ2UubmV0L2xpc3RzL2xpc3RpbmZvL3BhcHMtZGlzY3Vzcwo+Cj4KPgo=

Re: [Paps-discuss] Combining characters not rendered properly

[Jan W. S. <jst...@pl...>]

Arne G=C3=B6tje (=E9=AB=98=E7=9B=9B=E8=8F=AF) wrote:
> [..] yes, it depends totally on the font to define the
> *position* of the accents (and weather or not they can be
> stacked). But it depends on the rendering engine to *interpret*
> the information the font gives about the accents.
>=20
> BTW: there was no irony in my statement. In Fontforge it is
> really easy to define the "anchors". For all pre-composed Latin
> based combinations, you can get it done with around 10 anchor
> classes, including the stacked ones for Vietnamese.

Is it also easy to create a GPOS table for a font which does not
have one? My experience with Fontforge is very limited!

In the meantime I found that most fonts on my system do *not* have
this table (including the Bitstream Vera fonts and the MS "core"
fonts). It seems that including such a table is one of the things
that we must "badger" upstream font developers about.

> [..] Example: o <U+0301> <U+0358> -> o=CC=81=CD=98

This example displays OK on my system with your uming.ttf font
(naturally), but also (by "luck") with, for instance, Bitstream
Vera Serif (which does not have the combining accent characters,
nor a GPOS table). I suppose the rendering engine (pango) borrows
the accents from another font (yours, probably). But how can it
know where to place them? The base characters in Bitstream Vera
Serif do not have anchors.

Regards, Jan

Re: [Paps-discuss] Combining characters not rendered properly

[<ar...@li...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jan Willem Stumpel wrote:
> Arne Götje (高盛華) wrote:
>> [..] yes, it depends totally on the font to define the
>> *position* of the accents (and weather or not they can be
>> stacked). But it depends on the rendering engine to *interpret*
>> the information the font gives about the accents.
>>
>> BTW: there was no irony in my statement. In Fontforge it is
>> really easy to define the "anchors". For all pre-composed Latin
>> based combinations, you can get it done with around 10 anchor
>> classes, including the stacked ones for Vietnamese.
> 
> Is it also easy to create a GPOS table for a font which does not
> have one? My experience with Fontforge is very limited!

As soon as any GPOS feature ("anchors" is one if the features) is
present, the table is created automatically.

> In the meantime I found that most fonts on my system do *not* have
> this table (including the Bitstream Vera fonts and the MS "core"
> fonts). It seems that including such a table is one of the things
> that we must "badger" upstream font developers about.

Exactly.

>> [..] Example: o <U+0301> <U+0358> -> ó͘
> 
> This example displays OK on my system with your uming.ttf font
> (naturally), but also (by "luck") with, for instance, Bitstream
> Vera Serif (which does not have the combining accent characters,
> nor a GPOS table). I suppose the rendering engine (pango) borrows
> the accents from another font (yours, probably). But how can it
> know where to place them? The base characters in Bitstream Vera
> Serif do not have anchors.

This also puzzles me. but the glyphs do not come from my font, they look
different... I'd have to take a closer look on the fonts if I find some
time... I assume that if there are no GPOS information present, pango
and other rendering engines classify the combining accents by unicode
coderange and just center them over the preceding (latin) character...
(maybe someone of the pango guys can answer this...?)

Cheers
Arne
- --
Arne Götje (高盛華) <ar...@li...>
PGP/GnuPG key: 1024D/685D1E8C
Fingerprint: 2056 F6B7 DEA8 B478 311F  1C34 6E9F D06E 685D 1E8C
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFfS5Ybp/QbmhdHowRApPWAJsESNwk1M29ngc8op6Uf1RRJVnDBwCgmEOR
grh/GERXv0DrzsqXj+ousV8=
=m8wc
-----END PGP SIGNATURE-----