#3 troubleshooting fonts

closed
None
5
2008-02-21
2007-12-19
Anonymous
No

I'm encountering MathML font issues when I convert FO files to PDF.

I'm running FOP 0.94 with the JEuclid FOP plugin 2.0.3, on Debian Etch. I have the ttf-dejavu package version 2.15-1 installed.

I've also been using ttf-opensymbol version 2.0.4.dfsg.2-7etch1, and just upgraded it to version 2.0.4.dfsg.2-7etch4 - no change in the issue decribed here.

Attached is a zip with sample files:

broken1.fo is a combination and reduction of the example.mml and mathml.fo files that ship with the JEuclid FOP plugin.

broken2.fo contains another math equation that I have been working with.

No error messages are generated (as far as I can tell) when either of these is run through FOP. However the resulting PDFs (also included) contain "unknown character" boxes.

I can't figure out where this error is occurring, or how to correct it. Any help is appreciated.

Peter Williams
broadway at city-net dot com

Discussion

1 2 > >> (Page 1 of 2)
  • Nobody/Anonymous

    sample FO and PDF files

     
  • Max Berger

    Max Berger - 2007-12-20

    Logged In: YES
    user_id=99210
    Originator: NO

    Peter,

    the first one (broken1) is easy: the characters used (underbrace and overbrace, ︷ and ︸ ) are not available in dejavu or opensymbol. Here you can try to install "Arial Unicode MS" from a windows / office installation (ONLY if you have the license!), or install Stic Fonts: www.stixfonts.org, which should be supported in 3.0.2

    The second one is a little trickier, since the character used (an arrow, ⃗) is included in dejavu since version 2.8. There may be an additional font-loading issue here.

    What I would like you to try is to install stix fonts, and see if that solves the problem. If not, we'll have to dig a little deeper.

    Max

     
  • Nobody/Anonymous

    Logged In: NO

    I've downloaded the Stix fonts... but beyond unpacking and placing the .otf files in ~/.fonts (which doesn't produce a change) I'm not sure how I'm supposed to install them.

    Do I need to convert them over to .ttf? Place them somewhere else? Call them directly from JEuclid or FOP somehow?

    Does JEuclid work with Defoma... is that something I should check into?

    I'll also point out that my browsers (Evolution, IceWeasel) DO display these MathML characters correctly, so some font on my system must already have them, it's just not getting picked up by JEuclid/FOP.

    I obviously need to spend some time learning how fonts work here, as this is something I will want to have running smoothly (and if the issues are specific to individual characters, it may come up again... these were just my initial tests). I'll keep working at it on my end.

    Peter

     
  • Max Berger

    Max Berger - 2007-12-22

    Logged In: YES
    user_id=99210
    Originator: NO

    Depending on whether you use gnome or kde there should be a font selection tool somewhere (e.g. gnome-control-center or kde-control-center). See if the "STIXGeneral" font shows up there. I am not sure about the current status of OTF support in Debian. If you need the .ttf version, one is available at:

    http://jeuclid.svn.sourceforge.net/svnroot/jeuclid/trunk/stix-fonts/src/main/resources/fonts/

    at this time fop fonts and jeuclid fonts *should* be separately. Jeuclid should pick up the fonts from your system - so on linux make sure X is open and you don't use headless=true

    hope this helps a little

    Max

     
  • Nobody/Anonymous

    Logged In: NO

    Okay, I'm back. Been trying to work out the STIX fonts, and I'm still stuck.

    I have the libfreetype6 package installed, which says it does support OTF fonts. I am running Gnome (2.14.3). The fonts folder is accessed in Nautilus as "fonts:///" (and the things I do there manually, show up in ~/.fonts) ...

    Adding the STIX OTF fonts, they do show up. However there is no change in JEuclid's output.

    Adding the STIXGeneral.ttf font from that link, it only really shows up if the OTF fonts aren't there. And even then, trying to open it with Gnome's font viewer, it says it can't find the file. The OTFs come up correctly in the font viewer.

    Pulling up the list of glyphs on the STIX website, I get a lot near the end ("Private Use Area; 0E0000" and on) that don't display properly, regardless of how the fonts are installed.

    However, when I set up a simple .html page with the ︷ and ︸ characters that are at issue... they display in IceWeasel. They're getting picked up from SOME font... just not STIX (these display even when it's removed).

    It is indeed helpful to know that JEuclid should be accessing the fonts from X, and separately from FOP. Just to be extra sure... is this still the case when I'm running the JEuclid FOP plugin (and therefore making the call to FOP)?

    And, a question more out of curiosity, is there a reason that JEuclid doesn't use the Computer Modern TrueType TeX fonts and the Mathematica 4.1 Fonts, which are used for MathML display in browsers?

    That's where things stand at the moment. Let me know if you have further ideas, or any other info I can offer.

    Peter

     
  • Nobody/Anonymous

    Logged In: NO

    Quick update... I discovered I wasn't installing the fonts quite right. I have now reinstalled the OTF set of STIX fonts, and the list of glyphs on their site DOES display correctly all the way through.

    Still no change in the JEuclid FOP output, though.

    Peter

     
  • Max Berger

    Max Berger - 2008-01-09

    Logged In: YES
    user_id=99210
    Originator: NO

    Peter,

    new idea: I've changed the font selection in jeuclid to use ALL fonts available on the local system for fallback. Unfortunately this is in trunk, so could you please check with a snapshot from:

    http://www.dps.uibk.ac.at/~berger/j/

    and see if this helps? (you'll need to put these 3 files into your fop/lib directory and remove all old jeuclid / xmlgraphics libs there). If so, I might backport this to trunk.

    Max

     
  • Nobody/Anonymous

    Logged In: NO

    Unfortunately, that didn't do the trick.

    broken2.fo still comes out the same. broken1.fo now has wider rectangles in place of the squares for its unrecognized characters. But that's still not the braces we're looking for.

    I'm continuing to poke at things here, to see if I can get any other info.

    Peter

     
  • Max Berger

    Max Berger - 2008-01-15

    Logged In: YES
    user_id=99210
    Originator: NO

    Peter,

    ok, next question:

    Do you run fop "headless" ? do you set java.awt.headless to true, or do you run it remote, or in any other environment where the DISPLAY variable may be unset?

    (both files rendere fine here)

    Also, since the new jeuclid (3.1) can load fonts from jar files: Pleas try to add the stix-fonts jar and/or the dejavu-fonts jar to your classpath (add them to the fop lib/ directory). I've copied them to the same place as the other jars. See if that helps

    Max

     
  • Nobody/Anonymous

    Logged In: NO

    I can report some progress, but it's odd...

    I can get broken2.fo to process now (with the arrow symbol). Poking around a bit, it looks like changing to the jeuclid 3.1 files... just the three of them, and not the font jar files... is what made this possible. I have no idea why it didn't work the other day when I tried that (I definitely tested it, as I mention below how the file had no change). But it's working now, probably drawing that symbol from some other font.

    Running with jeuclid 3.1 and the font jars in place does not improve the output of broken1.fo.

    The "headless" setting has me confused. I haven't done anything proactive to tell anything to run in headless mode, nor have I told anything to NOT run in headless mode. And in all of my grepping and googling, I can't figure out /where/ this is supposed to be set?

    Thanks for bearing with me on all of this.
    Peter

     
  • Raphael Jolly

    Raphael Jolly - 2008-01-18

    Logged In: YES
    user_id=692744
    Originator: NO

    Hi,

    There seem to be an issue with FOP. I have executed carefully the instructions on http://xmlgraphics.apache.org/fop/0.94/fonts.html , and it turns out to be able to solve my (at least) problem.

    I can't get square roots to show up in my pdfs. I have a minimal .fo with this code:

    <fo:instream-foreign-object>
    <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msqrt><mrow><mi>x</mi></mrow></msqrt></mrow></math>
    </fo:instream-foreign-object>

    , and when I run the following command:

    fop-0.94/fop minimal.fo minimal.pdf

    , I get the error:

    Jan 19, 2008 12:06:20 AM org.apache.fop.fonts.FontInfo notifyFontReplacement
    WARNING: Font 'Lucida Bright,normal,400' not found. Substituting with 'any,normal,400'.

    , and a sharp (#) in place of the symbol. I am on Debian etch too.

    When running with a custom conf:

    fop-0.94/fop -c fop.xconf minimal.fo minimal.pdf

    , I am able to get rid of the error, and the square root displays correctly. My addition is:

    <font embed-url="LucidaBrightRegular.ttf">
    <font-triplet name="Lucida Bright" style="normal" weight="normal"/>
    </font>

    in the "fonts" part of fop.xconf. I have copied /usr/lib/j2se/1.6/jre/lib/fonts/LucidaBrightRegular.ttf in the same directory as fop.xconf.

    Unfortunately I am unable to automate this process, because the relevant part of FOP is quite buggy. So I can't get my software (jscl-meditor) to benefit from square roots as yet :-( Aside from that, It works really great with jeuclid : thanks and congratulations.

    Raphael

     
  • Raphael Jolly

    Raphael Jolly - 2008-01-19

    Logged In: YES
    user_id=692744
    Originator: NO

    Update:

    (Perhaps I should have used another Support Request, sorry for that)

    Using the jeuclid 3.1 snapshot jars solves my square root problem. I can however reproduce Peter's issue : broken2.fo works fine, broken1.fo has wider unrecognized rectangles, the font jars have no effect.

    Max, if it works on your system, it has to be related to Debian etch rather than to fop or jeuclid. What do you run as a system ?

    Raphael

     
  • Nobody/Anonymous

    Logged In: NO

    Raphael-
    I for one don't mind you chiming in here at all. Which XWindows are you running? If it's Gnome, then I wonder if /that/ is the issue (not Debian itself).

    Peter

     
  • Max Berger

    Max Berger - 2008-01-22

    Logged In: YES
    user_id=99210
    Originator: NO

    Hi,

    I am using redhat, debian, and os x, depeding on what tests are run.

    The broken1.fo uses the characters fe37 and fe38 for underbrace and overbrace. These are not available in DejaVu, and Stix fonts does not have it either. I'll go look which font actually has those characters. I am glad the font-search works, it is backported to jeuclid stable and will be available with the 3.0.3 release.

    Max

     
  • Max Berger

    Max Berger - 2008-01-24
    • status: open --> pending
     
  • Max Berger

    Max Berger - 2008-01-24

    Logged In: YES
    user_id=99210
    Originator: NO

    Hi,

    turns out these characters are part of some taiwanese fonts, on my system:

    AR PL KaitiM Big5
    AR PL KaitiM GB
    AR PL Mingti2L Big5
    AR PL SungtiL GB
    ZYSong18030

    These are available in debian as
    ttf-arphic-bkai00mp
    ttf-arphic-bsmi00lp
    ttf-arphic-gbsn00lp
    ttf-arphic-gkai00mp

    (one of these fonts should be sufficient)

    To use them you should not need to add them to fop, as JEuclid should pick them up from the system (with the patched version).

    I've created a 3.0.3 snapshot, available at:

    http://www.dps.uibk.ac.at/~berger/j/

    Please test that as well.

    Max

     
  • Nobody/Anonymous

    • status: pending --> open
     
  • Nobody/Anonymous

    Logged In: NO

    Success!

    At first, running with all four of the taiwanese fonts in place, it produced a change but not a complete fix (the brace characters didn't show up at all - where they used to show up with the "unknown character" boxes).

    Reducing to one of the fonts (ttf-arphic-bkai00mp), or perhaps it was just another round of updating system font settings, however gives us the over- and under-brace characters at last!

    This works with the new 3.0.3 files, as does the arrow character from broken2.fo.

    Odd that this has to resort to taiwanese fonts. I'm not sure how to check this, but shouldn't the characters be available in something like mtextra.ttf? (Available from http://www.mozilla.org/projects/mathml/fonts/, in the Windows installation only - I copied it out of that onto my linux machine, in ~/.fonts, where it didn't seem to have an effect. But then, nothing in ~/.fonts has seemed to have an effect in this process).

    Anyway, no need to quibble. That's just an idle question... for now I'm very pleased that these problems are finally solved. Thanks so much for all your help!

    Best,
    Peter

     
  • Max Berger

    Max Berger - 2008-02-21
    • assigned_to: nobody --> maxberger
    • status: open --> closed
     
  • Max Berger

    Max Berger - 2008-02-21

    Logged In: YES
    user_id=99210
    Originator: NO

    Since 3.0.3 is available the issue is closed now. Feel free to re-open if you have further problems.

    I have checked wny you need the Taiwanese fonts: MathML 2 uses some characters from the CJK range for some of its symbols, apparently the over- and underbrace are some of these characters. This was one of their originial design intentions.

    Max

     
  • Raphael Jolly

    Raphael Jolly - 2008-04-07

    Logged In: YES
    user_id=692744
    Originator: NO

    Max,

    The releases don't work for me (neither 3.0.3 nor 3.1.0). The 3.0.3.SNAPSHOT didn't work either, only 3.1.0.SNAPSHOT did. I still have the error:

    Apr 7, 2008 12:30:23 PM org.apache.fop.fonts.FontInfo notifyFontReplacement
    WARNING: Font 'Lucida Bright,normal,400' not found. Substituting with 'any,normal,400'.

    , and a nasty sharp in place of square root in my pdf.

    PS : Currently I use 3.1.0.SNAPSHOT in the meditor release, but I would like to go back to 3.0.3 because:

    1) I can't retroweave 3.1.0 to java 1.4
    2) In 3.1.0 the font is too small (fraction are unreadable)

    Raphael

     
  • Max Berger

    Max Berger - 2008-04-07

    Logged In: YES
    user_id=99210
    Originator: NO

    Raphael,

    could you please
    - open up a new bug reports for the new problems (fraction to small), as this is different from the ones mentioned below, including an example and description of the actual and desired result?

    Here are some general comments:
    - your first problem (# instead of sqrt) is a fop problem. Font management in fop has greatly improved in 0.95beta, so please try that and see if you still have the same problem. Fop is now also easier to configure, so you could ask this question on fop-users. Make sure you set "<auto-detect/>" in the <fonts> section of the fop configuration - this helps greatly. You can also try adding the dejavu jar files (found in the fop distribution) to your classpath - this may help.

    - the retroweaving is not, and will not be supported. Jeuclid, especially 3.1.0 uses a lot of the java 1.5 unicode functionality which is difficult to emulate, this is easier for 3.0.3.

    - when you say "3.1.0-snapshot" did, what version are you referring to (what is the revision or date of the snapshot)? What was the behavior that worked and is now broken?

    Max

     
  • Raphael Jolly

    Raphael Jolly - 2008-04-07

    Logged In: YES
    user_id=692744
    Originator: NO

    Max,

    I was wrong.
    There is no regression inside 3.1.0. The error I get is different. It is due to not having upgraded to fop0.95 as documented. Now I have, and it works.

    In fact there are new issues with fop0.95, for instance I can't begin my document with a math expression, I have to put some text ahead, otherwise the pdf reader is in error. As a consequence I will stay in fop0.94 and jeuclid3.1.0.SNAPSHOT (2008-01-19) for the moment.

    I have been able to solve my font size of fraction issue, with:

    ((MutableLayoutContext)LayoutContextImpl.getDefaultLayoutContext()).setParameter(LayoutContext.Parameter.SCRIPTMINSIZE, new Float(10f));

    , so I'm not opening a new ticket.

    Thanks for all,
    Raphael

     
  • Max Berger

    Max Berger - 2008-04-08

    Logged In: YES
    user_id=99210
    Originator: NO

    Ralph,

    can you send me the complete .fo document to reproduce the error with fop 0.95? I have added some context support, and it appears this is not working if there is no context, but i need an actual example to test this.

    Also: Are you using fop at all or just the mathml->pdf functionality? without any additional text? If so, it may make more sense to use JEuclid without fop: When you add the freehep libraries JEuclid has build-in pdf export functionality.

    The solution you showed for changing the default layout context is not very nice, but at this time probably the only thing that works when embedded in fop. I'll need to change that.

    Max

     
1 2 > >> (Page 1 of 2)

Log in to post a comment.