silgraphite-fonts

[Silgraphite-fonts] Another attachment problem

[Carsten B. <ca...@gm...>]

I've already said in a previous mail that I'm working myself on a font 
for an abugida at the moment, although a fictional one. Sharon had 
already suggested a solution in the past, and given the recent 
discussion with Shriramana, I wanted to finally make my thingy work 
properly.

However, I still don't quite understand how some of the rules for 
attachment are processed, so I'd be glad if someone could look over my 
GDL file again and maybe explain what I need to change to make 
everything attach as I wish to (cf. bottom two large-text lines of 
test.pdf in the linked ZIP file). A thing that's unclear to me, for 
example, is that I read somewhere here that precedence is given to 
longer rules – but is that the number of characters in a line, or the 
number of terms?

The GDL file in the ZIP isn't yet complete, it's still missing rules for 
contextual reordering and the TTF is still lacking kerning. Getting the 
reordering to work correctly will likely cause me the next bunch of 
headaches. I've also included the source TTF file, the Graphite-enabled 
TTF file, and the OpenOffice document I made the PDF from for good measure.

See this archive for the mentioned files: 
<http://dl.dropbox.com/u/8026017/tagbkrg-dev_snapshot_2012-07-26.zip> 
(it's too large to attach by email here)

Regards
C. Becker

Re: [Silgraphite-fonts] Another attachment problem

[Shriramana S. <sa...@gm...>]

On Fri, Jul 27, 2012 at 2:27 PM, Carsten Becker <ca...@gm...> wrote:
> A thing that's unclear to me, for
> example, is that I read somewhere here that precedence is given to
> longer rules – but is that the number of characters in a line, or the
> number of terms?

It would be useful to always have the GDL documentation on hand. Page
29, sec 4.1.7:

Within a pass, rules are given a priority order. This priority is
based on the length of the context; i.e.,
longer matches have priority over shorter matches. Thus:
A B C > X Y Z / _ _ _; // priority 1
A > D / _ B; // priority 2
will be tested in the order given here regardless of the order in
which the rules appear in the file. If
several rules have the same length, they will be processed in the
order they appear in the file. This
allows the GDL author to control these subtle ordering issues.

My Krishna project work is already occupying much timeslice of my
brain, so I'm unable to look into your font/GDL myself. Sorry. But I
will monitor the thread for pointers which are potentially helpful to
me.

-- 
Shriramana Sharma

Re: [Silgraphite-fonts] Another attachment problem

[Carsten B. <ca...@gm...>]

Am Fr 27 Jul 2012 11:12:10 CEST schrieb Shriramana Sharma:

> On Fri, Jul 27, 2012 at 2:27 PM, Carsten Becker <ca...@gm...> wrote:

> It would be useful to always have the GDL documentation on hand. Page
> 29, sec 4.1.7:

Ah, I thought somone would say that. I ought to study the documentation 
a little bit more closely indeed.

> My Krishna project work is already occupying much timeslice of my
> brain, so I'm unable to look into your font/GDL myself. Sorry. But I
> will monitor the thread for pointers which are potentially helpful to
> me.

No worries. I should have a look at the files you've posted, too.

C. Becker

Re: [Silgraphite-fonts] Another attachment problem

[Sharon C. <sha...@si...>]

If you use the -D option (if -D doesn't work try -d) on the compiler, 
it will output a set of debugger files including one called 
dbg_ruleprec.txt that shows the complete set of rules (taking optional 
items into account) in precedence order.

Sorry, maybe I'm too tired, but I glanced through your code and it 
wasn't immediately obviously which rules were supposed to apply in 
which situations, and I don't have the mental energy to 
reverse-engineer the whole thing. If you pick a simple example and 
tell me exactly which rules you expect to fire, I can probably help.

On 7/27/2012 3:57 AM, Carsten Becker wrote:
> I've already said in a previous mail that I'm working myself on a font
> for an abugida at the moment, although a fictional one. Sharon had
> already suggested a solution in the past, and given the recent
> discussion with Shriramana, I wanted to finally make my thingy work
> properly.
>
> However, I still don't quite understand how some of the rules for
> attachment are processed, so I'd be glad if someone could look over my
> GDL file again and maybe explain what I need to change to make
> everything attach as I wish to (cf. bottom two large-text lines of
> test.pdf in the linked ZIP file). A thing that's unclear to me, for
> example, is that I read somewhere here that precedence is given to
> longer rules – but is that the number of characters in a line, or the
> number of terms?
>
> The GDL file in the ZIP isn't yet complete, it's still missing rules for
> contextual reordering and the TTF is still lacking kerning. Getting the
> reordering to work correctly will likely cause me the next bunch of
> headaches. I've also included the source TTF file, the Graphite-enabled
> TTF file, and the OpenOffice document I made the PDF from for good measure.
>
> See this archive for the mentioned files:
> <http://dl.dropbox.com/u/8026017/tagbkrg-dev_snapshot_2012-07-26.zip>
> (it's too large to attach by email here)
>
> Regards
> C. Becker
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Silgraphite-fonts mailing list
> Sil...@li...
> https://lists.sourceforge.net/lists/listinfo/silgraphite-fonts

Re: [Silgraphite-fonts] Another attachment problem

[Carsten B. <ca...@gm...>]

On 27.07.2012 20:46, Sharon Correll wrote:

> If you use the -D option (if -D doesn't work try -d) on the compiler, it
> will output a set of debugger files including one called
> dbg_ruleprec.txt that shows the complete set of rules (taking optional
> items into account) in precedence order.

Ah, nice.

>If you pick a simple example and tell me exactly which
> rules you expect to fire, I can probably help.

Fortunately, I could already help myself by playing around a bit and 
peeking from Shriramana's GDL file after I sent my mail.

The problem was how to stack diacritics on diacrtics, but I've 
generalized my rules (a group for all top diacritics, and another for 
all bottom diacritics) and divided them into passes now, so that in the 
first pass, any diacritic is placed over/below a consonant matrice, and 
then in a second pass, diacritics can be attached to the syllable block 
from the first pass. The GDL snippets from the Graphite page were 
helpful, too.

If anyone is interested, I've uploaded the current working draft again, 
at <http://dl.dropbox.com/u/8026017/tagbkrg-dev_snapshot_2012-07-28.zip>

Regards
C. Becker

Re: [Silgraphite-fonts] Another attachment problem

[Sharon C. <sha...@si...>]

On 7/28/2012 5:24 AM, Carsten Becker wrote:
> The problem was how to stack diacritics on diacrtics, but I've
> generalized my rules (a group for all top diacritics, and another for
> all bottom diacritics) and divided them into passes now, so that in the
> first pass, any diacritic is placed over/below a consonant matrice, and
> then in a second pass, diacritics can be attached to the syllable block
> from the first pass. The GDL snippets from the Graphite page were
> helpful, too.

The example of stacking diacritics on the GDL snippets page shows how 
to do it in one pass:

http://scripts.sil.org/cms/scripts/page.php?site_id=projects&item_id=graphite_codeSnippets#fa1aea6b

Note that the takes_upper class includes both bases and other upper 
diacritics. So the single rule handles the chain:

takes_upperupperdiac {attach {to=@1; at=TopS; with=TopM}} / _ LOWERSEQ^ _;

You can just take out the references to lower diacritics if you don't 
have any.

Re: [Silgraphite-fonts] Another attachment problem

[Sharon C. <sha...@si...>]

Oops, HTML or something got me! There probably need to be spaces 
around the caret. That's the key magic that makes the chaining happen.

takes_upper  upperdiac {attach {to=@1; at=TopS; with=TopM}}  /  _ 
LOWERSEQ  ^  _;


On 7/30/2012 3:32 PM, Sharon Correll wrote:
> takes_upperupperdiac {attach {to=@1; at=TopS; with=TopM}} / _ LOWERSEQ^ _;

Re: [Silgraphite-fonts] Another attachment problem

[Carsten B. <ca...@gm...>]

Hi Sharon,

On 30.07.2012 22:36, Sharon Correll wrote:
> Oops, HTML or something got me! There probably need to be spaces
> around the caret. That's the key magic that makes the chaining happen.
>
> takes_upper  upperdiac {attach {to=@1; at=TopS; with=TopM}}  /  _
> LOWERSEQ  ^  _;

I'm afraid I can't get that to work like my two-pass solution for some 
reason. As far as I can tell, in my case I'd have to change the rules 
this way (all the indented lines should be a single one):

    takes_upper = ( clsCons, AnyTopDiacritic );
    takes_lower = ( clsCons, AnyBotDiacritic, clsDiaBotRight );

    takes_upper AnyTopDiacritic { attach { to=@1; at=topAnch;
       with=attGeneric }} / _ BOTSEQ ^ _;
    takes_lower AnyBotMidDiacritic { attach { to=@1; at=botAnchMid;
       with=attGeneric }} / _ TOPSEQ BOTSEQ ^ _;
    takes_lower clsDiaBotRight { attach { to=@1; at=botAnchRight;
       with=attGeneric }} / _ TOPSEQ ^ _;

However, this only stacks the top-diacritics correctly, while ignoring 
the bottom ones. Also, there is a twist in that there are two groups of 
diacritics that can be attached beneath consonants, one in the middle 
under them, one at the right side. It's possible to attach further 
bottom-middle diacritics to either group. My two-pass solution may not 
be as elegant as doing everything in one pass, but at least diacritic 
attachment works reliably for any possible combination:

    pass(1) // First-level attachment
	
       clsCons AnyTopDiacritic { attach { to=@1; level=1; at=topAnch;
          with=attGeneric }; user1 = 1 } / ^ _ BOTSEQ _ { user1 == 0 };
       clsCons AnyBotMidDiacritic { attach { to=@1; level=1;
          at=botAnchMid; with=attGeneric }; user1 = 1 } / ^ _ TOPSEQ _ {
          user1 == 0 };
       clsCons clsDiaBotRight { attach { to=@1; level=1;
          at=botAnchRight; with=attGeneric }; user1 = 1 } / ^ _ TOPSEQ _
          { user1 == 0 };

    endpass; // 1
	
    pass(2) // Second-level attachment (diacritic-to-diacritic)

       AnyTopDiacritic AnyTopDiacritic { attach { to=@1; level=2;
          at=topAnch; with=attGeneric }; user1 = 1 } / ^ _ BOTSEQ _ {
          user1 == 0 };
       (AnyBotMidDiacritic clsDiaBotRight) AnyBotMidDiacritic { attach {
          to=@1; level=2; at=botAnch; with=attGeneric }; user1 = 1 } / ^
          _ TOPSEQ _ { user1 == 0 };

    endpass; // 2

FWIW, I've also tested my font in Firefox 14 now that Firefox has been 
supporting Graphite since a couple of versions back. However, it will 
only follow the rules in the first positioning pass, so it won't do 
diacritic-to-diacritic attachment for me. I've tested this on both 
Ubuntu 12.04 and Windows XP (there with Firefox 16.0.1a). Is this a bug? 
Everything works fine in Libre Office 3.5.5.3.

Regards
Carsten

Re: [Silgraphite-fonts] Another attachment problem

[Sharon C. <sha...@si...>]

On 7/31/2012 12:52 PM, Carsten Becker wrote:
> Hi Sharon,
>
> On 30.07.2012 22:36, Sharon Correll wrote:
>> Oops, HTML or something got me! There probably need to be spaces
>> around the caret. That's the key magic that makes the chaining happen.
>>
>> takes_upper  upperdiac {attach {to=@1; at=TopS; with=TopM}}  /  _
>> LOWERSEQ  ^  _;
> I'm afraid I can't get that to work like my two-pass solution for some
> reason. As far as I can tell, in my case I'd have to change the rules
> this way (all the indented lines should be a single one):
>
>      takes_upper = ( clsCons, AnyTopDiacritic );
>      takes_lower = ( clsCons, AnyBotDiacritic, clsDiaBotRight );
>
>      takes_upper AnyTopDiacritic { attach { to=@1; at=topAnch;
>         with=attGeneric }} / _ BOTSEQ ^ _;
>      takes_lower AnyBotMidDiacritic { attach { to=@1; at=botAnchMid;
>         with=attGeneric }} / _ TOPSEQ BOTSEQ ^ _;
>      takes_lower clsDiaBotRight { attach { to=@1; at=botAnchRight;
>         with=attGeneric }} / _ TOPSEQ ^ _;

You need to move the caret to *before* the BOTSEQ, so the processing 
goes back and handles the bottom diacritics. In general, you want to 
make sure that each rule "consumes" only one character at a time, 
otherwise some will get skipped.

Also there is a problem with the second rule: including BOTSEQ (which 
I assume means "optional bottom diacritics") as part of the context 
means that the last bottom diacritic will be attached first, since the 
longest rule gets matched first. You probably need two optional 
sequences, say BOTMIDSEQ and BOTRTSEQ. And notice that optional items 
imply that the top diacritics come first, before the bottom ones 
(contrary to Unicode specification, but I guess it's up to you in this 
case).

Also you'd probably need to include tests to keep the same rules from 
being fired over and over. I would do it something like this:

#define NOTTOPSEQ [ NotTopDiac NotTopDiac?]?    // these sequences can 
be longer...
#define NOTBOTRIGHTSEQ [ NotBotRightDiac NotBotRightDiac?]?
#define NOTBOTMIDSEQ [ NotBotMidDiac NotBotMidDiac?]?

NotTopDiac = (clsDiaBotMid, clsDiaBotRight);
NotBotRightDiac = (clsDiaTop, clsDiaBotMid);
NotBotMidDiac = (clsDiaTop, clsDiaBotRight);

takes_upper  AnyTopDiacritic {attach...}  /  _  ^ NOTTOPSEQ  _ 
{attach.to == 0};
takes_lower  AnyBotMidDiacritic {attach...}  /  _ ^ NOTBOTMIDSEQ  _ 
{attach.to == 0};
takes_lower  AnyBotRightDiacritic {attach...}  /  _  ^ NOTBOTRIGHTSEQ  
_ {attach.to == 0};

Something like that anyway. Note that to make sure it can handle all 
the combinations, the optional sequences need to be very flexible. You 
might be able to simplify if you can assume that the diacritics appear 
in a particular order.

The tests will generate warnings; it's a slightly unconventional thing 
to do, but it works. Or you can use a user-defined slot attribute, eg:

#define isattached user1
takes_upper  AnyTopDiacritic {attach...; isattached =1}  /  _  ^ 
BOTSEQ  _ {isattached == 0};
takes_lower  AnyBotMidDiacritic {attach...; isattached =1}  /  _ ^ 
TOPorBRTSEQ  _ {isattached == 0};
takes_lower  AnyBotRightDiacritic {attach...; isattached =1}  /  _ ^ 
TOPorBMIDSEQ    _ {isattached == 0};

Just some ideas. Doing it in two passes is fine, but keep in mind that 
every pass slows down the processing. So if performance were an issue, 
one pass would be preferable if you could make it work.

> FWIW, I've also tested my font in Firefox 14 now that Firefox has been
> supporting Graphite since a couple of versions back. However, it will
> only follow the rules in the first positioning pass, so it won't do
> diacritic-to-diacritic attachment for me. I've tested this on both
> Ubuntu 12.04 and Windows XP (there with Firefox 16.0.1a). Is this a bug?
> Everything works fine in Libre Office 3.5.5.3.

:-(  I've never heard of a problem along those lines with Firefox. The 
only bug I know of is that features are not recognized if they don't 
have exactly 4-character IDs (numbers don't work, nor do 3-character IDs).

Re: [Silgraphite-fonts] Another attachment problem

[Sharon C. <sha...@si...>]

Oops, I should have written that to use the NOT... macros, but you get 
the idea.

On 7/31/2012 2:22 PM, Sharon Correll wrote:
> #define isattached user1
> takes_upper  AnyTopDiacritic {attach...; isattached =1}  /  _  ^
> BOTSEQ  _ {isattached == 0};
> takes_lower  AnyBotMidDiacritic {attach...; isattached =1}  /  _ ^
> TOPorBRTSEQ  _ {isattached == 0};
> takes_lower  AnyBotRightDiacritic {attach...; isattached =1}  /  _ ^
> TOPorBMIDSEQ    _ {isattached == 0};

Re: [Silgraphite-fonts] Another attachment problem

[Carsten B. <ca...@gm...>]

On 31.07.2012 21:22, Sharon Correll wrote:

> :-(  I've never heard of a problem along those lines with Firefox. The
> only bug I know of is that features are not recognized if they don't
> have exactly 4-character IDs (numbers don't work, nor do 3-character IDs).

I have no idea why, but if I import the font into Firefox with 
@font-face rather than drawing the data from ~/.fonts or 
C:\Windows\Fonts respectively, it works.

I'm too tired now to look at your suggestions regarding single passes 
(thanks for your patience with me, however!), but I'll try to give it a 
go tomorrow – or at least try to comprehend it!

Here's some testing/demonstration: 
<http://dl.dropbox.com/u/8026017/tagatibookg-test/tagatibookg-test.html>

Regards
Carsten

Re: [Silgraphite-fonts] Another attachment problem

[Sharon C. <sha...@si...>]

You do have Graphite turned on in Firefox, right? You have to 
deliberately do that:

http://scripts.sil.org/cms/scripts/page.php?site_id=projects&item_id=graphite_firefox


On 7/31/2012 3:24 PM, Carsten Becker wrote:
> On 31.07.2012 21:22, Sharon Correll wrote:
>
>> :-(  I've never heard of a problem along those lines with Firefox. The
>> only bug I know of is that features are not recognized if they don't
>> have exactly 4-character IDs (numbers don't work, nor do 3-character IDs).
> I have no idea why, but if I import the font into Firefox with
> @font-face rather than drawing the data from ~/.fonts or
> C:\Windows\Fonts respectively, it works.
>
> I'm too tired now to look at your suggestions regarding single passes
> (thanks for your patience with me, however!), but I'll try to give it a
> go tomorrow – or at least try to comprehend it!
>
> Here's some testing/demonstration:
> <http://dl.dropbox.com/u/8026017/tagatibookg-test/tagatibookg-test.html>
>
> Regards
> Carsten
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Silgraphite-fonts mailing list
> Sil...@li...
> https://lists.sourceforge.net/lists/listinfo/silgraphite-fonts

Re: [Silgraphite-fonts] Another attachment problem

[Carsten B. <ca...@gm...>]

(This was meant to go to the list, not just to Sharon. I clicked "Send" 
too quickly.)

On 01.08.2012 16:16, Sharon Correll wrote:
> You do have Graphite turned on in Firefox, right? You have to
> deliberately do that:

Yeah, I know. I had it turned on in both Ubuntu and Windows already. The 
Graphite testing page works for me, which is why I wondered. Even the 
paragraph that tests my locally installed version of Padauk does. Though 
looking at the GDL source file, Padauk does all positioning in one pass, 
too, and reserves the second pass for kerning ...

On the page with the example I posted yesterday I also included some 
notes about things that don't work. I really hope it's not just me. Now 
I'll have to figure your single-pass solution after all!

Carsten

Re: [Silgraphite-fonts] Another attachment problem

[Carsten B. <ca...@gm...>]

Am Mi 01 Aug 2012 17:04:34 CEST schrieb Carsten Becker:
> On the page with the example I posted yesterday I also included some
> notes about things that don't work. I really hope it's not just me.

After some more experimenting, I found out that the problem with 
Firefox is not due to handling attachment with two passes itself, but 
Unicode codepoints. The diacritics I was trying to attach are all 
relatively at the start of the Private Use Area, i.e. U+E000 upwards. I 
*assume* that this is a bug in how Graphite is compiled into Firefox 
rather than a bug in Graphite itself, since attaching those diacritics 
works flawlessly in Libre Office, as I said.

Carsten

Re: [Silgraphite-fonts] Another attachment problem

[Carsten B. <ca...@gm...>]

Am Mi 01 Aug 2012 23:56:08 CEST schrieb Carsten Becker:
> The diacritics I was trying to attach are all
> relatively at the start of the Private Use Area, i.e. U+E000 upwards.

Correction: U+E010 upwards. Attaching characters used as diacritics 
between U+E000 and U+E00F works for me, even if both passes 
(diacritic-to-base and diacritic-to-diacritic) are involved. I've 
updated the testing page I have already linked previously to 
demonstrate these findings (scroll to the bottom). I hope someone can 
confirm what I'm experiencing.

Carsten