#191 compound and twofold affix stripping uncompatible

[Arno Teigseth]

I'm trying to move to twofold affix stripping, but run into this problem:

STEM+COMPOUND does work.
STEM+AFFIX+COMPOUND does work.
STEM+AFFIX1+AFFIX2+COMPOUND does not.

Is this by design? Looks strange when only the first level stripping+compound works, and not the second...

-----------files--------------

# VERBS COMMON CONJUGATIONS common first level
SFX v Y 2
SFX v na na//+c na # stem
SFX v na kuna//+c na # gerund

# VERBS COMMON CONJUGATIONS common second level
SFX c Y 2
SFX c na na//+ na # Infinitive
SFX c na nki//+ na # Present 2nd person singular | Future 2nd person singular/plural

--------end files--------
Testing:
$ echo rikuna rikunapash rikumunapash rikunki rikunkipash|hunspell -d qu_EC -m

[STEM] works
rikuna st:rikuna
rikuna st:rikuna # stem
rikuna st:rikuna # stem # Infinitive

[STEM+COMPOUND] works
rikunapash pa:rikuna ( st:rikuna # stem | st:rikuna # stem # Infinitive ) pa:pash

[MORPH level1+COMPOUND] works
rikumunapash pa:rikumuna ( st:rikuna # coming here doing | st:rikuna # coming here doing # Infinitive ) pa:pash

[MORPH level 2] works
rikunki st:rikuna # stem # Present 2nd person singular | Future 2nd person singular/plural

[MORPH level 2+COMPOUND] does not work:
rikunkipash

[Arno Teigseth]

forgot the .dic file:
2
rikuna//vw
pash/x->

The x is COMPOUNDONLY flag, the / is COMPOUNDSTART flag.

[Arno Teigseth]

patch adding checking for twofold suffixes in affixmgr.cxx::compound_check

[Arno Teigseth]

The attached patch fixes checking (not morphing yet) of "stem+suffix_Level1+suffix_Level2+compound[s]".

Before only "stem+suffix_Level1+compound[s]" worked.

[Arno Teigseth]

patch adding checking for twofold suffixes in affixmgr.cxx::compound_check and affixmgr.cxx::compound_check_morph

[Arno Teigseth]

Adding a line calling suffix_check_twosfx to AffixMgr::compound_check and AffixMgr::compound_check_morph solves this bug for me. Patches attached; the biggest has both lines included. BEWARE that I don't know if the last argument to suffix_check_twosfx, FLAG, is useful in this context, and I sat it to 0 as that worked...

[Németh László]

extended patch

[Németh László]

Attached the extended patch, commited to the CVS (the original version caused some incompatibility with the existent dictionaries) Now for this feature, use the new global

COMPOUNDMORESUFFIXES

affix file parameter.

[Arno Teigseth]

Hi I wonder if this was actually fixed? Seems STEM+SUFFIXlevel1+SUFFIXlevel2+COMPOUND doesn't work:

(running latest hunspell from cvs)

[AFF FILE]
SET UTF-8

COMPOUNDFLAG /       # allow compound
COMPOUNDPERMITFLAG + # allow (stem+affix)+compound

COMPOUNDMORESUFFIXES
# Allow 2-fold suffix PLUS compound

# first level
SFX v Y 1
SFX v 0 explained//+c . # Level1

# second level 
SFX c Y 1
SFX c 0 ness//+Y .      # Level2

[DIC FILE]
2
self//v
ish//

[TESTING:]
$ cat testfile |hunspell -d qu_EC -m
self st:self
selfish pa:self st:self pa:ish
selfexplained st:self # Level1
selfexplainedish pa:selfexplained st:self # Level1 pa:ish
selfexplainedness st:self # Level1 # Level2
selfexplainednessish

As you see, hunspell recognizes level1 and level2 but not Level1+Level2+COMPOUND

Am I doing something wrong here, or is this not as fixed as we thought?

[Németh László]

Reopened. Hi Arno, have you tested with the recent code?

[Arno Teigseth]

Yep that was with latest CVS and built this way:

cvs
-d:pserver:anonymous@hunspell.cvs.sourceforge.net:/cvsroot/hunspell login

cvs -z3
-d:pserver:anonymous@hunspell.cvs.sourceforge.net:/cvsroot/hunspell co
-P hunspell

ch hunspell
./configure --prefix=/usr
make
sudo make install

$ hunspell --version
@(#) International Ispell Version 3.2.06 (but really Hunspell 1.3.3)

… … …

all configurue and build went smooth. I have no other hunspell packages installed (did a sudo apt-get purge hunspell before building).

[Arno Teigseth]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 22. nov. 2013 11:44, "Németh László" wrote:

Reopened. Hi Arno, have you tested with the recent code?

Yep that was with latest CVS and built this way:

cvs
- -d:pserver:anonymous@hunspell.cvs.sourceforge.net:/cvsroot/hunspell login

cvs -z3
- -d:pserver:anonymous@hunspell.cvs.sourceforge.net:/cvsroot/hunspell co
- -P hunspell

ch hunspell
./configure --prefix=/usr
make
sudo make install

$ hunspell --version
@(#) International Ispell Version 3.2.06 (but really Hunspell 1.3.3)

… … …

all configurue and build went smooth. I have no other (ubuntu)
hunspell packages (sudo apt-get purge hunspell)

testing:

cat testfile |hunspell -d qu_EC -m
self st:self
selfish pa:self st:self pa:ish
selfexplained st:self # Level1
selfexplainedish pa:selfexplained st:self # Level1 pa:ish
selfexplainedness st:self # Level1 # Level2
selfexplainednessish

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlKPsFsACgkQEMIGVCc8BjCqoACfScHbgYAWbiJr/PnTdqoNLZAf
vHIAoJQdYbSiL+XUuuXKDniasPXIPwq0
=tJKu
-----END PGP SIGNATURE-----

[Arno Teigseth]

Now, if I add a second level that is identical to 1st level, it is picked up by the morph analyst as a homonym (see >> below), so it is somehow recognized as a twofold comopound. It just doesn't work when being "pure" 2nd level.

# second level (hack on line 2: strip then add back "ed". It will be a homonym)
SFX c Y 2
SFX c 0 ness//+Y .      # Level2
SFX c ed ed//+Y .       # Level2 "blank"

TESTING:

cat testfile |hunspell -d qu_EC -m
self  st:self
selfish  pa:self st:self pa:ish
selfexplained  st:self # Level1
selfexplained  st:self # Level1 # Level2 "blank"
>> selfexplainedish  pa:selfexplained (  st:self # Level1 |  st:self # Level1 # Level2 "blank" )  pa:ish
selfexplainedness  st:self # Level1 # Level2
selfexplainednessish