postchunk gives misformatted output if space at end of chunk

The free and open-source rule-based machine translation platform

Brought to you by: jezral, mlforcada, nordfalk, sanmarf, and 3 others

#105 postchunk gives misformatted output if space at end of chunk

Status: open

Owner: nobody

Labels: transfer (5)

Updated: 2016-06-09

Created: 2016-06-09

Creator: Kevin Brubeck Unhammer

Private: No

say a transfer rule outputs
^Imp<SV><@+FAUXV>{^ij<vblex><neg><imp><p2><sg>$} $
(due to <chunk>...<b/></chunk> which the DTD allows)
then postchunk gives

$ echo '^Imp<SV><@+FAUXV>{^ij<vblex><neg><imp><p2><sg>$} $ ^adv<ADV><@←ADVL>{^olkese<adv>$}$'|apertium-postchunk apertium-sme-sma.sme-sma.t5x sme-sma.t5x.bin                                 
^Ij<vblex><neg><imp><p2><sg>$} $ ^adv<ADV><@←ADVL>{^olkese<adv>$

and hfst-proc crashes on that.

The DTD should disallow it, or interchunk should fix it behind the scenes, or postchunk should output correctly even if the input is weird.

postchunk gives misformatted output if space at end of chunk

The free and open-source rule-based machine translation platform

Searches

Help

#105 postchunk gives misformatted output if space at end of chunk

Discussion