No segment break after an abbreviation
Brought to you by:
ole_yansen
Menu
▾
▴
#11 No segment break after an abbreviation
See the test case to the report #1883425. If after a first attempt to open a segment, I opt to expand the segment, segmentation stops over the second part of the abbreviation, thus the segment ending with “e.g.”. Normally, this should be avoided. I see two ways of coping with the problem:
a) either to have list(s) of abbreviations for languages, or
b) simply not to break a segment, if a point is followed by a minuscule letter (for Latin/Cyrillic/Greek scripts, at least).
Logged In: YES
user_id=1938724
Originator: NO
Solution b won't work for German, where we have "z. B.", "i. A." and "m. E."
The easiest solution would be to support SRX format for sentence segmentation:
https://en.wikipedia.org/wiki/Segmentation_Rules_eXchange
There is open-source library that supports SRX:
https://sourceforge.net/projects/segment/?source=navbar
It's Java but since Anaphraseus uses Java, it might work at least for Anaphraseus-Java projects.
Thank you. I'll take look at this.