Search Results for "utf-16"

The Linguistic Analyzer (Almuhalil Alloghawy) is a free tool designed by a team from Al-Imam Muhammad bin Saud islamic university that can be used for corpus analysis and comparison in terms of the several linguistic characteristics, such as frequency lists generation, concordances, collocation extraction, the difference between two words, and keyword identification.

...It can generate index of word elements extracted from defined text set. Word elements may be words, N-grams or phrases (syntagmes). The program can process texts as in ordinary 1-byte encoding (ANSI), as in multibyte UTF-8 encoding.

...Lexical patterns search e. Two corpora frequency profile comparison based on MI, CHI, LL, T-Score, Z Score, Dice, Log Dice, Weirdness Coefficient f. Accept Windows and UTF-8 character encoding g. Accept TXT, DOC, DOCX, RTF and HTML formats h. Export the processing results in CSV file format

...Please note that the actual WordNet for Romanian (the XML file containing the network) is not included, it can be obtained (due to its license restrictions) from: http://ws.racai.ro:9191/repository/browse/romanian-wordnet-30/4611a43efb6811e2a8ad00237df3e3580b6b50d1111c4a6292694bded91d5c14/ **** If you would like a direct download of the API containing the RoWordNet xml file, please download them both from : http://www.racai.ro/tools/text/rowordnet/ **** Please cite: Dumitrescu, Ștefan Daniel. RoWordNetLib - The first API for the Romanian WordNet. In Proceedings of the Romanian Academy, Series A (The Publishing House of the Romanian Academy). vol. 16, no. 1, pp. 87--94, 2015 Article PDF at: http://www.acad.ro/sectii2002/proceedings/doc2015-1/12-Dumitrescu.pdf It's free for research use. Use the demo.java as a quick-start guide for using this software.

...There has been lot of significant work in the development and evaluation of stemmer for non-Indian languages, but very less or no significant work has been done on Indian front especially for Gujarati language.The code of this stemmer is based on algorithm designed under guidance of Prof. Nikita Desai, India. It takes input file of type .txt containing Gujarati text encoded as UTF-8 and then removes stop words which are unessential. After processing rest of the words, it outputs corresponding file containing all stem words plus other details.

Perstem is a Persian (Farsi) stemmer, morphological analyzer, transliterator, and partial part-of-speech tagger. Inflexional morphemes are separated or removed from their stems. Perstem can also tokenize and transliterate between various character set encodings and romanizations.

A (UTF-8) Chinese word segmentation program based on ngram language model. It is written in C++, provides multi-threading and high throughput.

Nasira is a Java library for reading text files with non-ASCII characters (e.g. documents in German, Swedish,...). To do so, it automatically determines the character encoding (iso-8859-1, utf-8) used to encode the file through user-provided hints.