Recent changes to AddingNewLanguage

AddingNewLanguage modified by Markus Toman

Markus Toman — Fri, 08 Nov 2013 12:14:12 -0000

--- v14
+++ v15
@@ -302,6 +302,14 @@

 To `libflhtse_a_SOURCES` add all your new source files.

+Call `automake`and `configure` in the project root directory.
+
+Example configure call:
+
+~~~~
+./configure --with-hts-engine-header-path=/.../hts_engine_API-1.07/include --with-hts-engine-library-path=/.../hts_engine_API-1.07/lib
+~~~~
+

 Adding a new voice to flite+hts_engine
 ======================================

AddingNewLanguage modified by Markus Toman

Markus Toman — Fri, 08 Nov 2013 12:09:13 -0000

--- v13
+++ v14
@@ -188,6 +188,8 @@

 TODO: How to create a new lexicon and lts rules instead of using conversion from festival

+From `flite/lang/cmulex` copy cmu_lex.c, cmu_lex.h, cmu_postlex.h. Rename them and replace all occurrences of "cmu" with your lexicon name.
+

 bin/Makefile.am
 ---------------

AddingNewLanguage modified by Markus Toman

Markus Toman — Thu, 07 Nov 2013 15:51:44 -0000

--- v12
+++ v13
@@ -69,7 +69,15 @@
 Conversion of lexicon
 ---------------------

-From `flite/tools` you need `make_lex.scm`, we put it into a directory `scripts` here.
+From `flite/tools` you need `make_lex.scm` and `huff_table`, we put it into a directory `scripts` here.
+
+In `huff_table` you should edit the following paths to match your configuration:
+
+~~~~
+   $ESTDIR/../festival/bin/festival -b $FLITEDIR/tools/make_lex.scm '(utf8entries "huff.entries.corpus" "huff.tmp.corpus")' 
+~~~~
+
+Of course you can also set the `ESTDIR` and `FLITEDIR` environment variables and put the scripts at the correct locations.

 You can convert your lexicon as following:

@@ -78,11 +86,15 @@
           --libdir $festpath/lib \
           scripts/make_lex.scm \
           -b "(begin (lextoC \"$lexname\" \"$inputlex\" \"$output\"))"
+
+scripts/huff_table phones $output/${lexname}_lex_data $output/${lexname}_lex_phones_huff_table.c
+scripts/huff_table entries $output/${lexname}_lex_data $output/${lexname}_lex_entries_huff_table.c
 ~~~~~

 Set `$lexname` to a name you choose for your lexicon.
 Set `$inputlex` to the path to your lexicon file.
 Set `$output` to the output directory where the .c and .h files should be placed.
+

AddingNewLanguage modified by Markus Toman

Markus Toman — Thu, 07 Nov 2013 13:49:52 -0000

--- v11
+++ v12
@@ -192,6 +192,8 @@
 and replace `cmu_us_kal` with `$voice`, `cmulex` with `$lexicon` and `usenglish` with `$language`.

+
+
 flite/lang/$language/$language.h
 --------------------------------

@@ -281,6 +283,12 @@
 Enter the correct names from above here and don't forget to correct the number of phones (96 in this example).

+lib/Makefile.am
+---------------
+
+To `libflhtse_a_SOURCES` add all your new source files.
+
+
 Adding a new voice to flite+hts_engine
 ======================================

AddingNewLanguage modified by Markus Toman

Markus Toman — Wed, 06 Nov 2013 15:56:32 -0000

--- v10
+++ v11
@@ -172,8 +172,9 @@

 Create a directory `flite/lang/$lexicon`.
 If you have converted your lexicon from a festival lexicon, copy the files here.
-
-TODO: How to create a new lexicon instead of using conversion from festival
+If you have converted your lts rules from a festival lts-tree, copy the files here.
+
+TODO: How to create a new lexicon and lts rules instead of using conversion from festival

 bin/Makefile.am
@@ -336,4 +337,3 @@
 ~~~~~

 And replace "cmu_postlex" with your own postlex function.
-

AddingNewLanguage modified by Markus Toman

Markus Toman — Wed, 06 Nov 2013 15:24:42 -0000

--- v9
+++ v10
@@ -180,7 +180,9 @@
 ---------------

 Copy the lines
+
 ~~~~~~
+::::bash
 -I$(top_srcdir)/flite/lang/cmu_us_kal \
 -I$(top_srcdir)/flite/lang/cmulex \
 -I$(top_srcdir)/flite/lang/usenglish \
@@ -287,37 +289,51 @@
 If you don't plan to use unit selection, you can remove all blocks of `\#ifndef FLITE_PLUS_HTS_ENGINE ... \#endif`.

 Instead of 
-~~~~~
-\#include "usenglish.h"
-\#include "cmu_lex.h"
-~~~~~
+
+~~~~~
+::::C
+#include "usenglish.h"
+#include "cmu_lex.h"
+~~~~~
+
 include your own text analysis and lexicon.

 Replace all occurrences of "cmu_us_kal" with `$voice`.

 In `register_$voice`, replace
+
 ~~~~~~
+::::C
 v->name = "kal";
 ~~~~~~
+
 with a voice name of your choice and

 ~~~~~
+::::C
 usenglish_init(v);
 ~~~~~
+
 with the initialization function of your language. 

 Also, change
-~~~~~
+
+~~~~~
+::::C
     /* Lexicon */
     lex = at_lex_init();
 ~~~~~
+
 to your own lexicon initialization function.

 Finally, find
-~~~~~
+
+~~~~~
+::::C
 static cst_utterance *$voice_postlex(cst_utterance *u)
 {
     cmu_postlex(u);
 ~~~~~
+
 And replace "cmu_postlex" with your own postlex function.

AddingNewLanguage modified by Markus Toman

Markus Toman — Wed, 06 Nov 2013 15:19:35 -0000

--- v8
+++ v9
@@ -48,8 +48,8 @@
 Conversion of phone set
 -----------------------

-Using make_phoneset.scm you can generate C-code for flite for the phone set definition.
-Output file will be called: ${phonesetname}_phoneset.c and placed in $outputdir.
+Using `make_phoneset.scm` you can generate C-code for flite for the phone set definition.
+Output file will be called: `${phonesetname}_phoneset.c` and placed in `$outputdir`.

 ~~~~~~~~~~~~~~~~~
 phonesetname=myPhoneset
@@ -63,13 +63,13 @@
        -b "(phonesettoC $phonesetname (car (load "$phonesetdef" t)) $silence $outputdir)"
 ~~~~~~~~~~~~~~~~~~~

-The output file should later be placed in flite+hts_engine/flite/lang/$language.
+The output file should later be placed in `flite+hts_engine/flite/lang/$language`.

 Conversion of lexicon
 ---------------------

-From flite/tools you need make_lex.scm, we put it into a directory "scripts" here.
+From `flite/tools` you need `make_lex.scm`, we put it into a directory `scripts` here.

 You can convert your lexicon as following:

@@ -80,18 +80,18 @@
           -b "(begin (lextoC \"$lexname\" \"$inputlex\" \"$output\"))"
 ~~~~~

-Set $lexname to a name you choose for your lexicon.
-Set $inputlex to the path to your lexicon file.
-Set $output to the output directory where the .c and .h files should be placed.
+Set `$lexname` to a name you choose for your lexicon.
+Set `$inputlex` to the path to your lexicon file.
+Set `$output` to the output directory where the .c and .h files should be placed.

 Conversion of letter-to-sound rules
 -----------------------------------

-From flite/tools you need make_lts_wfst.scm and make_lts.scm, we put them into a directory "scripts" here.
-
-For make_lts_wfst make sure you have set the $ESTDIR directory to the Edinburgh Speech Tools binary directory, else you can change the path in make_lts_wfst.scm here:
+From `flite/tools` you need `make_lts_wfst.scm` and `make_lts.scm`, we put them into a directory `scripts` here.
+
+For `make_lts_wfst.scm` make sure you have set the `$ESTDIR` directory to the Edinburgh Speech Tools binary directory, else you can change the path in `make_lts_wfst.scm` here:

 ~~~~
      (system 
@@ -100,7 +100,6 @@
          odir (car a)
          odir (car a)))
 ~~~~
-

 You can then convert your scheme LTS rules as following:
@@ -114,12 +113,11 @@
    -b "(begin (lts_to_rg_to_wfst lts_rules \"$output\") (ltsregextoC \"$ltsname\" lts_rules \"$output\" \"$output\"))"
 ~~~~

-Set $inputlts to a path to your lts scheme file.
-Set $output to an arbitrary output directory.
-Set $ltsname to a name for your lts rules.
-
-
-You will encounter problems if you have other features than defined in make_lts.scm:
+Set `$inputlts` to a path to your lts scheme file.
+Set `$output` to an arbitrary output directory.
+Set `$ltsname` to a name for your lts rules.
+
+You will encounter problems if you have other features than defined in `make_lts.scm`:

 ~~~~
 (define (lts_feat trans)
@@ -146,34 +144,33 @@
 =========================================

 Here we will use the approach to copy and modify the existing "usenglish" code.
-We will refer to your new language as $language. You will replace occurrences of "usenglish" with $language.
-Also we will refer to your language shortcut as $ln. You will replace occurrences of "us" with $ln.
-$voice will be used for the newly added voice (if any) and $lexicon for the name of the newly added lexicon.
+We will refer to your new language as `$language`. You will replace occurrences of "usenglish" with `$language`.
+Also we will refer to your language shortcut as `$ln`. You will replace occurrences of "us" with `$ln`.
+`$voice` will be used for the newly added voice (if any) and `$lexicon` for the name of the newly added lexicon.

 If you don't plan to use unit selection voices, you can remove all blocks of
-\#ifndef FLITE_PLUS_HTS_ENGINE
-\#endif
+`\#ifndef FLITE_PLUS_HTS_ENGINE ... \#endif`

 flite/lang/$language
 --------------------

-First copy flite+hts_engine/flite/lang/usenglish to flite+hts_engine/flite/lang/$language.
-
-The main entry point for your text analysis are $language.c and $language.h.
+First copy `flite+hts_engine/flite/lang/usenglish` to `flite+hts_engine/flite/lang/$language`.
+
+The main entry point for your text analysis are `$language.c` and `$language.h`.

 flite/lang/$voice
 -----------------

-Create a directory flite/lang/$voice for now if you plan to use a new voice model.
+Create a directory `flite/lang/$voice` for now if you plan to use a new voice model.
 See the separate chapter for adding a new voice to flite+hts_engine.

 flite/lang/$lexicon
 -------------------

-Create a directory flite/lang/$lexicon.
+Create a directory `flite/lang/$lexicon`.
 If you have converted your lexicon from a festival lexicon, copy the files here.

 TODO: How to create a new lexicon instead of using conversion from festival
@@ -184,25 +181,25 @@

 Copy the lines
 ~~~~~~
-        -I$(top_srcdir)/flite/lang/cmu_us_kal \
-        -I$(top_srcdir)/flite/lang/cmulex \
-        -I$(top_srcdir)/flite/lang/usenglish \
+-I$(top_srcdir)/flite/lang/cmu_us_kal \
+-I$(top_srcdir)/flite/lang/cmulex \
+-I$(top_srcdir)/flite/lang/usenglish \
 ~~~~~~

-and replace cmu_us_kal with $voice, cmulex with $lexicon and usenglish with $language.
+and replace `cmu_us_kal` with `$voice`, `cmulex` with `$lexicon` and `usenglish` with `$language`.

 flite/lang/$language/$language.h
-----------------------------------
-
-Don't forget to change the #ifndef and #define directives to something unique.
+--------------------------------
+
+Don't forget to change the `\#ifndef` and `\#define` directives to something unique.

 ~~~~
 /* Voices call this to use usenglish. */
 void usenglish_init(cst_voice *v);
 ~~~~

-This is the initialization function for your new module - rename usenglish to $language.
+This is the initialization function for your new module - rename "usenglish" to `$language`.

 ~~~~
 /* Default functions and values that you might need. */
@@ -284,10 +281,10 @@
 Adding a new voice to flite+hts_engine
 ======================================

-Here we will use $voice as the name of the new voice to be added.
-
-As a first step, copy flite/lang/cmu_us_kal to flite/lang/$voice.
-If you don't plan to use unit selection, you can remove all blocks of \#ifndef FLITE_PLUS_HTS_ENGINE ... \#endif.
+Here we will use `$voice` as the name of the new voice to be added.
+
+As a first step, copy `flite/lang/cmu_us_kal` to `flite/lang/$voice`.
+If you don't plan to use unit selection, you can remove all blocks of `\#ifndef FLITE_PLUS_HTS_ENGINE ... \#endif`.

 Instead of 
 ~~~~~
@@ -296,9 +293,9 @@
 ~~~~~
 include your own text analysis and lexicon.

-Replace all occurrences of "cmu_us_kal" with $voice.
-
-In register_$voice, replace
+Replace all occurrences of "cmu_us_kal" with `$voice`.
+
+In `register_$voice`, replace
 ~~~~~~
 v->name = "kal";
 ~~~~~~
@@ -322,6 +319,5 @@
 {
     cmu_postlex(u);
 ~~~~~
-And replace cmu_postlex with your own postlex function.
-
-
+And replace "cmu_postlex" with your own postlex function.
+

AddingNewLanguage modified by Markus Toman

Markus Toman — Wed, 06 Nov 2013 15:10:56 -0000

--- v7
+++ v8
@@ -183,11 +183,11 @@
 ---------------

 Copy the lines
-~~~~   
+~~~~~~
         -I$(top_srcdir)/flite/lang/cmu_us_kal \
         -I$(top_srcdir)/flite/lang/cmulex \
         -I$(top_srcdir)/flite/lang/usenglish \
-~~~~
+~~~~~~

 and replace cmu_us_kal with $voice, cmulex with $lexicon and usenglish with $language.

@@ -290,39 +290,38 @@
 If you don't plan to use unit selection, you can remove all blocks of \#ifndef FLITE_PLUS_HTS_ENGINE ... \#endif.

 Instead of 
-~~~~
-#include "usenglish.h"
-#include "cmu_lex.h"
-~~~~
+~~~~~
+\#include "usenglish.h"
+\#include "cmu_lex.h"
+~~~~~
 include your own text analysis and lexicon.

 Replace all occurrences of "cmu_us_kal" with $voice.

 In register_$voice, replace
-~~~~~
+~~~~~~
 v->name = "kal";
-~~~~~
+~~~~~~
 with a voice name of your choice and

-~~~~
+~~~~~
 usenglish_init(v);
-~~~~
+~~~~~
 with the initialization function of your language. 

 Also, change
-~~~~
+~~~~~
     /* Lexicon */
     lex = at_lex_init();
-~~~~
+~~~~~
 to your own lexicon initialization function.

 Finally, find
-~~~~
+~~~~~
 static cst_utterance *$voice_postlex(cst_utterance *u)
 {
     cmu_postlex(u);
-~~~~
+~~~~~
 And replace cmu_postlex with your own postlex function.

-

AddingNewLanguage modified by Markus Toman

Markus Toman — Wed, 06 Nov 2013 15:04:23 -0000

--- v6
+++ v7
@@ -192,24 +192,24 @@
 and replace cmu_us_kal with $voice, cmulex with $lexicon and usenglish with $language.

-$language.h
------------
-
-Don't forget to change the #ifndef and #define directives
+flite/lang/$language/$language.h
+----------------------------------
+
+Don't forget to change the #ifndef and #define directives to something unique.

 ~~~~
 /* Voices call this to use usenglish. */
-void atgerman_init(cst_voice *v);
-~~~~
-
-This is the initialization function for your new module - rename it to reflect your language.
+void usenglish_init(cst_voice *v);
+~~~~
+
+This is the initialization function for your new module - rename usenglish to $language.

 ~~~~
 /* Default functions and values that you might need. */
-extern const cst_phoneset at_phoneset;
-extern const cst_cart at_phrasing_cart;       //TODO: perhaps remove this
-extern const cst_cart at_int_accent_cart;     //TODO: perhaps remove this
-extern const cst_cart at_int_tone_cart;       //TODO: perhaps remove this
+extern const cst_phoneset us_phoneset;
+extern const cst_cart us_phrasing_cart;       //TODO: perhaps remove this
+extern const cst_cart us_int_accent_cart;     //TODO: perhaps remove this
+extern const cst_cart us_int_tone_cart;       //TODO: perhaps remove this
 extern const cst_cart us_pos_cart;            //TODO: perhaps remove this
 ~~~~

@@ -218,37 +218,37 @@
 Another possibility is to try the english CART trees.

-$language.c
-------------
+flite/lang/$language/$language.c
+--------------------------------

 Replace all occurrences of "en" or "us".

 TODO: Change CARTS

-${ln}_text.h
-------------
+flite/lang/$language/${ln}_text.h
+----------------------------------

 This is the main entry point for the text analysis.

 Replace all occurrences of "en", "us" or "usenglish".

 ~~~~
-extern const cst_cart at_nums_cart;
+extern const cst_cart us_nums_cart;
 ~~~~
 Remove or comment this number CART tree if you don't plan to use one.

-${ln}_text.h
-------------
+flite/lang/$language/${ln}_text.h
+----------------------------------

 This is the main entry point for the text analysis.

 Replace all occurrences of "en", "us" or "usenglish".

-${ln}_phoneset.c
-----------------
+flite/lang/$language/${ln}_phoneset.c
+-------------------------------------

 If you have a scheme phone set for festival you can generate this file (see the appropriate section above).

AddingNewLanguage modified by Markus Toman

Markus Toman — Wed, 06 Nov 2013 15:00:49 -0000

--- v5
+++ v6
@@ -148,19 +148,49 @@
 Here we will use the approach to copy and modify the existing "usenglish" code.
 We will refer to your new language as $language. You will replace occurrences of "usenglish" with $language.
 Also we will refer to your language shortcut as $ln. You will replace occurrences of "us" with $ln.
-Code samples will contain "atgerman" as $language as it is taken from our own process.
+$voice will be used for the newly added voice (if any) and $lexicon for the name of the newly added lexicon.
+

 If you don't plan to use unit selection voices, you can remove all blocks of
 \#ifndef FLITE_PLUS_HTS_ENGINE
 \#endif

-$language
----------
+flite/lang/$language
+--------------------

 First copy flite+hts_engine/flite/lang/usenglish to flite+hts_engine/flite/lang/$language.

 The main entry point for your text analysis are $language.c and $language.h.
+
+
+flite/lang/$voice
+-----------------
+
+Create a directory flite/lang/$voice for now if you plan to use a new voice model.
+See the separate chapter for adding a new voice to flite+hts_engine.
+
+flite/lang/$lexicon
+-------------------
+
+Create a directory flite/lang/$lexicon.
+If you have converted your lexicon from a festival lexicon, copy the files here.
+
+TODO: How to create a new lexicon instead of using conversion from festival
+
+
+bin/Makefile.am
+---------------
+
+Copy the lines
+~~~~   
+        -I$(top_srcdir)/flite/lang/cmu_us_kal \
+        -I$(top_srcdir)/flite/lang/cmulex \
+        -I$(top_srcdir)/flite/lang/usenglish \
+~~~~
+
+and replace cmu_us_kal with $voice, cmulex with $lexicon and usenglish with $language.
+

 $language.h
 -----------
@@ -249,3 +279,50 @@
 ~~~~

 Enter the correct names from above here and don't forget to correct the number of phones (96 in this example).
+
+
+Adding a new voice to flite+hts_engine
+======================================
+
+Here we will use $voice as the name of the new voice to be added.
+
+As a first step, copy flite/lang/cmu_us_kal to flite/lang/$voice.
+If you don't plan to use unit selection, you can remove all blocks of \#ifndef FLITE_PLUS_HTS_ENGINE ... \#endif.
+
+Instead of 
+~~~~
+#include "usenglish.h"
+#include "cmu_lex.h"
+~~~~
+include your own text analysis and lexicon.
+
+Replace all occurrences of "cmu_us_kal" with $voice.
+
+In register_$voice, replace
+~~~~~
+v->name = "kal";
+~~~~~
+with a voice name of your choice and
+
+~~~~
+usenglish_init(v);
+~~~~
+with the initialization function of your language. 
+
+Also, change
+~~~~
+    /* Lexicon */
+    lex = at_lex_init();
+~~~~
+to your own lexicon initialization function.
+
+Finally, find
+~~~~
+static cst_utterance *$voice_postlex(cst_utterance *u)
+{
+    cmu_postlex(u);
+~~~~
+And replace cmu_postlex with your own postlex function.
+
+
+