kaldi-developers — Kaldi Developers

Re: [Kaldi-developers] Windows patches to src/ (mostly)

[Daniel P. <dp...@gm...>]

Thanks a lot for these.
Yenda will deal with the checking-in issues.
Generally speaking I like your patches, but I'll just reiterate here
some comments I made on github:
I'm not happy with the way you worked around the MSVC compiler bug by
moving main() into namespace kaldi.  I believe this is not allowed,
e.g. see here http://stackoverflow.com/questions/3956678/main-in-namespace,
and even if it were allowed it seems very wrong to me; and I doubt it
would compile on all platforms.  I'm OK to deal with MSVC's brokenness
by adding various #ifdefs, but actually breaking Kaldi code is where I
draw the line.  I'm sure there must be less heavyweight ways to deal
with this problem?  I don't know what error message it was that you
got, that you are trying to get rid of?


Dan

On Mon, May 18, 2015 at 8:54 PM, Kirill Katsnelson
<kir...@sm...> wrote:
> I have collected all my source changes into 4 separate commits in my git fork. Please review the changes at the following links:
>
> The first change is trivial, but is necessary for the whole rig to work properly. EOL conventions are hard to fix later and cause the most confusion in version control.
> https://github.com/kaldi-asr/kaldi/commit/bba053
>
> Compatibility changes for Windows cl (VS2013) and icl (Intel XE 2015 Update 3)
> https://github.com/kaldi-asr/kaldi/commit/1cc30d
>
> This is a change to fix the apparent MSVC bug with namespace resolution in template expansion
> https://github.com/kaldi-asr/kaldi/commit/f0f9e7
>
> And the last deals with proper I/O binarization under Windows, and Cygwin path conversions.
> The change depends on the 2nd from top above.
> https://github.com/kaldi-asr/kaldi/commit/5d2cc3
>
> There are also changes to openfst files and headers, but I do not know how to deal with them:
>  - Check in the patches to be applied on top of existing patches?
>  - Provide pre-build binaries?
>  - Check in complete sources as part of kaldi repo?
>
> And there are build scripts, coming soon.
>
> Thanks,
>
>  -kkm
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers

[Kaldi-developers] Windows patches to src/ (mostly)

[Kirill K. <kir...@sm...>]

I have collected all my source changes into 4 separate commits in my git fork. Please review the changes at the following links:

The first change is trivial, but is necessary for the whole rig to work properly. EOL conventions are hard to fix later and cause the most confusion in version control.
https://github.com/kaldi-asr/kaldi/commit/bba053

Compatibility changes for Windows cl (VS2013) and icl (Intel XE 2015 Update 3)
https://github.com/kaldi-asr/kaldi/commit/1cc30d

This is a change to fix the apparent MSVC bug with namespace resolution in template expansion 
https://github.com/kaldi-asr/kaldi/commit/f0f9e7

And the last deals with proper I/O binarization under Windows, and Cygwin path conversions.
The change depends on the 2nd from top above.
https://github.com/kaldi-asr/kaldi/commit/5d2cc3

There are also changes to openfst files and headers, but I do not know how to deal with them:
 - Check in the patches to be applied on top of existing patches?
 - Provide pre-build binaries?
 - Check in complete sources as part of kaldi repo?

And there are build scripts, coming soon.

Thanks,

 -kkm

Re: [Kaldi-developers] text-utils.cc:27:17: error: ‘c’ was not declared in this scope

[Patrick J. S. <pat...@ld...>]

Thank you!

-----Original Message-----
From: Daniel Povey [mailto:dp...@gm...] 
Sent: Friday, May 15, 2015 8:24 PM
To: Patrick John Schone
Cc: kal...@li...; kal...@li...
Subject: Re: text-utils.cc:27:17: error: ‘c’ was not declared in this scope

Ubuntu is a very common variety of Linux and I'm confident that it's not a problem with Kaldi.
In fact, "isascii" does not appear on line 27 of text-utils.cc.
I think you may have accidentally modified or corrupted that file.  Do "svn revert -R" in that directory to revert to the checked-out version.  "svn status" will let you know how the code has changed.
Dan


On Fri, May 15, 2015 at 10:17 PM, Patrick John Schone <pat...@ld...> wrote:
> Hi, All,
>
>    Sorry to bother you with trivialities, but I hoped that someone 
> might know the answer to this question.
>
>   I have been using Kaldi on Cygwin for months -- no problems.  But 
> now I am now trying to compile it on Ubuntu and I keep getting the following problem:
>
>
>
> text-utils.cc:27:17: error: ‘c’ was not declared in this scope
>
> _DEFUN(isascii,(c),int c)
>
>                  ^
>
> text-utils.cc:27:20: error: expected primary-expression before ‘int’
>
> _DEFUN(isascii,(c),int c)
>
>                     ^
>
> text-utils.cc:27:25: error: expression list treated as compound 
> expression in initializer [-fpermissive]
>
> _DEFUN(isascii,(c),int c)
>
>                          ^
>
> text-utils.cc:28:1: error: expected ‘,’ or ‘;’ before ‘{’ token
>
> {
>
> ^
>
> make[1]: *** [text-utils.o] Error 1
>
>
>
> This is happening here:
>
> make[1]: Entering directory
> `/home/boisebound/Experiments/OCRandHR/KALDI/src/util'
>
> g++ -msse -msse2 -Wall -I.. -pthread -DKALDI_DOUBLEPRECISION=0
> -DHAVE_POSIX_MEMALIGN -Wno-sign-compare -Wno-unused-local-typedefs 
> -Winit-self -DHAVE_EXECINFO_H=1 -rdynamic -DHAVE_CXXABI_H -DHAVE_ATLAS 
> -I/home/.../KALDI/tools/ATLAS/include
> -I/home/.../KALDI/tools/openfst/include -DHAVE_OPENFST_GE_10400 -std=c++0x
> -O2    -c -o text-utils.o text-utils.cc
>
>
>
> I have tried compiling with various optimization levels since that 
> helped in Cygwin, but no success.  Any thoughts?
>
>
>
> Thank you!
>
> Pat Schone

[Kaldi-developers] text-utils.cc:27:17: error: ‘c’ was not declared in this scope

[Patrick J. S. <pat...@ld...>]

Hi, All,

   Sorry to bother you with trivialities, but I hoped that someone might know the answer to this question.

  I have been using Kaldi on Cygwin for months -- no problems.  But now I am now trying to compile it on Ubuntu and I keep getting the following problem:



text-utils.cc:27:17: error: ‘c’ was not declared in this scope

_DEFUN(isascii,(c),int c)

                 ^

text-utils.cc:27:20: error: expected primary-expression before ‘int’

_DEFUN(isascii,(c),int c)

                    ^

text-utils.cc:27:25: error: expression list treated as compound expression in initializer [-fpermissive]

_DEFUN(isascii,(c),int c)

                         ^

text-utils.cc:28:1: error: expected ‘,’ or ‘;’ before ‘{’ token

{

^

make[1]: *** [text-utils.o] Error 1



This is happening here:

make[1]: Entering directory `/home/boisebound/Experiments/OCRandHR/KALDI/src/util'

g++ -msse -msse2 -Wall -I.. -pthread -DKALDI_DOUBLEPRECISION=0 -DHAVE_POSIX_MEMALIGN -Wno-sign-compare -Wno-unused-local-typedefs -Winit-self -DHAVE_EXECINFO_H=1 -rdynamic -DHAVE_CXXABI_H -DHAVE_ATLAS -I/home/.../KALDI/tools/ATLAS/include -I/home/.../KALDI/tools/openfst/include -DHAVE_OPENFST_GE_10400 -std=c++0x -O2    -c -o text-utils.o text-utils.cc



I have tried compiling with various optimization levels since that helped in Cygwin, but no success.  Any thoughts?



Thank you!

Pat Schone

Re: [Kaldi-developers] text-utils.cc:27:17: error: ‘c’ was not declared in this scope

[Daniel P. <dp...@gm...>]

Ubuntu is a very common variety of Linux and I'm confident that it's
not a problem with Kaldi.
In fact, "isascii" does not appear on line 27 of text-utils.cc.
I think you may have accidentally modified or corrupted that file.  Do
"svn revert -R" in that directory to revert to the checked-out
version.  "svn status" will let you know how the code has changed.
Dan


On Fri, May 15, 2015 at 10:17 PM, Patrick John Schone
<pat...@ld...> wrote:
> Hi, All,
>
>    Sorry to bother you with trivialities, but I hoped that someone might
> know the answer to this question.
>
>   I have been using Kaldi on Cygwin for months -- no problems.  But now I am
> now trying to compile it on Ubuntu and I keep getting the following problem:
>
>
>
> text-utils.cc:27:17: error: ‘c’ was not declared in this scope
>
> _DEFUN(isascii,(c),int c)
>
>                  ^
>
> text-utils.cc:27:20: error: expected primary-expression before ‘int’
>
> _DEFUN(isascii,(c),int c)
>
>                     ^
>
> text-utils.cc:27:25: error: expression list treated as compound expression
> in initializer [-fpermissive]
>
> _DEFUN(isascii,(c),int c)
>
>                          ^
>
> text-utils.cc:28:1: error: expected ‘,’ or ‘;’ before ‘{’ token
>
> {
>
> ^
>
> make[1]: *** [text-utils.o] Error 1
>
>
>
> This is happening here:
>
> make[1]: Entering directory
> `/home/boisebound/Experiments/OCRandHR/KALDI/src/util'
>
> g++ -msse -msse2 -Wall -I.. -pthread -DKALDI_DOUBLEPRECISION=0
> -DHAVE_POSIX_MEMALIGN -Wno-sign-compare -Wno-unused-local-typedefs
> -Winit-self -DHAVE_EXECINFO_H=1 -rdynamic -DHAVE_CXXABI_H -DHAVE_ATLAS
> -I/home/.../KALDI/tools/ATLAS/include
> -I/home/.../KALDI/tools/openfst/include -DHAVE_OPENFST_GE_10400 -std=c++0x
> -O2    -c -o text-utils.o text-utils.cc
>
>
>
> I have tried compiling with various optimization levels since that helped in
> Cygwin, but no success.  Any thoughts?
>
>
>
> Thank you!
>
> Pat Schone

Re: [Kaldi-developers] copy-transition-model - binary is 70 times larger than text

[Kirill K. <kir...@sm...>]

> From: Kirill Katsnelson [mailto:kir...@sm...]
> Sent: 2015-05-13 1752
> 
> Why is the binary transition model file 70 times larger than the same
> model in the text form? I am talking 33.5 vs 0.5 MB.

Silly me. Pilot error, the .mdl contained an acoustic model too.

 --kkm

Re: [Kaldi-developers] copy-transition-model - binary is 70 times larger than text

[Daniel P. <dp...@gm...>]

Are you sure it is that large in binary form?  transition models are
not normally written by themselves to disk (except in Karel's
scripts).  There might be something else after the transition-model
itself, in the file.
Dan


On Wed, May 13, 2015 at 8:52 PM, Kirill Katsnelson
<kir...@sm...> wrote:
> Why is the binary transition model file 70 times larger than the same model in the text form? I am talking 33.5 vs 0.5 MB.
>
>  -kkm
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers

[Kaldi-developers] copy-transition-model - binary is 70 times larger than text

[Kirill K. <kir...@sm...>]

Why is the binary transition model file 70 times larger than the same model in the text form? I am talking 33.5 vs 0.5 MB.

 -kkm

Re: [Kaldi-developers] sph audio, sox vs sph2pipe

[Jan T. <jt...@gm...>]

I will try to provide you with a file.  I believe  timit  and perhaps wsj
files are shorten-compressed - perhaps you could just look at the ldc
site,  as sometimes  they provide samples of the recordings.
Y.
On May 13, 2015 8:55 PM, "Kirill Katsnelson" <
kir...@sm...> wrote:

> shorten(1) is very outdated, so there would be no benefit adopting it over
> nist sphere tools. Current programs that support .shn files and the shorten
> compression are ffmpeg/avconv, but I doubt they can directly uncompress a
> shorten'd .sph. It all gets complicated quickly.
>
> But using sox on uncompressed sets like TEDlium is probably more portable.
>
> If anyone has a sample shorten-compressed .sph file, send it my way please.
>
>  -kkm
>
> > -----Original Message-----
> > From: Jan Trmal [mailto:jt...@gm...]
> > Sent: 2015-05-13 0315
> > To: Kirill Katsnelson
> > Cc: Anthony Rousseau; Dan Povey; kal...@li...
> > Subject: RE: [Kaldi-developers] sph audio, sox vs sph2pipe
> >
> > I remember I tried several times and I had the problems with "shorten".
> >
> > Sox refuses to work with those sph files.
> >
> > What I did IIRC, I took shorten.exe (I don't remember where I got it,
> > but something can be found here:
> > http://www.rarewares.org/lossless.php),
> > which does not support sph format -- but has some switch that allows to
> > ignore first 1024 bytes of the recording (or something like that). That
> > binary produced either A/mu-law or 16bit-linear raw audio, which I
> > processed using sox as usuall.
> >
> > HTH
> >
> > y.
> >
> >
> >
> >
> > On May 13, 2015 7:40 AM, "Kirill Katsnelson"
> > <kir...@sm...> wrote:
> >
> >
> >       It very likely does not. I did not know it was common though.
> > AFAIK, it just outputs a compressed stream, but I have no way to test.
> >
> >       Can it be piped to ffmpeg or avconv? If I could get hold of a
> > compressed file example, I'd test.
> >
> >        -kkm
> >
> >       > -----Original Message-----
> >       > From: Jan Trmal [mailto:jt...@gm...]
> >       > Sent: 2015-05-12 2231
> >       > To: Anthony Rousseau
> >       > Cc: Kirill Katsnelson; Dan Povey; kaldi-
> >       > dev...@li...
> >       > Subject: Re: [Kaldi-developers] sph audio, sox vs sph2pipe
> >       >
> >       > Guys,  if I remember correctly,  sox does support sph,  but it
> > does
> >       > not support sph with "shorten" compression,  which is commonly
> > used in
> >       > the ldc  sph files.
> >       > Y.
> >       >
> >       > On May 13, 2015 6:57 AM, "Anthony Rousseau"
> >       > <ant...@li...> wrote:
> >       >
> >       >
> >       >       Dear Dan,
> >       >
> >       >       I think sox has been supporting the NIST Sphere format
> > for a long
> >       > time now, as I’ve been using it for years. Here is a sample
> > command
> >       > line I use to do the WAV -> SPH conversion:
> >       >       $sox -q -t wav $file.wav -c 1 -B -r 16000 -b 16 -t sph
> > $file.sph
> >       >
> >       >       Best,
> >       >
> >       >       —
> >       >       Anthony
> >       >
> >       >       > Le 13 mai 2015 à 01:16, Daniel Povey
> > <dp...@gm...> a écrit
> >       > :
> >       >       >
> >       >       > I did not realize that sox supports sphere format.
> > Maybe this
> >       > is new.
> >       >       > It would probably be a good idea to switch the scripts
> > over to
> >       > using
> >       >       > sox, since it a more standard tool.  But this may take
> > a little
> >       > while
> >       >       > to switch over, as it's in the local/ parts of the
> > example
> >       > scripts.
> >       >       > We'd also need, in that case, to add sox to the list
> > of
> >       > dependencies
> >       >       > that we check while installing Kaldi.
> >       >       > If you can test whether it works in place of sph2pipe,
> > that
> >       > would be a
> >       >       > good thing.
> >       >       >
> >       >       > Dan
> >       >       >
> >       >       >
> >       >       > On Tue, May 12, 2015 at 6:59 PM, Kirill Katsnelson
> >       >       > <kir...@sm...> wrote:
> >       >       >> Kaldi uses sph2pipe, although sox supports the .sph
> > format.
> >       > Is the use of sph2pipe just a historical artifact, or is sox'
> > support
> >       > for sphere incomplete/unreliable?
> >       >       >>
> >       >       >> I am asking because there is apparently no working
> > binary of
> >       > sph2pipe on windows, and I did not port it.
> >       >       >>
> >       >       >> -kkm
> >       >       >>
> >       >       >>
> >       >
> > -----------------------------------------------------------------------
> >       > -
> >       > ------
> >       >       >> One dashboard for servers and applications across
> > Physical-
> >       > Virtual-Cloud
> >       >       >> Widest out-of-the-box monitoring support with 50+
> > applications
> >       >       >> Performance metrics, stats and reports that give you
> >       > Actionable Insights
> >       >       >> Deep dive visibility with transaction tracing using
> > APM
> >       > Insight.
> >       >       >>
> > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> >       >       >> _______________________________________________
> >       >       >> Kaldi-developers mailing list
> >       >       >> Kal...@li...
> >       >       >>
> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >       >       >
> >       >       >
> >       >
> > -----------------------------------------------------------------------
> >       > -
> >       > ------
> >       >       > One dashboard for servers and applications across
> > Physical-
> >       > Virtual-Cloud
> >       >       > Widest out-of-the-box monitoring support with 50+
> > applications
> >       >       > Performance metrics, stats and reports that give you
> > Actionable
> >       > Insights
> >       >       > Deep dive visibility with transaction tracing using
> > APM
> >       > Insight.
> >       >       >
> > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> >       >       > _______________________________________________
> >       >       > Kaldi-developers mailing list
> >       >       > Kal...@li...
> >       >       >
> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >       >
> >       >
> >       >
> >       >
> > -----------------------------------------------------------------------
> >       > -
> >       > ------
> >       >       One dashboard for servers and applications across
> > Physical-
> >       > Virtual-Cloud
> >       >       Widest out-of-the-box monitoring support with 50+
> > applications
> >       >       Performance metrics, stats and reports that give you
> > Actionable
> >       > Insights
> >       >       Deep dive visibility with transaction tracing using APM
> > Insight.
> >       >       http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> >       >       _______________________________________________
> >       >       Kaldi-developers mailing list
> >       >       Kal...@li...
> >       >
> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >       >
> >
> >
>
>

Re: [Kaldi-developers] libtool package changed in debian

[Vijayaditya P. <p.v...@gm...>]

OK.

Vijay

On Wed, May 13, 2015 at 3:36 PM, Daniel Povey <dp...@gm...> wrote:

> I suggest you check the contents of /etc/debian_version (and of course
> that it exists) before adding libtool-bin to the dependencies.  That
> package doesn't exist pre-jessie, and it might cause the apt-get
> install command to fail if you add it.
> Dan
>
>
> On Wed, May 13, 2015 at 3:33 PM, Vijayaditya Peddinti
> <p.v...@gm...> wrote:
> > Hi,
> > In debian-jessie  the libtool package has been split. So an additional
> > package has to be installed to get the libtool binary. The package
> > libtool-bin has to be added to tools/check_dependencies.sh script.
> >
> > I do not know how this change affects other debian/ubuntu versions. Let
> me
> > know if I should go ahead and make the change to check_dependencies.sh.
> >
> > (Reference: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=761791)
> >
> > Vijay
> >
> >
> ------------------------------------------------------------------------------
> > One dashboard for servers and applications across Physical-Virtual-Cloud
> > Widest out-of-the-box monitoring support with 50+ applications
> > Performance metrics, stats and reports that give you Actionable Insights
> > Deep dive visibility with transaction tracing using APM Insight.
> > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> > _______________________________________________
> > Kaldi-developers mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >
>

Re: [Kaldi-developers] libtool package changed in debian

[Daniel P. <dp...@gm...>]

I suggest you check the contents of /etc/debian_version (and of course
that it exists) before adding libtool-bin to the dependencies.  That
package doesn't exist pre-jessie, and it might cause the apt-get
install command to fail if you add it.
Dan


On Wed, May 13, 2015 at 3:33 PM, Vijayaditya Peddinti
<p.v...@gm...> wrote:
> Hi,
> In debian-jessie  the libtool package has been split. So an additional
> package has to be installed to get the libtool binary. The package
> libtool-bin has to be added to tools/check_dependencies.sh script.
>
> I do not know how this change affects other debian/ubuntu versions. Let me
> know if I should go ahead and make the change to check_dependencies.sh.
>
> (Reference: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=761791)
>
> Vijay
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>

[Kaldi-developers] libtool package changed in debian

[Vijayaditya P. <p.v...@gm...>]

Hi,
In debian-jessie  the libtool package has been split. So an additional
package has to be installed to get the libtool binary. The package
libtool-bin has to be added to tools/check_dependencies.sh script.

I do not know how this change affects other debian/ubuntu versions. Let me
know if I should go ahead and make the change to check_dependencies.sh.

(Reference: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=761791)

Vijay

Re: [Kaldi-developers] sph audio, sox vs sph2pipe

[Kirill K. <kir...@sm...>]

shorten(1) is very outdated, so there would be no benefit adopting it over nist sphere tools. Current programs that support .shn files and the shorten compression are ffmpeg/avconv, but I doubt they can directly uncompress a shorten'd .sph. It all gets complicated quickly.

But using sox on uncompressed sets like TEDlium is probably more portable.

If anyone has a sample shorten-compressed .sph file, send it my way please.

 -kkm

> -----Original Message-----
> From: Jan Trmal [mailto:jt...@gm...]
> Sent: 2015-05-13 0315
> To: Kirill Katsnelson
> Cc: Anthony Rousseau; Dan Povey; kal...@li...
> Subject: RE: [Kaldi-developers] sph audio, sox vs sph2pipe
> 
> I remember I tried several times and I had the problems with "shorten".
> 
> Sox refuses to work with those sph files.
> 
> What I did IIRC, I took shorten.exe (I don't remember where I got it,
> but something can be found here:
> http://www.rarewares.org/lossless.php),
> which does not support sph format -- but has some switch that allows to
> ignore first 1024 bytes of the recording (or something like that). That
> binary produced either A/mu-law or 16bit-linear raw audio, which I
> processed using sox as usuall.
> 
> HTH
> 
> y.
> 
> 
> 
> 
> On May 13, 2015 7:40 AM, "Kirill Katsnelson"
> <kir...@sm...> wrote:
> 
> 
> 	It very likely does not. I did not know it was common though.
> AFAIK, it just outputs a compressed stream, but I have no way to test.
> 
> 	Can it be piped to ffmpeg or avconv? If I could get hold of a
> compressed file example, I'd test.
> 
> 	 -kkm
> 
> 	> -----Original Message-----
> 	> From: Jan Trmal [mailto:jt...@gm...]
> 	> Sent: 2015-05-12 2231
> 	> To: Anthony Rousseau
> 	> Cc: Kirill Katsnelson; Dan Povey; kaldi-
> 	> dev...@li...
> 	> Subject: Re: [Kaldi-developers] sph audio, sox vs sph2pipe
> 	>
> 	> Guys,  if I remember correctly,  sox does support sph,  but it
> does
> 	> not support sph with "shorten" compression,  which is commonly
> used in
> 	> the ldc  sph files.
> 	> Y.
> 	>
> 	> On May 13, 2015 6:57 AM, "Anthony Rousseau"
> 	> <ant...@li...> wrote:
> 	>
> 	>
> 	>       Dear Dan,
> 	>
> 	>       I think sox has been supporting the NIST Sphere format
> for a long
> 	> time now, as I’ve been using it for years. Here is a sample
> command
> 	> line I use to do the WAV -> SPH conversion:
> 	>       $sox -q -t wav $file.wav -c 1 -B -r 16000 -b 16 -t sph
> $file.sph
> 	>
> 	>       Best,
> 	>
> 	>       —
> 	>       Anthony
> 	>
> 	>       > Le 13 mai 2015 à 01:16, Daniel Povey
> <dp...@gm...> a écrit
> 	> :
> 	>       >
> 	>       > I did not realize that sox supports sphere format.
> Maybe this
> 	> is new.
> 	>       > It would probably be a good idea to switch the scripts
> over to
> 	> using
> 	>       > sox, since it a more standard tool.  But this may take
> a little
> 	> while
> 	>       > to switch over, as it's in the local/ parts of the
> example
> 	> scripts.
> 	>       > We'd also need, in that case, to add sox to the list
> of
> 	> dependencies
> 	>       > that we check while installing Kaldi.
> 	>       > If you can test whether it works in place of sph2pipe,
> that
> 	> would be a
> 	>       > good thing.
> 	>       >
> 	>       > Dan
> 	>       >
> 	>       >
> 	>       > On Tue, May 12, 2015 at 6:59 PM, Kirill Katsnelson
> 	>       > <kir...@sm...> wrote:
> 	>       >> Kaldi uses sph2pipe, although sox supports the .sph
> format.
> 	> Is the use of sph2pipe just a historical artifact, or is sox'
> support
> 	> for sphere incomplete/unreliable?
> 	>       >>
> 	>       >> I am asking because there is apparently no working
> binary of
> 	> sph2pipe on windows, and I did not port it.
> 	>       >>
> 	>       >> -kkm
> 	>       >>
> 	>       >>
> 	>
> -----------------------------------------------------------------------
> 	> -
> 	> ------
> 	>       >> One dashboard for servers and applications across
> Physical-
> 	> Virtual-Cloud
> 	>       >> Widest out-of-the-box monitoring support with 50+
> applications
> 	>       >> Performance metrics, stats and reports that give you
> 	> Actionable Insights
> 	>       >> Deep dive visibility with transaction tracing using
> APM
> 	> Insight.
> 	>       >>
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> 	>       >> _______________________________________________
> 	>       >> Kaldi-developers mailing list
> 	>       >> Kal...@li...
> 	>       >>
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 	>       >
> 	>       >
> 	>
> -----------------------------------------------------------------------
> 	> -
> 	> ------
> 	>       > One dashboard for servers and applications across
> Physical-
> 	> Virtual-Cloud
> 	>       > Widest out-of-the-box monitoring support with 50+
> applications
> 	>       > Performance metrics, stats and reports that give you
> Actionable
> 	> Insights
> 	>       > Deep dive visibility with transaction tracing using
> APM
> 	> Insight.
> 	>       >
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> 	>       > _______________________________________________
> 	>       > Kaldi-developers mailing list
> 	>       > Kal...@li...
> 	>       >
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 	>
> 	>
> 	>
> 	>
> -----------------------------------------------------------------------
> 	> -
> 	> ------
> 	>       One dashboard for servers and applications across
> Physical-
> 	> Virtual-Cloud
> 	>       Widest out-of-the-box monitoring support with 50+
> applications
> 	>       Performance metrics, stats and reports that give you
> Actionable
> 	> Insights
> 	>       Deep dive visibility with transaction tracing using APM
> Insight.
> 	>       http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> 	>       _______________________________________________
> 	>       Kaldi-developers mailing list
> 	>       Kal...@li...
> 	>
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 	>
> 
> 

Re: [Kaldi-developers] sph audio, sox vs sph2pipe

[Jan T. <jt...@gm...>]

I remember I tried several times and I had the problems with "shorten".

Sox refuses to work with those sph files.

What I did IIRC, I took shorten.exe (I don't remember where I got it, but
something can be found here: http://www.rarewares.org/lossless.php), which
does not support sph format -- but has some switch that allows to ignore
first 1024 bytes of the recording (or something like that). That binary
produced either A/mu-law or 16bit-linear raw audio, which I processed using
sox as usuall.

HTH

y.


On May 13, 2015 7:40 AM, "Kirill Katsnelson" <
kir...@sm...> wrote:

> It very likely does not. I did not know it was common though. AFAIK, it
> just outputs a compressed stream, but I have no way to test.
>
> Can it be piped to ffmpeg or avconv? If I could get hold of a compressed
> file example, I'd test.
>
>  -kkm
>
> > -----Original Message-----
> > From: Jan Trmal [mailto:jt...@gm...]
> > Sent: 2015-05-12 2231
> > To: Anthony Rousseau
> > Cc: Kirill Katsnelson; Dan Povey; kaldi-
> > dev...@li...
> > Subject: Re: [Kaldi-developers] sph audio, sox vs sph2pipe
> >
> > Guys,  if I remember correctly,  sox does support sph,  but it does
> > not support sph with "shorten" compression,  which is commonly used in
> > the ldc  sph files.
> > Y.
> >
> > On May 13, 2015 6:57 AM, "Anthony Rousseau"
> > <ant...@li...> wrote:
> >
> >
> >       Dear Dan,
> >
> >       I think sox has been supporting the NIST Sphere format for a long
> > time now, as I’ve been using it for years. Here is a sample command
> > line I use to do the WAV -> SPH conversion:
> >       $sox -q -t wav $file.wav -c 1 -B -r 16000 -b 16 -t sph $file.sph
> >
> >       Best,
> >
> >       —
> >       Anthony
> >
> >       > Le 13 mai 2015 à 01:16, Daniel Povey <dp...@gm...> a écrit
> > :
> >       >
> >       > I did not realize that sox supports sphere format.  Maybe this
> > is new.
> >       > It would probably be a good idea to switch the scripts over to
> > using
> >       > sox, since it a more standard tool.  But this may take a little
> > while
> >       > to switch over, as it's in the local/ parts of the example
> > scripts.
> >       > We'd also need, in that case, to add sox to the list of
> > dependencies
> >       > that we check while installing Kaldi.
> >       > If you can test whether it works in place of sph2pipe, that
> > would be a
> >       > good thing.
> >       >
> >       > Dan
> >       >
> >       >
> >       > On Tue, May 12, 2015 at 6:59 PM, Kirill Katsnelson
> >       > <kir...@sm...> wrote:
> >       >> Kaldi uses sph2pipe, although sox supports the .sph format.
> > Is the use of sph2pipe just a historical artifact, or is sox' support
> > for sphere incomplete/unreliable?
> >       >>
> >       >> I am asking because there is apparently no working binary of
> > sph2pipe on windows, and I did not port it.
> >       >>
> >       >> -kkm
> >       >>
> >       >>
> > -----------------------------------------------------------------------
> > -
> > ------
> >       >> One dashboard for servers and applications across Physical-
> > Virtual-Cloud
> >       >> Widest out-of-the-box monitoring support with 50+ applications
> >       >> Performance metrics, stats and reports that give you
> > Actionable Insights
> >       >> Deep dive visibility with transaction tracing using APM
> > Insight.
> >       >> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> >       >> _______________________________________________
> >       >> Kaldi-developers mailing list
> >       >> Kal...@li...
> >       >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >       >
> >       >
> > -----------------------------------------------------------------------
> > -
> > ------
> >       > One dashboard for servers and applications across Physical-
> > Virtual-Cloud
> >       > Widest out-of-the-box monitoring support with 50+ applications
> >       > Performance metrics, stats and reports that give you Actionable
> > Insights
> >       > Deep dive visibility with transaction tracing using APM
> > Insight.
> >       > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> >       > _______________________________________________
> >       > Kaldi-developers mailing list
> >       > Kal...@li...
> >       > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >
> >
> >
> > -----------------------------------------------------------------------
> > -
> > ------
> >       One dashboard for servers and applications across Physical-
> > Virtual-Cloud
> >       Widest out-of-the-box monitoring support with 50+ applications
> >       Performance metrics, stats and reports that give you Actionable
> > Insights
> >       Deep dive visibility with transaction tracing using APM Insight.
> >       http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> >       _______________________________________________
> >       Kaldi-developers mailing list
> >       Kal...@li...
> >       https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >
>
>

Re: [Kaldi-developers] sph audio, sox vs sph2pipe

[Kirill K. <kir...@sm...>]

It very likely does not. I did not know it was common though. AFAIK, it just outputs a compressed stream, but I have no way to test. 

Can it be piped to ffmpeg or avconv? If I could get hold of a compressed file example, I'd test.

 -kkm

> -----Original Message-----
> From: Jan Trmal [mailto:jt...@gm...]
> Sent: 2015-05-12 2231
> To: Anthony Rousseau
> Cc: Kirill Katsnelson; Dan Povey; kaldi-
> dev...@li...
> Subject: Re: [Kaldi-developers] sph audio, sox vs sph2pipe
> 
> Guys,  if I remember correctly,  sox does support sph,  but it does
> not support sph with "shorten" compression,  which is commonly used in
> the ldc  sph files.
> Y.
> 
> On May 13, 2015 6:57 AM, "Anthony Rousseau"
> <ant...@li...> wrote:
> 
> 
> 	Dear Dan,
> 
> 	I think sox has been supporting the NIST Sphere format for a long
> time now, as I’ve been using it for years. Here is a sample command
> line I use to do the WAV -> SPH conversion:
> 	$sox -q -t wav $file.wav -c 1 -B -r 16000 -b 16 -t sph $file.sph
> 
> 	Best,
> 
> 	—
> 	Anthony
> 
> 	> Le 13 mai 2015 à 01:16, Daniel Povey <dp...@gm...> a écrit
> :
> 	>
> 	> I did not realize that sox supports sphere format.  Maybe this
> is new.
> 	> It would probably be a good idea to switch the scripts over to
> using
> 	> sox, since it a more standard tool.  But this may take a little
> while
> 	> to switch over, as it's in the local/ parts of the example
> scripts.
> 	> We'd also need, in that case, to add sox to the list of
> dependencies
> 	> that we check while installing Kaldi.
> 	> If you can test whether it works in place of sph2pipe, that
> would be a
> 	> good thing.
> 	>
> 	> Dan
> 	>
> 	>
> 	> On Tue, May 12, 2015 at 6:59 PM, Kirill Katsnelson
> 	> <kir...@sm...> wrote:
> 	>> Kaldi uses sph2pipe, although sox supports the .sph format.
> Is the use of sph2pipe just a historical artifact, or is sox' support
> for sphere incomplete/unreliable?
> 	>>
> 	>> I am asking because there is apparently no working binary of
> sph2pipe on windows, and I did not port it.
> 	>>
> 	>> -kkm
> 	>>
> 	>>
> -----------------------------------------------------------------------
> -
> ------
> 	>> One dashboard for servers and applications across Physical-
> Virtual-Cloud
> 	>> Widest out-of-the-box monitoring support with 50+ applications
> 	>> Performance metrics, stats and reports that give you
> Actionable Insights
> 	>> Deep dive visibility with transaction tracing using APM
> Insight.
> 	>> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> 	>> _______________________________________________
> 	>> Kaldi-developers mailing list
> 	>> Kal...@li...
> 	>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 	>
> 	>
> -----------------------------------------------------------------------
> -
> ------
> 	> One dashboard for servers and applications across Physical-
> Virtual-Cloud
> 	> Widest out-of-the-box monitoring support with 50+ applications
> 	> Performance metrics, stats and reports that give you Actionable
> Insights
> 	> Deep dive visibility with transaction tracing using APM
> Insight.
> 	> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> 	> _______________________________________________
> 	> Kaldi-developers mailing list
> 	> Kal...@li...
> 	> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 
> 
> 
> -----------------------------------------------------------------------
> -
> ------
> 	One dashboard for servers and applications across Physical-
> Virtual-Cloud
> 	Widest out-of-the-box monitoring support with 50+ applications
> 	Performance metrics, stats and reports that give you Actionable
> Insights
> 	Deep dive visibility with transaction tracing using APM Insight.
> 	http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> 	_______________________________________________
> 	Kaldi-developers mailing list
> 	Kal...@li...
> 	https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 

Re: [Kaldi-developers] sph audio, sox vs sph2pipe

[Jan T. <jt...@gm...>]

Guys,  if I remember correctly,  sox does support sph,  but it does  not
support sph with "shorten" compression,  which is commonly used in the ldc
sph files.
Y.
On May 13, 2015 6:57 AM, "Anthony Rousseau" <
ant...@li...> wrote:

> Dear Dan,
>
> I think sox has been supporting the NIST Sphere format for a long time
> now, as I’ve been using it for years. Here is a sample command line I use
> to do the WAV -> SPH conversion:
> $sox -q -t wav $file.wav -c 1 -B -r 16000 -b 16 -t sph $file.sph
>
> Best,
>
> —
> Anthony
>
> > Le 13 mai 2015 à 01:16, Daniel Povey <dp...@gm...> a écrit :
> >
> > I did not realize that sox supports sphere format.  Maybe this is new.
> > It would probably be a good idea to switch the scripts over to using
> > sox, since it a more standard tool.  But this may take a little while
> > to switch over, as it's in the local/ parts of the example scripts.
> > We'd also need, in that case, to add sox to the list of dependencies
> > that we check while installing Kaldi.
> > If you can test whether it works in place of sph2pipe, that would be a
> > good thing.
> >
> > Dan
> >
> >
> > On Tue, May 12, 2015 at 6:59 PM, Kirill Katsnelson
> > <kir...@sm...> wrote:
> >> Kaldi uses sph2pipe, although sox supports the .sph format. Is the use
> of sph2pipe just a historical artifact, or is sox' support for sphere
> incomplete/unreliable?
> >>
> >> I am asking because there is apparently no working binary of sph2pipe
> on windows, and I did not port it.
> >>
> >> -kkm
> >>
> >>
> ------------------------------------------------------------------------------
> >> One dashboard for servers and applications across Physical-Virtual-Cloud
> >> Widest out-of-the-box monitoring support with 50+ applications
> >> Performance metrics, stats and reports that give you Actionable Insights
> >> Deep dive visibility with transaction tracing using APM Insight.
> >> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> >> _______________________________________________
> >> Kaldi-developers mailing list
> >> Kal...@li...
> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >
> >
> ------------------------------------------------------------------------------
> > One dashboard for servers and applications across Physical-Virtual-Cloud
> > Widest out-of-the-box monitoring support with 50+ applications
> > Performance metrics, stats and reports that give you Actionable Insights
> > Deep dive visibility with transaction tracing using APM Insight.
> > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> > _______________________________________________
> > Kaldi-developers mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>
>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>

Re: [Kaldi-developers] sph audio, sox vs sph2pipe

[Anthony R. <ant...@li...>]

Dear Dan,

I think sox has been supporting the NIST Sphere format for a long time now, as I’ve been using it for years. Here is a sample command line I use to do the WAV -> SPH conversion:
$sox -q -t wav $file.wav -c 1 -B -r 16000 -b 16 -t sph $file.sph

Best,

—
Anthony

> Le 13 mai 2015 à 01:16, Daniel Povey <dp...@gm...> a écrit :
> 
> I did not realize that sox supports sphere format.  Maybe this is new.
> It would probably be a good idea to switch the scripts over to using
> sox, since it a more standard tool.  But this may take a little while
> to switch over, as it's in the local/ parts of the example scripts.
> We'd also need, in that case, to add sox to the list of dependencies
> that we check while installing Kaldi.
> If you can test whether it works in place of sph2pipe, that would be a
> good thing.
> 
> Dan
> 
> 
> On Tue, May 12, 2015 at 6:59 PM, Kirill Katsnelson
> <kir...@sm...> wrote:
>> Kaldi uses sph2pipe, although sox supports the .sph format. Is the use of sph2pipe just a historical artifact, or is sox' support for sphere incomplete/unreliable?
>> 
>> I am asking because there is apparently no working binary of sph2pipe on windows, and I did not port it.
>> 
>> -kkm
>> 
>> ------------------------------------------------------------------------------
>> One dashboard for servers and applications across Physical-Virtual-Cloud
>> Widest out-of-the-box monitoring support with 50+ applications
>> Performance metrics, stats and reports that give you Actionable Insights
>> Deep dive visibility with transaction tracing using APM Insight.
>> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
>> _______________________________________________
>> Kaldi-developers mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud 
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers

Re: [Kaldi-developers] sph audio, sox vs sph2pipe

[Kirill K. <kir...@sm...>]

> From: Daniel Povey [mailto:dp...@gm...]
> Sent: 2015-05-12 1616
> If you can test whether [sox] works in place of sph2pipe, that would be a
> good thing.

It does. In fact I am using sox to do rate conversion and filtering on the fly. Being also available in cygwin, it was the only tool I could use for TEDlium on windows.

 -kkm

Re: [Kaldi-developers] sph audio, sox vs sph2pipe

[Daniel P. <dp...@gm...>]

I did not realize that sox supports sphere format.  Maybe this is new.
It would probably be a good idea to switch the scripts over to using
sox, since it a more standard tool.  But this may take a little while
to switch over, as it's in the local/ parts of the example scripts.
We'd also need, in that case, to add sox to the list of dependencies
that we check while installing Kaldi.
If you can test whether it works in place of sph2pipe, that would be a
good thing.

Dan


On Tue, May 12, 2015 at 6:59 PM, Kirill Katsnelson
<kir...@sm...> wrote:
> Kaldi uses sph2pipe, although sox supports the .sph format. Is the use of sph2pipe just a historical artifact, or is sox' support for sphere incomplete/unreliable?
>
> I am asking because there is apparently no working binary of sph2pipe on windows, and I did not port it.
>
>  -kkm
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers

[Kaldi-developers] sph audio, sox vs sph2pipe

[Kirill K. <kir...@sm...>]

Kaldi uses sph2pipe, although sox supports the .sph format. Is the use of sph2pipe just a historical artifact, or is sox' support for sphere incomplete/unreliable?

I am asking because there is apparently no working binary of sph2pipe on windows, and I did not port it.

 -kkm

Re: [Kaldi-developers] [Kaldi-users] Kaldi compiles now under VS 2013

[Jan T. <jt...@gm...>]

[Removing the kaldi-users@ and kaldi-developers@] via BCC

Kirill, I will review the github howto you've created and we can talk about
details afterwards...

In the meantime, please try to merge the  HEAD of kaldi repository (feel
free to use the github mirror), so that we can review the changes.

Thanks for all your work!
y.

On Tue, May 12, 2015 at 6:49 AM, Kirill Katsnelson <
kir...@sm...> wrote:

> Thanks, an interesting find. I missed it, I believe. You are most probably
> correct and that section should apply to the type resolution too. Full
> support for C++11 is still not out there; to Microsoft's defense, the value
> of the __cplusplus macro does not indicate a C++11 compliance. Which is to
> make our lives even harder indeed... Intel's compiler sided with gcc on
> this point, IIRC. And Microsoft is notorious for breaking standards in
> many, well, interesting ways, no doubt about that.
>
> FWIW, I got a word from Intel's support today that the bug in the MSBuild
> integration I reported together with a proposed fix was released. I have
> not tested it yet though.
>
>  -kkm
>
> > -----Original Message-----
> > From: Daniel Povey [mailto:dp...@gm...]
> > Sent: 2015-05-11 1914
> > To: Kirill Katsnelson
> > Cc: Jan Trmal; kal...@li...; kaldi-
> > dev...@li...; 洪孝宗
> > Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
> >
> > BTW, this is one of many bugs that I specifically pointed out to the
> > Visual Studio people while at Microsoft (and I looked it up just now
> > because I remembered checking the standard before, so I knew it was
> > there).  Their attitude was that compliance with the standard doesn't
> > really matter; what really matters is to not break the Windows build.
> > Eventually I think I gained a bad reputation in that circle as not
> > being sufficiently "with the program".  Anyway, I'm willing to patch
> > Kaldi to support their broken compiler as far as is reasonable, but it
> > doesn't make me very happy.
> >
> > Dan
> >
> >
> > On Mon, May 11, 2015 at 7:06 PM, Daniel Povey <dp...@gm...> wrote:
> > > It isn't a gray area in the standard; see
> > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4296.pdf
> > > the last paragraph on page 367:
> > >   If a name does not depend on a template-parameter (as defined in
> > > 14.6.2), a declaration (or set of declarations) for that name shall
> > be
> > > in scope at the point where the name appears in the template
> > > definition; the name is bound to the declaration (or declarations)
> > > found at that point and this binding is not affected by declarations
> > > that are visible at the point of instantiation.
> > >
> > > Dan
> > >
> > >
> > > On Mon, May 11, 2015 at 2:50 PM, Kirill Katsnelson
> > > <kir...@sm...> wrote:
> > >> I thought it was but this really appears to be a grey area in the
> > standard. But indeed, from the practical standpoint, I think the best
> > thing to do is just make all compilers happy. I can figure out if this
> > is really undefined C++ later, if there is a theoretical interest.
> > >>
> > >>  -kkm
> > >>
> > >>> -----Original Message-----
> > >>> From: Daniel Povey [mailto:dp...@gm...]
> > >>> Sent: 2015-05-11 1444
> > >>> To: Jan Trmal
> > >>> Cc: Kirill Katsnelson; kal...@li...; kaldi-
> > >>> dev...@li...; 洪孝宗
> > >>> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
> > >>>
> > >>> OK fine.
> > >>> I'm pretty sure this is a bug in MSVC, that the namespace lookup
> > for
> > >>> a templated function happens in the context in which the template
> > is
> > >>> evaluated (as opposed to where it was defined).  But it makes sense
> > >>> to patch it anyway.
> > >>> Dan
> > >>>
> > >>>
> > >>> On Mon, May 11, 2015 at 2:09 PM, Jan Trmal <jt...@gm...>
> > wrote:
> > >>> > Ok, I will have a.look tmrw. I think I've resolved the types
> > issue
> > >>> > (last one you mentioned). I generated the patch for openfst, that
> > >>> > defines the int types within the openfst namespace - originally
> > >>> > they exist on.the highest scope (i.e. ::int32) The patch has
> > >>> > something
> > >>> like
> > >>> > namespace fst {
> > >>> >   typedef int32 ::int32;
> > >>> > }
> > >>> >
> > >>> > Doping this, the lookup works fine even under the ms c++.
> > >>> > To me, this is optimal solution as we have to patch the openfst
> > >>> anyway
> > >>> > (and my change wont break any code).
> > >>> > You can.have a look ať the openfstwin-1.3.3.patch in
> > tools/extras.
> > >>> > I am open to any suggestion or general discussion on how this
> > >>> > should be made better.
> > >>> > Y.
> > >>> >
> > >>> > On May 11, 2015 9:03 PM, "Daniel Povey" <dp...@gm...> wrote:
> > >>> >>
> > >>> >> > Well, the build was not "painless" at all. In fact, the perl
> > >>> script
> > >>> >> > generated a solution with 600+ projects that simply failed to
> > >>> >> > build. VS was churning out dependencies on these, and crashed
> > >>> >> > in about 10 minutes of just sitting there with a 100% CPU use.
> > >>> >> > Not a single file was compiled, so I think it was just
> > >>> >> > autogenerating
> > >>> dependencies at this point.
> > >>> >>
> > >>> >> Build systems on Windows are pretty broken.  I think in
> > Microsoft
> > >>> >> they use some other system, not the one used in VS, which is not
> > >>> >> publicly released.
> > >>> >>
> > >>> >> > I did not try to build from the command line though. Am I
> > >>> >> > understanding it would compile?
> > >>> >>
> > >>> >> No, not everything would compile.  Sometimes the VS compiler
> > just
> > >>> crashes.
> > >>> >>
> > >>> >> > What I did not like about the existing system is that I could
> > >>> >> > not plug in CUDA or intel compiler. It is extremely rigid in
> > >>> >> > the
> > >>> option
> > >>> >> > part, and to change compiler options you are most likely to
> > >>> >> > change the perl code that generates the project files, or
> > >>> >> > modify the different MSBuild include files (called "property
> > >>> >> > sheets" in this
> > >>> >> > context) that are interplaying quite non-trivially. No, I have
> > >>> >> > spent a few days on the build system not for a love of
> > >>> >> > tinkering with MSBuils at all! :)
> > >>> >>
> > >>> >> What we committed was just a hack to try to get something
> > working.
> > >>> >> IIRC we just looked at the build files generated by VS to try to
> > >>> >> guess what the format of those files was supposed to be; I'm not
> > >>> sure
> > >>> >> that they are really intended to be human readable/writable
> > >>> >> (unless that human really loves XML).
> > >>> >>
> > >>> >> > Anyway, you can look at it at
> > >>> >> > https://github.com/kkm000/kaldi/tree/winbuild/msbuild. (NB I
> > >>> >> > have not updated the branch in a while) If you think this is
> > >>> >> > not an acceptable solution, I'm just dropping the ball, no
> > >>> >> > offence taken, naturally. I can maintain this as an
> > "unofficial
> > >>> >> > port" for a
> > >>> while.
> > >>> >> > In any case, build is the least problem, as Dan has pointed
> > out
> > >>> already...
> > >>> >> >
> > >>> >> > The biggest (in the number of changed lines) changes I had to
> > >>> >> > make were in the namespace using declarations, because of
> > >>> >> > conflicts between the types
> > >>> >> > int32 and friends from different namespaces. I ended up
> > reading
> > >>> the
> > >>> >> > C++ standard to understand whether gcc, icl or msvc is
> > "correct"
> > >>> in
> > >>> >> > different cases they are complaining about, and the standard
> > >>> >> > appears not to define any behavior there. These changes I
> > would
> > >>> >> > prefer accepted into the mainline of development, because they
> > >>> >> > are
> > >>> harder to maintain in a fork.
> > >>> >>
> > >>> >> In any case where you have to do something like typedef
> > >>> >> kaldi::int32 int32, we should definitely get this committed-
> > talk to Yenda.
> > >>> >> Actually, this is really a bug in OpenFST which they haven't
> > >>> >> fixed yet.  There is a "proper" POSIX way to get types like
> > >>> >> int32, which
> > >>> we
> > >>> >> use in Kaldi, but in OpenFst they do it in a different, ad-hoc
> > >>> >> way which sometimes generates different types (e.g. on a system
> > >>> >> where
> > >>> int
> > >>> >> and long int are both 32-bit).  So the Kaldi int32 and the
> > >>> >> OpenFst
> > >>> >> int32 end up being different, incompatible types.
> > >>> >>
> > >>> >> Dan
> > >>> >>
> > >>> >>
> > >>> >> >> -----Original Message-----
> > >>> >> >> From: Jan Trmal [mailto:jt...@gm...]
> > >>> >> >> Sent: 2015-05-11 0201
> > >>> >> >> To: Dan Povey
> > >>> >> >> Cc: Kirill Katsnelson; kaldi-
> > dev...@li...;
> > >>> >> >> kaldi- us...@li...; 洪孝宗
> > >>> >> >> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
> > >>> >> >>
> > >>> >> >> Guys, the migration to git is "unofficially" done -- unless
> > we
> > >>> >> >> find some problems with it, the repo
> > >>> >> >> (https://github.com/kaldi-
> > >>> >> >> asr/kaldi.git) will stay as it is now.
> > >>> >> >>
> > >>> >> >> Kirill,  there is no reason why wait. We should start working
> > >>> >> >> on
> > >>> it.
> > >>> >> >> I propose you could first send the Kaldi code patches -- I
> > >>> >> >> actually worked on this and I'm able to compile Kaldi, so
> > this
> > >>> >> >> should be quite straightforward, provided you will generate
> > >>> >> >> the
> > >>> diff against the HEAD.
> > >>> >> >>
> > >>> >> >> Then we can start looking at the  "build system". I
> > understand
> > >>> >> >> your reasons and I think they might be legit. My feeling is,
> > >>> >> >> however, that a lot of people are not using MSBuild and will
> > >>> >> >> expect the MSVC solution, just because this is how they
> > >>> >> >> usually work. I haven't tested it, but you should be able to
> > >>> >> >> use the current SLN to build the binaries from command line -
> > -
> > >>> >> >> that
> > >>> should
> > >>> >> >> be completely painless -- but again, I haven't tested it.
> > >>> >> >> y.
> > >>> >> >>
> > >>> >> >>
> > >>> >> >> On Thu, May 7, 2015 at 10:53 PM, Daniel Povey
> > >>> >> >> <dp...@gm...>
> > >>> wrote:
> > >>> >> >>
> > >>> >> >>
> > >>> >> >>       >> To the extent that your patches are simple fixes and
> > >>> >> >> minor changes to
> > >>> >> >>       >> the code, I think we could apply them.  Perhaps you
> > >>> could
> > >>> >> >> work with
> > >>> >> >>       >> Yenda to get your changes checked?
> > >>> >> >>       >
> > >>> >> >>       > Easy enough, and as easy to hold till the migration
> > is
> > >>> done.
> > >>> >> >> Was there a change in the plans?
> > >>> >> >>
> > >>> >> >>       Not really a change in the plans, but we don't have a
> > >>> definite
> > >>> >> >>       timeline for migrating to git, and it could be that
> > >>> >> >> we'll keep the svn
> > >>> >> >>       as upstream for a while.  Your changes seem reasonable,
> > >>> >> >> but I think it
> > >>> >> >>       would be better to get things started now rather than
> > >>> later.
> > >>> >> >> The
> > >>> >> >>       git/svn issues should not be that hard.  Just start
> > >>> >> >> sending patches to
> > >>> >> >>       Yenda and we'll handle your changes bit by bit.
> > >>> >> >>       Dan
> > >>> >> >>
> > >>> >> >>
> > >>> >> >>
> > >>> >> >>       >
> > >>> >> >>       >> If you update your git repo to
> > >>> >> >>       >> point to the current code, then a patch should be
> > >>> >> >> applicable by the
> > >>> >> >>       >> program "patch" to the svn repo too.
> > >>> >> >>       >
> > >>> >> >>       > There are many changes, and I certainly want to split
> > >>> >> >> the patch so it is readable. One megapatch is not reviewable.
> > >>> >> >>       >
> > >>> >> >>       >> Regarding your build approach, you could send me and
> > >>> >> >> Yenda some details
> > >>> >> >>       >> about how you did it, and we could consider whether
> > >>> >> >> to support that.
> > >>> >> >>       >
> > >>> >> >>       > There's a Powershell script that takes information
> > >>> >> >> from every src/*/Makefile into a simple declarative MSBuild
> > >>> >> >> script $dirname.kwproj, and a top-level MSBuild "makefile"
> > >>> >> >> that drives the whole thing. A little bit more complex than
> > >>> >> >> that to allow for
> > >>> >> >> user- defined options, but generally this is it. This
> > supports
> > >>> >> >> building libraries, tools, tests and running the tests with
> > >>> >> >> command line switches. The whole thing pretty much mimics a
> > >>> >> >> linux make process but using MS tools.
> > >>> >> >>       >
> > >>> >> >>       >> I don't think it makes sense to try to maintain a
> > >>> >> >> parallel
> > >>> >> >> "windows-
> > >>> >> >>       >> compatible" version of the scripts, if there are
> > >>> >> >> larger changes
> > >>> >> >>       >> required there.
> > >>> >> >>       >
> > >>> >> >>       > I was targeting for no changes at all. There maybe a
> > >>> >> >> few very small patches that supposed not to break
> > compatibility.
> > >>> >> >>       >
> > >>> >> >>       >> Anyway you depend on cygwin for scripting, and the
> > set
> > >>> >> >>       >> of people who want to run cygwin and build with MSVC
> > >>> >> >> is probably
> > >>> >> >>       >> limited enough that I don't think it makes sense for
> > >>> >> >> us to try to
> > >>> >> >>       >> support it.
> > >>> >> >>       >
> > >>> >> >>       > This is not as simple a choice as it seems at first.
> > >>> >> >> Alex Hung just posted a trick to build CUDA under Cygwin;
> > >>> >> >> before he
> > >>> did
> > >>> >> >> I thought it was not possible. MKL is another story. I would
> > >>> >> >> rather build under native Windows than try to pull this
> > >>> >> >> monster into the Cygwin build environment.
> > >>> >> >>       >
> > >>> >> >>       >  -kkm
> > >>> >> >>       >
> > >>> >> >>
> > >>> >> >>
> > >>> >> >
>

Re: [Kaldi-developers] [Kaldi-users] Kaldi compiles now under VS 2013

[Kirill K. <kir...@sm...>]

Thanks, an interesting find. I missed it, I believe. You are most probably correct and that section should apply to the type resolution too. Full support for C++11 is still not out there; to Microsoft's defense, the value of the __cplusplus macro does not indicate a C++11 compliance. Which is to make our lives even harder indeed... Intel's compiler sided with gcc on this point, IIRC. And Microsoft is notorious for breaking standards in many, well, interesting ways, no doubt about that.

FWIW, I got a word from Intel's support today that the bug in the MSBuild integration I reported together with a proposed fix was released. I have not tested it yet though.

 -kkm

> -----Original Message-----
> From: Daniel Povey [mailto:dp...@gm...]
> Sent: 2015-05-11 1914
> To: Kirill Katsnelson
> Cc: Jan Trmal; kal...@li...; kaldi-
> dev...@li...; 洪孝宗
> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
> 
> BTW, this is one of many bugs that I specifically pointed out to the
> Visual Studio people while at Microsoft (and I looked it up just now
> because I remembered checking the standard before, so I knew it was
> there).  Their attitude was that compliance with the standard doesn't
> really matter; what really matters is to not break the Windows build.
> Eventually I think I gained a bad reputation in that circle as not
> being sufficiently "with the program".  Anyway, I'm willing to patch
> Kaldi to support their broken compiler as far as is reasonable, but it
> doesn't make me very happy.
> 
> Dan
> 
> 
> On Mon, May 11, 2015 at 7:06 PM, Daniel Povey <dp...@gm...> wrote:
> > It isn't a gray area in the standard; see
> > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4296.pdf
> > the last paragraph on page 367:
> >   If a name does not depend on a template-parameter (as defined in
> > 14.6.2), a declaration (or set of declarations) for that name shall
> be
> > in scope at the point where the name appears in the template
> > definition; the name is bound to the declaration (or declarations)
> > found at that point and this binding is not affected by declarations
> > that are visible at the point of instantiation.
> >
> > Dan
> >
> >
> > On Mon, May 11, 2015 at 2:50 PM, Kirill Katsnelson
> > <kir...@sm...> wrote:
> >> I thought it was but this really appears to be a grey area in the
> standard. But indeed, from the practical standpoint, I think the best
> thing to do is just make all compilers happy. I can figure out if this
> is really undefined C++ later, if there is a theoretical interest.
> >>
> >>  -kkm
> >>
> >>> -----Original Message-----
> >>> From: Daniel Povey [mailto:dp...@gm...]
> >>> Sent: 2015-05-11 1444
> >>> To: Jan Trmal
> >>> Cc: Kirill Katsnelson; kal...@li...; kaldi-
> >>> dev...@li...; 洪孝宗
> >>> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
> >>>
> >>> OK fine.
> >>> I'm pretty sure this is a bug in MSVC, that the namespace lookup
> for
> >>> a templated function happens in the context in which the template
> is
> >>> evaluated (as opposed to where it was defined).  But it makes sense
> >>> to patch it anyway.
> >>> Dan
> >>>
> >>>
> >>> On Mon, May 11, 2015 at 2:09 PM, Jan Trmal <jt...@gm...>
> wrote:
> >>> > Ok, I will have a.look tmrw. I think I've resolved the types
> issue
> >>> > (last one you mentioned). I generated the patch for openfst, that
> >>> > defines the int types within the openfst namespace - originally
> >>> > they exist on.the highest scope (i.e. ::int32) The patch has
> >>> > something
> >>> like
> >>> > namespace fst {
> >>> >   typedef int32 ::int32;
> >>> > }
> >>> >
> >>> > Doping this, the lookup works fine even under the ms c++.
> >>> > To me, this is optimal solution as we have to patch the openfst
> >>> anyway
> >>> > (and my change wont break any code).
> >>> > You can.have a look ať the openfstwin-1.3.3.patch in
> tools/extras.
> >>> > I am open to any suggestion or general discussion on how this
> >>> > should be made better.
> >>> > Y.
> >>> >
> >>> > On May 11, 2015 9:03 PM, "Daniel Povey" <dp...@gm...> wrote:
> >>> >>
> >>> >> > Well, the build was not "painless" at all. In fact, the perl
> >>> script
> >>> >> > generated a solution with 600+ projects that simply failed to
> >>> >> > build. VS was churning out dependencies on these, and crashed
> >>> >> > in about 10 minutes of just sitting there with a 100% CPU use.
> >>> >> > Not a single file was compiled, so I think it was just
> >>> >> > autogenerating
> >>> dependencies at this point.
> >>> >>
> >>> >> Build systems on Windows are pretty broken.  I think in
> Microsoft
> >>> >> they use some other system, not the one used in VS, which is not
> >>> >> publicly released.
> >>> >>
> >>> >> > I did not try to build from the command line though. Am I
> >>> >> > understanding it would compile?
> >>> >>
> >>> >> No, not everything would compile.  Sometimes the VS compiler
> just
> >>> crashes.
> >>> >>
> >>> >> > What I did not like about the existing system is that I could
> >>> >> > not plug in CUDA or intel compiler. It is extremely rigid in
> >>> >> > the
> >>> option
> >>> >> > part, and to change compiler options you are most likely to
> >>> >> > change the perl code that generates the project files, or
> >>> >> > modify the different MSBuild include files (called "property
> >>> >> > sheets" in this
> >>> >> > context) that are interplaying quite non-trivially. No, I have
> >>> >> > spent a few days on the build system not for a love of
> >>> >> > tinkering with MSBuils at all! :)
> >>> >>
> >>> >> What we committed was just a hack to try to get something
> working.
> >>> >> IIRC we just looked at the build files generated by VS to try to
> >>> >> guess what the format of those files was supposed to be; I'm not
> >>> sure
> >>> >> that they are really intended to be human readable/writable
> >>> >> (unless that human really loves XML).
> >>> >>
> >>> >> > Anyway, you can look at it at
> >>> >> > https://github.com/kkm000/kaldi/tree/winbuild/msbuild. (NB I
> >>> >> > have not updated the branch in a while) If you think this is
> >>> >> > not an acceptable solution, I'm just dropping the ball, no
> >>> >> > offence taken, naturally. I can maintain this as an
> "unofficial
> >>> >> > port" for a
> >>> while.
> >>> >> > In any case, build is the least problem, as Dan has pointed
> out
> >>> already...
> >>> >> >
> >>> >> > The biggest (in the number of changed lines) changes I had to
> >>> >> > make were in the namespace using declarations, because of
> >>> >> > conflicts between the types
> >>> >> > int32 and friends from different namespaces. I ended up
> reading
> >>> the
> >>> >> > C++ standard to understand whether gcc, icl or msvc is
> "correct"
> >>> in
> >>> >> > different cases they are complaining about, and the standard
> >>> >> > appears not to define any behavior there. These changes I
> would
> >>> >> > prefer accepted into the mainline of development, because they
> >>> >> > are
> >>> harder to maintain in a fork.
> >>> >>
> >>> >> In any case where you have to do something like typedef
> >>> >> kaldi::int32 int32, we should definitely get this committed-
> talk to Yenda.
> >>> >> Actually, this is really a bug in OpenFST which they haven't
> >>> >> fixed yet.  There is a "proper" POSIX way to get types like
> >>> >> int32, which
> >>> we
> >>> >> use in Kaldi, but in OpenFst they do it in a different, ad-hoc
> >>> >> way which sometimes generates different types (e.g. on a system
> >>> >> where
> >>> int
> >>> >> and long int are both 32-bit).  So the Kaldi int32 and the
> >>> >> OpenFst
> >>> >> int32 end up being different, incompatible types.
> >>> >>
> >>> >> Dan
> >>> >>
> >>> >>
> >>> >> >> -----Original Message-----
> >>> >> >> From: Jan Trmal [mailto:jt...@gm...]
> >>> >> >> Sent: 2015-05-11 0201
> >>> >> >> To: Dan Povey
> >>> >> >> Cc: Kirill Katsnelson; kaldi-
> dev...@li...;
> >>> >> >> kaldi- us...@li...; 洪孝宗
> >>> >> >> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
> >>> >> >>
> >>> >> >> Guys, the migration to git is "unofficially" done -- unless
> we
> >>> >> >> find some problems with it, the repo
> >>> >> >> (https://github.com/kaldi-
> >>> >> >> asr/kaldi.git) will stay as it is now.
> >>> >> >>
> >>> >> >> Kirill,  there is no reason why wait. We should start working
> >>> >> >> on
> >>> it.
> >>> >> >> I propose you could first send the Kaldi code patches -- I
> >>> >> >> actually worked on this and I'm able to compile Kaldi, so
> this
> >>> >> >> should be quite straightforward, provided you will generate
> >>> >> >> the
> >>> diff against the HEAD.
> >>> >> >>
> >>> >> >> Then we can start looking at the  "build system". I
> understand
> >>> >> >> your reasons and I think they might be legit. My feeling is,
> >>> >> >> however, that a lot of people are not using MSBuild and will
> >>> >> >> expect the MSVC solution, just because this is how they
> >>> >> >> usually work. I haven't tested it, but you should be able to
> >>> >> >> use the current SLN to build the binaries from command line -
> -
> >>> >> >> that
> >>> should
> >>> >> >> be completely painless -- but again, I haven't tested it.
> >>> >> >> y.
> >>> >> >>
> >>> >> >>
> >>> >> >> On Thu, May 7, 2015 at 10:53 PM, Daniel Povey
> >>> >> >> <dp...@gm...>
> >>> wrote:
> >>> >> >>
> >>> >> >>
> >>> >> >>       >> To the extent that your patches are simple fixes and
> >>> >> >> minor changes to
> >>> >> >>       >> the code, I think we could apply them.  Perhaps you
> >>> could
> >>> >> >> work with
> >>> >> >>       >> Yenda to get your changes checked?
> >>> >> >>       >
> >>> >> >>       > Easy enough, and as easy to hold till the migration
> is
> >>> done.
> >>> >> >> Was there a change in the plans?
> >>> >> >>
> >>> >> >>       Not really a change in the plans, but we don't have a
> >>> definite
> >>> >> >>       timeline for migrating to git, and it could be that
> >>> >> >> we'll keep the svn
> >>> >> >>       as upstream for a while.  Your changes seem reasonable,
> >>> >> >> but I think it
> >>> >> >>       would be better to get things started now rather than
> >>> later.
> >>> >> >> The
> >>> >> >>       git/svn issues should not be that hard.  Just start
> >>> >> >> sending patches to
> >>> >> >>       Yenda and we'll handle your changes bit by bit.
> >>> >> >>       Dan
> >>> >> >>
> >>> >> >>
> >>> >> >>
> >>> >> >>       >
> >>> >> >>       >> If you update your git repo to
> >>> >> >>       >> point to the current code, then a patch should be
> >>> >> >> applicable by the
> >>> >> >>       >> program "patch" to the svn repo too.
> >>> >> >>       >
> >>> >> >>       > There are many changes, and I certainly want to split
> >>> >> >> the patch so it is readable. One megapatch is not reviewable.
> >>> >> >>       >
> >>> >> >>       >> Regarding your build approach, you could send me and
> >>> >> >> Yenda some details
> >>> >> >>       >> about how you did it, and we could consider whether
> >>> >> >> to support that.
> >>> >> >>       >
> >>> >> >>       > There's a Powershell script that takes information
> >>> >> >> from every src/*/Makefile into a simple declarative MSBuild
> >>> >> >> script $dirname.kwproj, and a top-level MSBuild "makefile"
> >>> >> >> that drives the whole thing. A little bit more complex than
> >>> >> >> that to allow for
> >>> >> >> user- defined options, but generally this is it. This
> supports
> >>> >> >> building libraries, tools, tests and running the tests with
> >>> >> >> command line switches. The whole thing pretty much mimics a
> >>> >> >> linux make process but using MS tools.
> >>> >> >>       >
> >>> >> >>       >> I don't think it makes sense to try to maintain a
> >>> >> >> parallel
> >>> >> >> "windows-
> >>> >> >>       >> compatible" version of the scripts, if there are
> >>> >> >> larger changes
> >>> >> >>       >> required there.
> >>> >> >>       >
> >>> >> >>       > I was targeting for no changes at all. There maybe a
> >>> >> >> few very small patches that supposed not to break
> compatibility.
> >>> >> >>       >
> >>> >> >>       >> Anyway you depend on cygwin for scripting, and the
> set
> >>> >> >>       >> of people who want to run cygwin and build with MSVC
> >>> >> >> is probably
> >>> >> >>       >> limited enough that I don't think it makes sense for
> >>> >> >> us to try to
> >>> >> >>       >> support it.
> >>> >> >>       >
> >>> >> >>       > This is not as simple a choice as it seems at first.
> >>> >> >> Alex Hung just posted a trick to build CUDA under Cygwin;
> >>> >> >> before he
> >>> did
> >>> >> >> I thought it was not possible. MKL is another story. I would
> >>> >> >> rather build under native Windows than try to pull this
> >>> >> >> monster into the Cygwin build environment.
> >>> >> >>       >
> >>> >> >>       >  -kkm
> >>> >> >>       >
> >>> >> >>
> >>> >> >>
> >>> >> >

Re: [Kaldi-developers] [Kaldi-users] Kaldi compiles now under VS 2013

[Daniel P. <dp...@gm...>]

BTW, this is one of many bugs that I specifically pointed out to the
Visual Studio people while at Microsoft (and I looked it up just now
because I remembered checking the standard before, so I knew it was
there).  Their attitude was that compliance with the standard doesn't
really matter; what really matters is to not break the Windows build.
Eventually I think I gained a bad reputation in that circle as not
being sufficiently "with the program".  Anyway, I'm willing to patch
Kaldi to support their broken compiler as far as is reasonable, but it
doesn't make me very happy.

Dan


On Mon, May 11, 2015 at 7:06 PM, Daniel Povey <dp...@gm...> wrote:
> It isn't a gray area in the standard; see
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4296.pdf
> the last paragraph on page 367:
>   If a name does not depend on a template-parameter (as defined in
> 14.6.2), a declaration (or set of declarations) for that name shall be
> in scope at the point where the name appears in the template
> definition; the name is bound to the declaration (or declarations)
> found at that point and this binding is not affected by declarations
> that are visible at the point of instantiation.
>
> Dan
>
>
> On Mon, May 11, 2015 at 2:50 PM, Kirill Katsnelson
> <kir...@sm...> wrote:
>> I thought it was but this really appears to be a grey area in the standard. But indeed, from the practical standpoint, I think the best thing to do is just make all compilers happy. I can figure out if this is really undefined C++ later, if there is a theoretical interest.
>>
>>  -kkm
>>
>>> -----Original Message-----
>>> From: Daniel Povey [mailto:dp...@gm...]
>>> Sent: 2015-05-11 1444
>>> To: Jan Trmal
>>> Cc: Kirill Katsnelson; kal...@li...; kaldi-
>>> dev...@li...; 洪孝宗
>>> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
>>>
>>> OK fine.
>>> I'm pretty sure this is a bug in MSVC, that the namespace lookup for a
>>> templated function happens in the context in which the template is
>>> evaluated (as opposed to where it was defined).  But it makes sense to
>>> patch it anyway.
>>> Dan
>>>
>>>
>>> On Mon, May 11, 2015 at 2:09 PM, Jan Trmal <jt...@gm...> wrote:
>>> > Ok, I will have a.look tmrw. I think I've resolved the types issue
>>> > (last one you mentioned). I generated the patch for openfst, that
>>> > defines the int types within the openfst namespace - originally they
>>> > exist on.the highest scope (i.e. ::int32) The patch has something
>>> like
>>> > namespace fst {
>>> >   typedef int32 ::int32;
>>> > }
>>> >
>>> > Doping this, the lookup works fine even under the ms c++.
>>> > To me, this is optimal solution as we have to patch the openfst
>>> anyway
>>> > (and my change wont break any code).
>>> > You can.have a look ať the openfstwin-1.3.3.patch in tools/extras.
>>> > I am open to any suggestion or general discussion on how this should
>>> > be made better.
>>> > Y.
>>> >
>>> > On May 11, 2015 9:03 PM, "Daniel Povey" <dp...@gm...> wrote:
>>> >>
>>> >> > Well, the build was not "painless" at all. In fact, the perl
>>> script
>>> >> > generated a solution with 600+ projects that simply failed to
>>> >> > build. VS was churning out dependencies on these, and crashed in
>>> >> > about 10 minutes of just sitting there with a 100% CPU use. Not a
>>> >> > single file was compiled, so I think it was just autogenerating
>>> dependencies at this point.
>>> >>
>>> >> Build systems on Windows are pretty broken.  I think in Microsoft
>>> >> they use some other system, not the one used in VS, which is not
>>> >> publicly released.
>>> >>
>>> >> > I did not try to build from the command line though. Am I
>>> >> > understanding it would compile?
>>> >>
>>> >> No, not everything would compile.  Sometimes the VS compiler just
>>> crashes.
>>> >>
>>> >> > What I did not like about the existing system is that I could not
>>> >> > plug in CUDA or intel compiler. It is extremely rigid in the
>>> option
>>> >> > part, and to change compiler options you are most likely to change
>>> >> > the perl code that generates the project files, or modify the
>>> >> > different MSBuild include files (called "property sheets" in this
>>> >> > context) that are interplaying quite non-trivially. No, I have
>>> >> > spent a few days on the build system not for a love of tinkering
>>> >> > with MSBuils at all! :)
>>> >>
>>> >> What we committed was just a hack to try to get something working.
>>> >> IIRC we just looked at the build files generated by VS to try to
>>> >> guess what the format of those files was supposed to be; I'm not
>>> sure
>>> >> that they are really intended to be human readable/writable (unless
>>> >> that human really loves XML).
>>> >>
>>> >> > Anyway, you can look at it at
>>> >> > https://github.com/kkm000/kaldi/tree/winbuild/msbuild. (NB I have
>>> >> > not updated the branch in a while) If you think this is not an
>>> >> > acceptable solution, I'm just dropping the ball, no offence taken,
>>> >> > naturally. I can maintain this as an "unofficial port" for a
>>> while.
>>> >> > In any case, build is the least problem, as Dan has pointed out
>>> already...
>>> >> >
>>> >> > The biggest (in the number of changed lines) changes I had to make
>>> >> > were in the namespace using declarations, because of conflicts
>>> >> > between the types
>>> >> > int32 and friends from different namespaces. I ended up reading
>>> the
>>> >> > C++ standard to understand whether gcc, icl or msvc is "correct"
>>> in
>>> >> > different cases they are complaining about, and the standard
>>> >> > appears not to define any behavior there. These changes I would
>>> >> > prefer accepted into the mainline of development, because they are
>>> harder to maintain in a fork.
>>> >>
>>> >> In any case where you have to do something like typedef kaldi::int32
>>> >> int32, we should definitely get this committed- talk to Yenda.
>>> >> Actually, this is really a bug in OpenFST which they haven't fixed
>>> >> yet.  There is a "proper" POSIX way to get types like int32, which
>>> we
>>> >> use in Kaldi, but in OpenFst they do it in a different, ad-hoc way
>>> >> which sometimes generates different types (e.g. on a system where
>>> int
>>> >> and long int are both 32-bit).  So the Kaldi int32 and the OpenFst
>>> >> int32 end up being different, incompatible types.
>>> >>
>>> >> Dan
>>> >>
>>> >>
>>> >> >> -----Original Message-----
>>> >> >> From: Jan Trmal [mailto:jt...@gm...]
>>> >> >> Sent: 2015-05-11 0201
>>> >> >> To: Dan Povey
>>> >> >> Cc: Kirill Katsnelson; kal...@li...;
>>> >> >> kaldi- us...@li...; 洪孝宗
>>> >> >> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
>>> >> >>
>>> >> >> Guys, the migration to git is "unofficially" done -- unless we
>>> >> >> find some problems with it, the repo (https://github.com/kaldi-
>>> >> >> asr/kaldi.git) will stay as it is now.
>>> >> >>
>>> >> >> Kirill,  there is no reason why wait. We should start working on
>>> it.
>>> >> >> I propose you could first send the Kaldi code patches -- I
>>> >> >> actually worked on this and I'm able to compile Kaldi, so this
>>> >> >> should be quite straightforward, provided you will generate the
>>> diff against the HEAD.
>>> >> >>
>>> >> >> Then we can start looking at the  "build system". I understand
>>> >> >> your reasons and I think they might be legit. My feeling is,
>>> >> >> however, that a lot of people are not using MSBuild and will
>>> >> >> expect the MSVC solution, just because this is how they usually
>>> >> >> work. I haven't tested it, but you should be able to use the
>>> >> >> current SLN to build the binaries from command line -- that
>>> should
>>> >> >> be completely painless -- but again, I haven't tested it.
>>> >> >> y.
>>> >> >>
>>> >> >>
>>> >> >> On Thu, May 7, 2015 at 10:53 PM, Daniel Povey <dp...@gm...>
>>> wrote:
>>> >> >>
>>> >> >>
>>> >> >>       >> To the extent that your patches are simple fixes and
>>> >> >> minor changes to
>>> >> >>       >> the code, I think we could apply them.  Perhaps you
>>> could
>>> >> >> work with
>>> >> >>       >> Yenda to get your changes checked?
>>> >> >>       >
>>> >> >>       > Easy enough, and as easy to hold till the migration is
>>> done.
>>> >> >> Was there a change in the plans?
>>> >> >>
>>> >> >>       Not really a change in the plans, but we don't have a
>>> definite
>>> >> >>       timeline for migrating to git, and it could be that we'll
>>> >> >> keep the svn
>>> >> >>       as upstream for a while.  Your changes seem reasonable, but
>>> >> >> I think it
>>> >> >>       would be better to get things started now rather than
>>> later.
>>> >> >> The
>>> >> >>       git/svn issues should not be that hard.  Just start sending
>>> >> >> patches to
>>> >> >>       Yenda and we'll handle your changes bit by bit.
>>> >> >>       Dan
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>       >
>>> >> >>       >> If you update your git repo to
>>> >> >>       >> point to the current code, then a patch should be
>>> >> >> applicable by the
>>> >> >>       >> program "patch" to the svn repo too.
>>> >> >>       >
>>> >> >>       > There are many changes, and I certainly want to split the
>>> >> >> patch so it is readable. One megapatch is not reviewable.
>>> >> >>       >
>>> >> >>       >> Regarding your build approach, you could send me and
>>> >> >> Yenda some details
>>> >> >>       >> about how you did it, and we could consider whether to
>>> >> >> support that.
>>> >> >>       >
>>> >> >>       > There's a Powershell script that takes information from
>>> >> >> every src/*/Makefile into a simple declarative MSBuild script
>>> >> >> $dirname.kwproj, and a top-level MSBuild "makefile" that drives
>>> >> >> the whole thing. A little bit more complex than that to allow for
>>> >> >> user- defined options, but generally this is it. This supports
>>> >> >> building libraries, tools, tests and running the tests with
>>> >> >> command line switches. The whole thing pretty much mimics a linux
>>> >> >> make process but using MS tools.
>>> >> >>       >
>>> >> >>       >> I don't think it makes sense to try to maintain a
>>> >> >> parallel
>>> >> >> "windows-
>>> >> >>       >> compatible" version of the scripts, if there are larger
>>> >> >> changes
>>> >> >>       >> required there.
>>> >> >>       >
>>> >> >>       > I was targeting for no changes at all. There maybe a few
>>> >> >> very small patches that supposed not to break compatibility.
>>> >> >>       >
>>> >> >>       >> Anyway you depend on cygwin for scripting, and the set
>>> >> >>       >> of people who want to run cygwin and build with MSVC is
>>> >> >> probably
>>> >> >>       >> limited enough that I don't think it makes sense for us
>>> >> >> to try to
>>> >> >>       >> support it.
>>> >> >>       >
>>> >> >>       > This is not as simple a choice as it seems at first. Alex
>>> >> >> Hung just posted a trick to build CUDA under Cygwin; before he
>>> did
>>> >> >> I thought it was not possible. MKL is another story. I would
>>> >> >> rather build under native Windows than try to pull this monster
>>> >> >> into the Cygwin build environment.
>>> >> >>       >
>>> >> >>       >  -kkm
>>> >> >>       >
>>> >> >>
>>> >> >>
>>> >> >

Re: [Kaldi-developers] [Kaldi-users] Kaldi compiles now under VS 2013

[Daniel P. <dp...@gm...>]

It isn't a gray area in the standard; see
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4296.pdf
the last paragraph on page 367:
  If a name does not depend on a template-parameter (as defined in
14.6.2), a declaration (or set of declarations) for that name shall be
in scope at the point where the name appears in the template
definition; the name is bound to the declaration (or declarations)
found at that point and this binding is not affected by declarations
that are visible at the point of instantiation.

Dan


On Mon, May 11, 2015 at 2:50 PM, Kirill Katsnelson
<kir...@sm...> wrote:
> I thought it was but this really appears to be a grey area in the standard. But indeed, from the practical standpoint, I think the best thing to do is just make all compilers happy. I can figure out if this is really undefined C++ later, if there is a theoretical interest.
>
>  -kkm
>
>> -----Original Message-----
>> From: Daniel Povey [mailto:dp...@gm...]
>> Sent: 2015-05-11 1444
>> To: Jan Trmal
>> Cc: Kirill Katsnelson; kal...@li...; kaldi-
>> dev...@li...; 洪孝宗
>> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
>>
>> OK fine.
>> I'm pretty sure this is a bug in MSVC, that the namespace lookup for a
>> templated function happens in the context in which the template is
>> evaluated (as opposed to where it was defined).  But it makes sense to
>> patch it anyway.
>> Dan
>>
>>
>> On Mon, May 11, 2015 at 2:09 PM, Jan Trmal <jt...@gm...> wrote:
>> > Ok, I will have a.look tmrw. I think I've resolved the types issue
>> > (last one you mentioned). I generated the patch for openfst, that
>> > defines the int types within the openfst namespace - originally they
>> > exist on.the highest scope (i.e. ::int32) The patch has something
>> like
>> > namespace fst {
>> >   typedef int32 ::int32;
>> > }
>> >
>> > Doping this, the lookup works fine even under the ms c++.
>> > To me, this is optimal solution as we have to patch the openfst
>> anyway
>> > (and my change wont break any code).
>> > You can.have a look ať the openfstwin-1.3.3.patch in tools/extras.
>> > I am open to any suggestion or general discussion on how this should
>> > be made better.
>> > Y.
>> >
>> > On May 11, 2015 9:03 PM, "Daniel Povey" <dp...@gm...> wrote:
>> >>
>> >> > Well, the build was not "painless" at all. In fact, the perl
>> script
>> >> > generated a solution with 600+ projects that simply failed to
>> >> > build. VS was churning out dependencies on these, and crashed in
>> >> > about 10 minutes of just sitting there with a 100% CPU use. Not a
>> >> > single file was compiled, so I think it was just autogenerating
>> dependencies at this point.
>> >>
>> >> Build systems on Windows are pretty broken.  I think in Microsoft
>> >> they use some other system, not the one used in VS, which is not
>> >> publicly released.
>> >>
>> >> > I did not try to build from the command line though. Am I
>> >> > understanding it would compile?
>> >>
>> >> No, not everything would compile.  Sometimes the VS compiler just
>> crashes.
>> >>
>> >> > What I did not like about the existing system is that I could not
>> >> > plug in CUDA or intel compiler. It is extremely rigid in the
>> option
>> >> > part, and to change compiler options you are most likely to change
>> >> > the perl code that generates the project files, or modify the
>> >> > different MSBuild include files (called "property sheets" in this
>> >> > context) that are interplaying quite non-trivially. No, I have
>> >> > spent a few days on the build system not for a love of tinkering
>> >> > with MSBuils at all! :)
>> >>
>> >> What we committed was just a hack to try to get something working.
>> >> IIRC we just looked at the build files generated by VS to try to
>> >> guess what the format of those files was supposed to be; I'm not
>> sure
>> >> that they are really intended to be human readable/writable (unless
>> >> that human really loves XML).
>> >>
>> >> > Anyway, you can look at it at
>> >> > https://github.com/kkm000/kaldi/tree/winbuild/msbuild. (NB I have
>> >> > not updated the branch in a while) If you think this is not an
>> >> > acceptable solution, I'm just dropping the ball, no offence taken,
>> >> > naturally. I can maintain this as an "unofficial port" for a
>> while.
>> >> > In any case, build is the least problem, as Dan has pointed out
>> already...
>> >> >
>> >> > The biggest (in the number of changed lines) changes I had to make
>> >> > were in the namespace using declarations, because of conflicts
>> >> > between the types
>> >> > int32 and friends from different namespaces. I ended up reading
>> the
>> >> > C++ standard to understand whether gcc, icl or msvc is "correct"
>> in
>> >> > different cases they are complaining about, and the standard
>> >> > appears not to define any behavior there. These changes I would
>> >> > prefer accepted into the mainline of development, because they are
>> harder to maintain in a fork.
>> >>
>> >> In any case where you have to do something like typedef kaldi::int32
>> >> int32, we should definitely get this committed- talk to Yenda.
>> >> Actually, this is really a bug in OpenFST which they haven't fixed
>> >> yet.  There is a "proper" POSIX way to get types like int32, which
>> we
>> >> use in Kaldi, but in OpenFst they do it in a different, ad-hoc way
>> >> which sometimes generates different types (e.g. on a system where
>> int
>> >> and long int are both 32-bit).  So the Kaldi int32 and the OpenFst
>> >> int32 end up being different, incompatible types.
>> >>
>> >> Dan
>> >>
>> >>
>> >> >> -----Original Message-----
>> >> >> From: Jan Trmal [mailto:jt...@gm...]
>> >> >> Sent: 2015-05-11 0201
>> >> >> To: Dan Povey
>> >> >> Cc: Kirill Katsnelson; kal...@li...;
>> >> >> kaldi- us...@li...; 洪孝宗
>> >> >> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
>> >> >>
>> >> >> Guys, the migration to git is "unofficially" done -- unless we
>> >> >> find some problems with it, the repo (https://github.com/kaldi-
>> >> >> asr/kaldi.git) will stay as it is now.
>> >> >>
>> >> >> Kirill,  there is no reason why wait. We should start working on
>> it.
>> >> >> I propose you could first send the Kaldi code patches -- I
>> >> >> actually worked on this and I'm able to compile Kaldi, so this
>> >> >> should be quite straightforward, provided you will generate the
>> diff against the HEAD.
>> >> >>
>> >> >> Then we can start looking at the  "build system". I understand
>> >> >> your reasons and I think they might be legit. My feeling is,
>> >> >> however, that a lot of people are not using MSBuild and will
>> >> >> expect the MSVC solution, just because this is how they usually
>> >> >> work. I haven't tested it, but you should be able to use the
>> >> >> current SLN to build the binaries from command line -- that
>> should
>> >> >> be completely painless -- but again, I haven't tested it.
>> >> >> y.
>> >> >>
>> >> >>
>> >> >> On Thu, May 7, 2015 at 10:53 PM, Daniel Povey <dp...@gm...>
>> wrote:
>> >> >>
>> >> >>
>> >> >>       >> To the extent that your patches are simple fixes and
>> >> >> minor changes to
>> >> >>       >> the code, I think we could apply them.  Perhaps you
>> could
>> >> >> work with
>> >> >>       >> Yenda to get your changes checked?
>> >> >>       >
>> >> >>       > Easy enough, and as easy to hold till the migration is
>> done.
>> >> >> Was there a change in the plans?
>> >> >>
>> >> >>       Not really a change in the plans, but we don't have a
>> definite
>> >> >>       timeline for migrating to git, and it could be that we'll
>> >> >> keep the svn
>> >> >>       as upstream for a while.  Your changes seem reasonable, but
>> >> >> I think it
>> >> >>       would be better to get things started now rather than
>> later.
>> >> >> The
>> >> >>       git/svn issues should not be that hard.  Just start sending
>> >> >> patches to
>> >> >>       Yenda and we'll handle your changes bit by bit.
>> >> >>       Dan
>> >> >>
>> >> >>
>> >> >>
>> >> >>       >
>> >> >>       >> If you update your git repo to
>> >> >>       >> point to the current code, then a patch should be
>> >> >> applicable by the
>> >> >>       >> program "patch" to the svn repo too.
>> >> >>       >
>> >> >>       > There are many changes, and I certainly want to split the
>> >> >> patch so it is readable. One megapatch is not reviewable.
>> >> >>       >
>> >> >>       >> Regarding your build approach, you could send me and
>> >> >> Yenda some details
>> >> >>       >> about how you did it, and we could consider whether to
>> >> >> support that.
>> >> >>       >
>> >> >>       > There's a Powershell script that takes information from
>> >> >> every src/*/Makefile into a simple declarative MSBuild script
>> >> >> $dirname.kwproj, and a top-level MSBuild "makefile" that drives
>> >> >> the whole thing. A little bit more complex than that to allow for
>> >> >> user- defined options, but generally this is it. This supports
>> >> >> building libraries, tools, tests and running the tests with
>> >> >> command line switches. The whole thing pretty much mimics a linux
>> >> >> make process but using MS tools.
>> >> >>       >
>> >> >>       >> I don't think it makes sense to try to maintain a
>> >> >> parallel
>> >> >> "windows-
>> >> >>       >> compatible" version of the scripts, if there are larger
>> >> >> changes
>> >> >>       >> required there.
>> >> >>       >
>> >> >>       > I was targeting for no changes at all. There maybe a few
>> >> >> very small patches that supposed not to break compatibility.
>> >> >>       >
>> >> >>       >> Anyway you depend on cygwin for scripting, and the set
>> >> >>       >> of people who want to run cygwin and build with MSVC is
>> >> >> probably
>> >> >>       >> limited enough that I don't think it makes sense for us
>> >> >> to try to
>> >> >>       >> support it.
>> >> >>       >
>> >> >>       > This is not as simple a choice as it seems at first. Alex
>> >> >> Hung just posted a trick to build CUDA under Cygwin; before he
>> did
>> >> >> I thought it was not possible. MKL is another story. I would
>> >> >> rather build under native Windows than try to pull this monster
>> >> >> into the Cygwin build environment.
>> >> >>       >
>> >> >>       >  -kkm
>> >> >>       >
>> >> >>
>> >> >>
>> >> >

Re: [Kaldi-developers] [Kaldi-users] Kaldi compiles now under VS 2013

[Kirill K. <kir...@sm...>]

I thought it was but this really appears to be a grey area in the standard. But indeed, from the practical standpoint, I think the best thing to do is just make all compilers happy. I can figure out if this is really undefined C++ later, if there is a theoretical interest. 

 -kkm

> -----Original Message-----
> From: Daniel Povey [mailto:dp...@gm...]
> Sent: 2015-05-11 1444
> To: Jan Trmal
> Cc: Kirill Katsnelson; kal...@li...; kaldi-
> dev...@li...; 洪孝宗
> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
> 
> OK fine.
> I'm pretty sure this is a bug in MSVC, that the namespace lookup for a
> templated function happens in the context in which the template is
> evaluated (as opposed to where it was defined).  But it makes sense to
> patch it anyway.
> Dan
> 
> 
> On Mon, May 11, 2015 at 2:09 PM, Jan Trmal <jt...@gm...> wrote:
> > Ok, I will have a.look tmrw. I think I've resolved the types issue
> > (last one you mentioned). I generated the patch for openfst, that
> > defines the int types within the openfst namespace - originally they
> > exist on.the highest scope (i.e. ::int32) The patch has something
> like
> > namespace fst {
> >   typedef int32 ::int32;
> > }
> >
> > Doping this, the lookup works fine even under the ms c++.
> > To me, this is optimal solution as we have to patch the openfst
> anyway
> > (and my change wont break any code).
> > You can.have a look ať the openfstwin-1.3.3.patch in tools/extras.
> > I am open to any suggestion or general discussion on how this should
> > be made better.
> > Y.
> >
> > On May 11, 2015 9:03 PM, "Daniel Povey" <dp...@gm...> wrote:
> >>
> >> > Well, the build was not "painless" at all. In fact, the perl
> script
> >> > generated a solution with 600+ projects that simply failed to
> >> > build. VS was churning out dependencies on these, and crashed in
> >> > about 10 minutes of just sitting there with a 100% CPU use. Not a
> >> > single file was compiled, so I think it was just autogenerating
> dependencies at this point.
> >>
> >> Build systems on Windows are pretty broken.  I think in Microsoft
> >> they use some other system, not the one used in VS, which is not
> >> publicly released.
> >>
> >> > I did not try to build from the command line though. Am I
> >> > understanding it would compile?
> >>
> >> No, not everything would compile.  Sometimes the VS compiler just
> crashes.
> >>
> >> > What I did not like about the existing system is that I could not
> >> > plug in CUDA or intel compiler. It is extremely rigid in the
> option
> >> > part, and to change compiler options you are most likely to change
> >> > the perl code that generates the project files, or modify the
> >> > different MSBuild include files (called "property sheets" in this
> >> > context) that are interplaying quite non-trivially. No, I have
> >> > spent a few days on the build system not for a love of tinkering
> >> > with MSBuils at all! :)
> >>
> >> What we committed was just a hack to try to get something working.
> >> IIRC we just looked at the build files generated by VS to try to
> >> guess what the format of those files was supposed to be; I'm not
> sure
> >> that they are really intended to be human readable/writable (unless
> >> that human really loves XML).
> >>
> >> > Anyway, you can look at it at
> >> > https://github.com/kkm000/kaldi/tree/winbuild/msbuild. (NB I have
> >> > not updated the branch in a while) If you think this is not an
> >> > acceptable solution, I'm just dropping the ball, no offence taken,
> >> > naturally. I can maintain this as an "unofficial port" for a
> while.
> >> > In any case, build is the least problem, as Dan has pointed out
> already...
> >> >
> >> > The biggest (in the number of changed lines) changes I had to make
> >> > were in the namespace using declarations, because of conflicts
> >> > between the types
> >> > int32 and friends from different namespaces. I ended up reading
> the
> >> > C++ standard to understand whether gcc, icl or msvc is "correct"
> in
> >> > different cases they are complaining about, and the standard
> >> > appears not to define any behavior there. These changes I would
> >> > prefer accepted into the mainline of development, because they are
> harder to maintain in a fork.
> >>
> >> In any case where you have to do something like typedef kaldi::int32
> >> int32, we should definitely get this committed- talk to Yenda.
> >> Actually, this is really a bug in OpenFST which they haven't fixed
> >> yet.  There is a "proper" POSIX way to get types like int32, which
> we
> >> use in Kaldi, but in OpenFst they do it in a different, ad-hoc way
> >> which sometimes generates different types (e.g. on a system where
> int
> >> and long int are both 32-bit).  So the Kaldi int32 and the OpenFst
> >> int32 end up being different, incompatible types.
> >>
> >> Dan
> >>
> >>
> >> >> -----Original Message-----
> >> >> From: Jan Trmal [mailto:jt...@gm...]
> >> >> Sent: 2015-05-11 0201
> >> >> To: Dan Povey
> >> >> Cc: Kirill Katsnelson; kal...@li...;
> >> >> kaldi- us...@li...; 洪孝宗
> >> >> Subject: Re: [Kaldi-users] Kaldi compiles now under VS 2013
> >> >>
> >> >> Guys, the migration to git is "unofficially" done -- unless we
> >> >> find some problems with it, the repo (https://github.com/kaldi-
> >> >> asr/kaldi.git) will stay as it is now.
> >> >>
> >> >> Kirill,  there is no reason why wait. We should start working on
> it.
> >> >> I propose you could first send the Kaldi code patches -- I
> >> >> actually worked on this and I'm able to compile Kaldi, so this
> >> >> should be quite straightforward, provided you will generate the
> diff against the HEAD.
> >> >>
> >> >> Then we can start looking at the  "build system". I understand
> >> >> your reasons and I think they might be legit. My feeling is,
> >> >> however, that a lot of people are not using MSBuild and will
> >> >> expect the MSVC solution, just because this is how they usually
> >> >> work. I haven't tested it, but you should be able to use the
> >> >> current SLN to build the binaries from command line -- that
> should
> >> >> be completely painless -- but again, I haven't tested it.
> >> >> y.
> >> >>
> >> >>
> >> >> On Thu, May 7, 2015 at 10:53 PM, Daniel Povey <dp...@gm...>
> wrote:
> >> >>
> >> >>
> >> >>       >> To the extent that your patches are simple fixes and
> >> >> minor changes to
> >> >>       >> the code, I think we could apply them.  Perhaps you
> could
> >> >> work with
> >> >>       >> Yenda to get your changes checked?
> >> >>       >
> >> >>       > Easy enough, and as easy to hold till the migration is
> done.
> >> >> Was there a change in the plans?
> >> >>
> >> >>       Not really a change in the plans, but we don't have a
> definite
> >> >>       timeline for migrating to git, and it could be that we'll
> >> >> keep the svn
> >> >>       as upstream for a while.  Your changes seem reasonable, but
> >> >> I think it
> >> >>       would be better to get things started now rather than
> later.
> >> >> The
> >> >>       git/svn issues should not be that hard.  Just start sending
> >> >> patches to
> >> >>       Yenda and we'll handle your changes bit by bit.
> >> >>       Dan
> >> >>
> >> >>
> >> >>
> >> >>       >
> >> >>       >> If you update your git repo to
> >> >>       >> point to the current code, then a patch should be
> >> >> applicable by the
> >> >>       >> program "patch" to the svn repo too.
> >> >>       >
> >> >>       > There are many changes, and I certainly want to split the
> >> >> patch so it is readable. One megapatch is not reviewable.
> >> >>       >
> >> >>       >> Regarding your build approach, you could send me and
> >> >> Yenda some details
> >> >>       >> about how you did it, and we could consider whether to
> >> >> support that.
> >> >>       >
> >> >>       > There's a Powershell script that takes information from
> >> >> every src/*/Makefile into a simple declarative MSBuild script
> >> >> $dirname.kwproj, and a top-level MSBuild "makefile" that drives
> >> >> the whole thing. A little bit more complex than that to allow for
> >> >> user- defined options, but generally this is it. This supports
> >> >> building libraries, tools, tests and running the tests with
> >> >> command line switches. The whole thing pretty much mimics a linux
> >> >> make process but using MS tools.
> >> >>       >
> >> >>       >> I don't think it makes sense to try to maintain a
> >> >> parallel
> >> >> "windows-
> >> >>       >> compatible" version of the scripts, if there are larger
> >> >> changes
> >> >>       >> required there.
> >> >>       >
> >> >>       > I was targeting for no changes at all. There maybe a few
> >> >> very small patches that supposed not to break compatibility.
> >> >>       >
> >> >>       >> Anyway you depend on cygwin for scripting, and the set
> >> >>       >> of people who want to run cygwin and build with MSVC is
> >> >> probably
> >> >>       >> limited enough that I don't think it makes sense for us
> >> >> to try to
> >> >>       >> support it.
> >> >>       >
> >> >>       > This is not as simple a choice as it seems at first. Alex
> >> >> Hung just posted a trick to build CUDA under Cygwin; before he
> did
> >> >> I thought it was not possible. MKL is another story. I would
> >> >> rather build under native Windows than try to pull this monster
> >> >> into the Cygwin build environment.
> >> >>       >
> >> >>       >  -kkm
> >> >>       >
> >> >>
> >> >>
> >> >