kaldi-developers — Kaldi Developers

[Kaldi-developers] GridEngine (or similar) on Red Hat

[Daniel P. <dp...@gm...>]

Hi,
Does anyone on this list have any advice or experience for installing
some kind of GridEngine on Red Hat linux or related distributions?
Generally speaking I advise people to use Debian if they want
GridEngine because it has packages that work.  But I'm wondering if
there is a solution available for Red Hat or Red Hat-derived
distributions.
Dan

Re: [Kaldi-developers] Speech sythesis with Kaldi

[Daniel P. <dp...@gm...>]

Matthew Aylett (cc'd) is working on a speech synthesis project called
"idlak", which is part of the Kaldi repository.  I believe he recently
got it to the point where it produces output.  Matthew, perhaps you
can comment, and show him where to look?
Dan


On Tue, May 5, 2015 at 2:45 AM, Michal Klíma <mic...@gm...> wrote:
> Hello,
> My name is Michal Klíma and I'm student from Czech Republic (University of
> West Bohemia). I'm working on my master's thesis and I want to try your
> system Kaldi. I have one important question. Are here any possibilities to
> do speech synthesis with Kaldi? I have tried to find something about it, but
> I haven't been succesfull yet. I know that Kaldi is very similar to another
> speech regogniser toolkit named HTK and I know there is possibble to do
> speech synthesis.
> Yours sincerely
> Michal Klíma
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>

Re: [Kaldi-developers] A fix to the use of a nonstandard awk extension and a missing error check after it

[Vesely K. <ive...@fi...>]

Hi Kirill, Dan,
thanks for fixing the issue and letting me know.

I think the easiest solution to the original problem was to simply replace
'gensub' by 'gsub' or 'sub', which is available in all the versions of awk.

The perl fix is also fine...
Thanks,
Karel.

On 05/02/2015 07:10 AM, Daniel Povey wrote:
> Hi,
> Thanks for the patch.
> Unfortunately your patch brings up another compatibility problem: it 
> looks like the BSD-style 'sed' that is available on Mac does not 
> recognize the \0.  Instead I used a perl expression (perl is always 
> compatible across systems).
>  perl -ape 's:.*/([^/]+)\.gz$:$1 gunzip -c $& |:; '
> I committed this.
> Karel, I have tested the expression but not the whole script.
> I also made it use an absolute pathname in the .scp file, which is 
> always recommended.
> Dan
>
>
> On Fri, May 1, 2015 at 8:37 PM, Kirill Katsnelson 
> <kir...@sm... 
> <mailto:kir...@sm...>> wrote:
>
>     > -----Original Message-----
>     > From: Kirill Katsnelson
>     [mailto:kir...@sm...
>     <mailto:kir...@sm...>]
>     > Sent: 2015-05-01 1733
>     >
>     > Here's a small patch to make_denlats.sh. My awk does not have a
>     > function gensub() (Ubuntu 14.04 packages gawk separately from awk!),
>     > and because of that the generated lat.scp was empty. Compounding the
>     > problem, mpe training happily proceeded, even though missing
>     lattices
>     > for every utterance. Took me a while to track back.
>     >
>     > I have changed the code to use sed and to fail if the generated
>     scp has
>     > a zero length.
>     [kkm]
>
>     Ah, and the sort. My find(1) finds files quite out of order, maybe
>     because of ext4?
>
>     $ find dnn4_pretrain-dbn_dnn_denlats/lat1 -name '*.gz' | head -3
>     dnn4_pretrain-dbn_dnn_denlats/lat1/41/BertrandPiccard_2009G-0097823-0098802.gz
>     dnn4_pretrain-dbn_dnn_denlats/lat1/41/BernieKrause_2013G-0047540-0048091.gz
>     dnn4_pretrain-dbn_dnn_denlats/lat1/41/BethNoveck_2012G-0024724-0026655.gz
>
>     I do not know if sorted promise is used on the lat.scp later, so I
>     sorted it just in case. May not be really required.
>
>      -kkm
>
>
>
>
>     ------------------------------------------------------------------------------
>     One dashboard for servers and applications across
>     Physical-Virtual-Cloud
>     Widest out-of-the-box monitoring support with 50+ applications
>     Performance metrics, stats and reports that give you Actionable
>     Insights
>     Deep dive visibility with transaction tracing using APM Insight.
>     http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
>     _______________________________________________
>     Kaldi-developers mailing list
>     Kal...@li...
>     <mailto:Kal...@li...>
>     https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>
>

-- 
Karel Vesely, Brno University of Technology
ive...@fi..., +420-54114-1300

[Kaldi-developers] Speech sythesis with Kaldi

[Michal K. <mic...@gm...>]

Hello,
My name is Michal Klíma and I'm student from Czech Republic (University of
West Bohemia). I'm working on my master's thesis and I want to try your
system Kaldi. I have one important question. Are here any possibilities to
do speech synthesis with Kaldi? I have tried to find something about it,
but I haven't been succesfull yet. I know that Kaldi is very similar to
another speech regogniser toolkit named HTK and I know there is possibble
to do speech synthesis.
Yours sincerely
Michal Klíma

Re: [Kaldi-developers] A fix to the use of a nonstandard awk extension and a missing error check after it

[Daniel P. <dp...@gm...>]

Hi,
Thanks for the patch.
Unfortunately your patch brings up another compatibility problem: it looks
like the BSD-style 'sed' that is available on Mac does not recognize the
\0.  Instead I used a perl expression (perl is always compatible across
systems).
 perl -ape 's:.*/([^/]+)\.gz$:$1 gunzip -c $& |:; '
I committed this.
Karel, I have tested the expression but not the whole script.
I also made it use an absolute pathname in the .scp file, which is always
recommended.
Dan


On Fri, May 1, 2015 at 8:37 PM, Kirill Katsnelson <
kir...@sm...> wrote:

> > -----Original Message-----
> > From: Kirill Katsnelson [mailto:kir...@sm...]
> > Sent: 2015-05-01 1733
> >
> > Here's a small patch to make_denlats.sh. My awk does not have a
> > function gensub() (Ubuntu 14.04 packages gawk separately from awk!),
> > and because of that the generated lat.scp was empty. Compounding the
> > problem, mpe training happily proceeded, even though missing lattices
> > for every utterance. Took me a while to track back.
> >
> > I have changed the code to use sed and to fail if the generated scp has
> > a zero length.
> [kkm]
>
> Ah, and the sort. My find(1) finds files quite out of order, maybe because
> of ext4?
>
> $ find dnn4_pretrain-dbn_dnn_denlats/lat1 -name '*.gz' | head -3
>
> dnn4_pretrain-dbn_dnn_denlats/lat1/41/BertrandPiccard_2009G-0097823-0098802.gz
> dnn4_pretrain-dbn_dnn_denlats/lat1/41/BernieKrause_2013G-0047540-0048091.gz
> dnn4_pretrain-dbn_dnn_denlats/lat1/41/BethNoveck_2012G-0024724-0026655.gz
>
> I do not know if sorted promise is used on the lat.scp later, so I sorted
> it just in case. May not be really required.
>
>  -kkm
>
>
>
>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>

Re: [Kaldi-developers] A fix to the use of a nonstandard awk extension and a missing error check after it

[Kirill K. <kir...@sm...>]

> -----Original Message-----
> From: Kirill Katsnelson [mailto:kir...@sm...]
> Sent: 2015-05-01 1733
> 
> Here's a small patch to make_denlats.sh. My awk does not have a
> function gensub() (Ubuntu 14.04 packages gawk separately from awk!),
> and because of that the generated lat.scp was empty. Compounding the
> problem, mpe training happily proceeded, even though missing lattices
> for every utterance. Took me a while to track back.
> 
> I have changed the code to use sed and to fail if the generated scp has
> a zero length.
[kkm] 

Ah, and the sort. My find(1) finds files quite out of order, maybe because of ext4?

$ find dnn4_pretrain-dbn_dnn_denlats/lat1 -name '*.gz' | head -3
dnn4_pretrain-dbn_dnn_denlats/lat1/41/BertrandPiccard_2009G-0097823-0098802.gz
dnn4_pretrain-dbn_dnn_denlats/lat1/41/BernieKrause_2013G-0047540-0048091.gz
dnn4_pretrain-dbn_dnn_denlats/lat1/41/BethNoveck_2012G-0024724-0026655.gz

I do not know if sorted promise is used on the lat.scp later, so I sorted it just in case. May not be really required.

 -kkm

[Kaldi-developers] A fix to the use of a nonstandard awk extension and a missing error check after it

[Kirill K. <kir...@sm...>]

Here's a small patch to make_denlats.sh. My awk does not have a function gensub() (Ubuntu 14.04 packages gawk separately from awk!), and because of that the generated lat.scp was empty. Compounding the problem, mpe training happily proceeded, even though missing lattices for every utterance. Took me a while to track back.

I have changed the code to use sed and to fail if the generated scp has a zero length.

 -kkm

Index: egs/wsj/s5/steps/nnet/make_denlats.sh
===================================================================
--- egs/wsj/s5/steps/nnet/make_denlats.sh       (revision 5032)
+++ egs/wsj/s5/steps/nnet/make_denlats.sh       (working copy)
@@ -181,7 +181,8 @@

 #2) Generate 'scp' for reading the lattices
 for n in `seq $nj`; do
-  find $dir/lat${n} -name "*.gz" | awk -v FS="/" '{ print gensub(".gz","","",$NF)" gunzip -c "$0" |"; }'
-done >$dir/lat.scp
+  find $dir/lat${n} -name "*.gz" | sed -e 's%.*/\(.*\)\.gz%\1 gunzip -c \0 |%'
+done | sort >$dir/lat.scp
+[ -s $dir/lat.scp ] || exit 1

 echo "$0: done generating denominator lattices."

Re: [Kaldi-developers] DTMF, dialtone and ringtone examples

[Chiwei C. <ch...@vo...>]

Hello, premature idea, did we try to simply “record” these tone with some real phones (analog, digital phone are just re-synthesize) pressing?

From: Daniel Povey [mailto:dp...@gm...]
Sent: Thursday, April 30, 2015 11:35 AM
To: kal...@li...
Cc: David Snyder
Subject: [Kaldi-developers] DTMF, dialtone and ringtone examples

Everyone,
I want to add some code for DTMF, dialtone and ringtone detection and removal, to help clean up training data and hopefully to be used in telephone applications.  However, it seems hard to find "real-world" examples of these kinds of tones in speech.  Does anyone have any idea where we could find data with these tones in it?  We could consider non-releasable data, as it's possible to just tune the methods on it but not release the setup.
Dan

Re: [Kaldi-developers] DTMF, dialtone and ringtone examples

[Daniel P. <dp...@gm...>]

We haven't done this here at JHU.  It's not something I have that much time
to put energy into.  I was hoping someone else on the list might have
access to such data already and would be willing to help out.
Dan


On Thu, Apr 30, 2015 at 4:46 PM, Chiwei Che <ch...@vo...> wrote:

>  Hello, premature idea, did we try to simply “record” these tone with
> some real phones (analog, digital phone are just re-synthesize) pressing?
>
>
>
> *From:* Daniel Povey [mailto:dp...@gm...]
> *Sent:* Thursday, April 30, 2015 11:35 AM
> *To:* kal...@li...
> *Cc:* David Snyder
> *Subject:* [Kaldi-developers] DTMF, dialtone and ringtone examples
>
>
>
> Everyone,
>
> I want to add some code for DTMF, dialtone and ringtone detection and
> removal, to help clean up training data and hopefully to be used in
> telephone applications.  However, it seems hard to find "real-world"
> examples of these kinds of tones in speech.  Does anyone have any idea
> where we could find data with these tones in it?  We could consider
> non-releasable data, as it's possible to just tune the methods on it but
> not release the setup.
>
> Dan
>
>
>

[Kaldi-developers] DTMF, dialtone and ringtone examples

[Daniel P. <dp...@gm...>]

Everyone,
I want to add some code for DTMF, dialtone and ringtone detection and
removal, to help clean up training data and hopefully to be used in
telephone applications.  However, it seems hard to find "real-world"
examples of these kinds of tones in speech.  Does anyone have any idea
where we could find data with these tones in it?  We could consider
non-releasable data, as it's possible to just tune the methods on it but
not release the setup.
Dan

Re: [Kaldi-developers] Update on GStreamer related developments

[Daniel P. <dp...@gm...>]

Thanks for the update.
I had a look at your duration-model paper
https://phon.ioc.ee/dokuwiki/lib/exe/fetch.php?media=people:tanel:icassp2014-durmodel.pdf
and it's quite exciting that you were able to get so much improvement.  It
would be good if you could put in the effort to "Kaldi-ify" your recipe,
i.e. get rid of the dependency on Python and external tools, and make it
compatible with the existing structure of the scripts.  A good duration
model is a feature we need.  I can get someone else who knows the nnet2
code (e.g. Vijay) to help with the core neural-net-training part of it, or
help myself, if you can do the other parts.
Dan




On Wed, Apr 15, 2015 at 10:25 AM, Tanel Alumäe <tan...@ph...>
wrote:

> Hello everybody,
>
> Daniel asked me to give and update on the GStreamer related work with
> Kaldi that I have been doing.
>
> GStreamer is a multimedia framework. It consists of different plugins
> (audio and video decoders and encoders, resamplers, effect modules,
> input/output modules) that can be formed into pipelines. GStreamer can
> be used via its GObject introspection bindings. Thus, one can use
> GStreamer in any programming language that has support for GObject
> introspection, which includes Python, Ruby, Java, Vala.
>
> The Kaldi code base includes a GStreamer plugin that supports GMM
> models. More recently, I have also developed a similar plugin that
> supports "online DNN" models. It's available at
> https://github.com/alumae/gst-kaldi-nnet2-online. I am planning to
> maintain it as separate project from Kaldi (I believe Daniel agrees).
> The plugin has a very similar functionality as Kaldi's
> online2-wav-nnet2-latgen-faster, with some extensions. First, it can do
> on-the-fly audio segmentation, based on silences in the audio. It's
> based on the endpointing code in Kaldi's nnet2 code, but instead of
> terminating when an endpoint is encountered, it simply starts decoding
> the next segment. It can also do language model rescoring, as in
> lattice-lmrescore.
>
> It's very easy to create GUI speech recognition applications  using the
> plugin, or apply the plugin from command line to e.g. transcribe a long
> audio file. Check the 'demo' folder at Github.
>
> My other project is https://github.com/alumae/kaldi-gstreamer-server.
> It's a real-time full duplex speech recognition server, built around the
> Kaldi's GStreamer plugins. Features (copied from the README):
>
>   * Full duplex communication based on websockets: speech goes in,
> partial hypotheses come out (think of Android's voice typing)
>   * Very scalable: the server consists of a master component and
> workers; one worker is needed per concurrent recognition session;
> workers can be started and stopped independently of the master on remote
> machines
>   * Can do speech segmentation, i.e., a long speech signal is broken
> into shorter segments based on silences
>   * Supports arbitrarily long speech input (e.g., you can stream live
> speech into it)
>   * Supports Kaldi's GMM and "online DNN" models
>   * Supports rescoring of the recognition lattice with a large language
> model
>   * Supports persisting the acoustic model adaptation state between
> requests
>   * Supports unlimited set of audio codecs (actually only those
> supported by GStreamer)
>   * Supports rewriting raw recognition results using external programs
> (can be used for converting words to numbers, etc)
>   * Python, Java, Javascript clients are available
>
> We are using the server in several real world speech recognition
> applications, mainly for the Estonian language. E.g., we have developed
> an Android application that can act as a speech-recognition based
> "keyboard" (as Google's voice typing), and in a radiology dictation
> application that achieves 5% WER in real clinical environment.
>
> Not related to GStreamer, I have also an implementation of a novel phone
> duration model available at Github:
> https://github.com/alumae/kaldi-nnet-dur-model
> It's probably more interesting for researchers, but nevertheless, on the
> TEDLIUM test set it gives a drop in WER from 11.7% to 11.0%, from the
> online multisplice speed-perturbed DNN system with Cantab large LM
> rescoring.
>
> Regards,
> Tanel
>
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>

[Kaldi-developers] Update on GStreamer related developments

[Tanel A. <tan...@ph...>]

Hello everybody,

Daniel asked me to give and update on the GStreamer related work with
Kaldi that I have been doing.

GStreamer is a multimedia framework. It consists of different plugins
(audio and video decoders and encoders, resamplers, effect modules,
input/output modules) that can be formed into pipelines. GStreamer can
be used via its GObject introspection bindings. Thus, one can use
GStreamer in any programming language that has support for GObject
introspection, which includes Python, Ruby, Java, Vala.

The Kaldi code base includes a GStreamer plugin that supports GMM
models. More recently, I have also developed a similar plugin that
supports "online DNN" models. It's available at
https://github.com/alumae/gst-kaldi-nnet2-online. I am planning to
maintain it as separate project from Kaldi (I believe Daniel agrees).
The plugin has a very similar functionality as Kaldi's
online2-wav-nnet2-latgen-faster, with some extensions. First, it can do
on-the-fly audio segmentation, based on silences in the audio. It's
based on the endpointing code in Kaldi's nnet2 code, but instead of
terminating when an endpoint is encountered, it simply starts decoding
the next segment. It can also do language model rescoring, as in
lattice-lmrescore.

It's very easy to create GUI speech recognition applications  using the
plugin, or apply the plugin from command line to e.g. transcribe a long
audio file. Check the 'demo' folder at Github. 

My other project is https://github.com/alumae/kaldi-gstreamer-server.
It's a real-time full duplex speech recognition server, built around the
Kaldi's GStreamer plugins. Features (copied from the README):

  * Full duplex communication based on websockets: speech goes in,
partial hypotheses come out (think of Android's voice typing)
  * Very scalable: the server consists of a master component and
workers; one worker is needed per concurrent recognition session;
workers can be started and stopped independently of the master on remote
machines
  * Can do speech segmentation, i.e., a long speech signal is broken
into shorter segments based on silences
  * Supports arbitrarily long speech input (e.g., you can stream live
speech into it)
  * Supports Kaldi's GMM and "online DNN" models
  * Supports rescoring of the recognition lattice with a large language
model
  * Supports persisting the acoustic model adaptation state between
requests
  * Supports unlimited set of audio codecs (actually only those
supported by GStreamer)
  * Supports rewriting raw recognition results using external programs
(can be used for converting words to numbers, etc)
  * Python, Java, Javascript clients are available

We are using the server in several real world speech recognition
applications, mainly for the Estonian language. E.g., we have developed
an Android application that can act as a speech-recognition based
"keyboard" (as Google's voice typing), and in a radiology dictation
application that achieves 5% WER in real clinical environment.

Not related to GStreamer, I have also an implementation of a novel phone
duration model available at Github:
https://github.com/alumae/kaldi-nnet-dur-model 
It's probably more interesting for researchers, but nevertheless, on the
TEDLIUM test set it gives a drop in WER from 11.7% to 11.0%, from the
online multisplice speed-perturbed DNN system with Cantab large LM
rescoring. 

Regards,
Tanel

[Kaldi-developers] [Jenkins] Kaldi automatic build and test

[<jen...@a2...>]

Kaldi - Build # 1129 - Fixed:

See the build log in attachment for the details.

[Kaldi-developers] [Jenkins] Kaldi automatic build and test

[<jen...@a2...>]

Kaldi - Build # 1128 - Failure:

See the build log in attachment for the details.

Re: [Kaldi-developers] Testing Github repository

[Kirill K. <kir...@sm...>]

I am entirely with you. Rebasing is changing history. It is totally OOK and useful to requite history to squash commits, fix unintended check-ins of huge binaries etc. Absolutely OK before you pushed your changes anywhere. But if there is a chance that the branch (simply speaking) was shared, then rebasing would create a problem for the sharing party, and must be closely coordinated with them, or better abandoned altogether.

 -kkm

> -----Original Message-----
> From: Ondrej Platek [mailto:ond...@gm...]
> Sent: 2015-04-07 0304
> To: Vassil Panayotov
> Cc: Alexander Solovets; kal...@li...; Kirill
> Katsnelson
> Subject: Re: [Kaldi-developers] Testing Github repository
> 
> Hi,
> 
> I just want to mentioned that I like git rebase for cleaning of my
> messy LOCAL history into much cleaner one.
> On the other hand, AFAIK github does not support rebasing as accepting
> somebodies work very well, and generally rebasing is sometimes
> discourage for public repositories.
> 
> There is quite nice comparison at
> https://www.atlassian.com/git/articles/git-team-workflows-merge-or-
> rebas
> e/
> 
> 
> Personally, I see the problem elsewhere with merging a PR.
> Let say that the PR branch history is clean at the beginning and a user
> submitted PR.
> The problem with merging and PR is when you decide not to merge the PR
> immediately and substantial work is needed.
> Probably multiple people will work on the PR branch and multiple
> branches will be created.
> Rebasing is now dangerous since it is already published, and the
> history gets messy.
> 
> Still I find it far more trivial than svn branching.
> 
> Ondra
> 
> 
> 
> 
> On Tue, Apr 7, 2015 at 11:24 AM, Vassil Panayotov
> <vas...@gm...> wrote:
> 
> 
> 	FWIW +1 from me to the migration to Git. Don't know if that will
> really encourage people to contribute more, but some say the ease of
> creating pull requests is helpful... Having a "Kaldi" GitHub
> organization may also increase the visibility of projects, built on top
> of Kaldi(e.g. Tanel Alumae's ASR server), or related to Kaldi(like PDNN
> or the Python wrappers that were discussed here not long ago). GitHub's
> support for organizations is really basic (AFAIK they don't even
> support following an organization), but hopefully they will (finally)
> improve that.
> 
> 	IMHO the workflow suggested by Kirill, where everyone submits
> pull requests to the "official" repository, that are merged (possibly
> after a review) seems good. I don't think the branching will be a
> problem. As I understand it, in the proposed workflow the history of
> the official repo will be entirely linear(fast forward commits into
> "master"). I've mostly used Git with single-user(i.e. me) repositories
> so far, but AFAIK it only becomes problematic if you rewrite the
> history by rebasing or amending commits in a branch that is jointly
> worked on with others. And even if that happens it should be easy to
> recover, because with Git everyone has the full project history on his
> desktop.
> 
> 	It's not very clear to me if this transition period of few months
> is needed and how it would be helpful if people are still using only
> SVN? I'd say if everyone agrees that Git is the way to go, the switch
> can be made as soon as the process of switching and the new workflow
> are documented. From the user's viewpoint it probably won't matter much
> if they type "svn up" or "git pull origin master". Also, I agree with
> Kirill that keeping SVN in sync after the switch could be probably an
> unnecessary hassle. Having a backup is obviously not a problem, because
> in couple of days of migration to Git, there will be enough full copies
> of the history distributed around the world to survive a dinosaur
> extinction- causing event (even though the usefulness of Kaldi in the
> aftermath is uncertain...).
> 
> 	I haven't been involved with such migrations though, so maybe I'm
> overlooking something...
> 
> 	Vassil
> 
> 
> 
> 	On Tue, Apr 7, 2015 at 9:12 AM, Alexander Solovets
> <aso...@gm...> wrote:
> 
> 
> 		Jan, I don't know how Nikolay does it, but I simply
> download and
> 		apply the diff, and then run `git svn dcommit`. I know that
> you can
> 		also accept the pull request into a custom branch (i.e.
> non-master).
> 		In this case you can do it via GitHub's UI, and then rebase
> that
> 		branch on top of master and push changes.
> 
> 		Kirill, by "branching hell" I meant non-linear history with
> a lot of
> 		"merge" points. You not only loose the linearity, but also
> the merge
> 		conflict resolution becomes way more complex, the chance to
> 		accidentally drop someone else's changes increases. I do
> know about
> 		variety of git workflows and different techniques of
> accepting new
> 		commits, and I'm not going to argue, I just wanted to share
> my
> 		experience.
> 
> 
> 		On Tue, Apr 7, 2015 at 1:47 AM, Kirill Katsnelson
> 		<kir...@sm...> wrote:
> 		>> -----Original Message-----
> 		>> From: Daniel Povey [mailto:dp...@gm...]
> 		>> Perhaps you could explain how this is done?
> 		>> It might help us make an informed decision if you could
> give us some
> 		>> details on the workflow that you use, at the level of
> specific
> 		>> commands.
> 		>
> 		> Let me set up a pair of cloned demo repositories to show
> different practices, as time permits. Give me a day or two.
> 		>
> 		>  -kkm
> 		>
> -----------------------------------------------------------------------
> -
> ------
> 		> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM
> EDT
> 		> Develop your own process in accordance with the BPMN 2
> standard
> 		> Learn Process modeling best practices with Bonita BPM
> through live exercises
> 		>
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> 		>
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> 		> _______________________________________________
> 		> Kaldi-developers mailing list
> 		> Kal...@li...
> 		>
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 
> 
> 
> 
> 		--
> 		Sincerely, Alexander
> 
> 
> 
> -----------------------------------------------------------------------
> -
> ------
> 		BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM
> EDT
> 		Develop your own process in accordance with the BPMN 2
> standard
> 		Learn Process modeling best practices with Bonita BPM
> through live exercises
> 
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> 
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> 		_______________________________________________
> 		Kaldi-developers mailing list
> 		Kal...@li...
> 
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 
> 
> 
> 
> 
> -----------------------------------------------------------------------
> -
> ------
> 	BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> 	Develop your own process in accordance with the BPMN 2 standard
> 	Learn Process modeling best practices with Bonita BPM through
> live exercises
> 	http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> 
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> 	_______________________________________________
> 	Kaldi-developers mailing list
> 	Kal...@li...
> 	https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> 
> 
> 
> 
> 
> 
> --
> 
> Ondřej Plátek, +420 737 758 650, skype:ondrejplatek,
> ond...@gm...

Re: [Kaldi-developers] Testing Github repository

[Kirill K. <kir...@sm...>]

> -----Original Message-----
> From: Daniel Povey [mailto:dp...@gm...]
> Sent: 2015-04-06 1148
> Cc: kal...@li...
>
> Perhaps you could explain how this is done?
> It might help us make an informed decision if you could give us some
> details on the workflow that you use, at the level of specific
> commands.

Here's a quick walkthrough I composed:
https://github.com/workflow-demo-org/workflow-demo/wiki/GitHub-pull-request-based-workflow

Let me know if it's unclear or hard to understand otherwise.

 -kkm

Re: [Kaldi-developers] Testing Github repository

[Jan T. <af...@ce...>]

Vassil, the transition period is mostly for the active kaldi developers --
to give them time to learn git, finding out the best way for the
transition, writing docs, deciding on the workflow (and to allow them to
make mistakes). If the SVN stays as the primary repository during this
period, all the mistakes are non-fatal (we can always delete everything and
recreate the git repo again from SVN).
y.

On Tue, Apr 7, 2015 at 5:24 AM, Vassil Panayotov <vas...@gm...
> wrote:

> FWIW +1 from me to the migration to Git. Don't know if that will really
> encourage people to contribute more, but some say the ease of creating pull
> requests is helpful... Having a "Kaldi" GitHub organization may also
> increase the visibility of projects, built on top of Kaldi(e.g. Tanel
> Alumae's ASR server), or related to Kaldi(like PDNN or the Python wrappers
> that were discussed here not long ago). GitHub's support for organizations
> is really basic (AFAIK they don't even support following an organization),
> but hopefully they will (finally) improve that.
>
> IMHO the workflow suggested by Kirill, where everyone submits pull
> requests to the "official" repository, that are merged (possibly after a
> review) seems good. I don't think the branching will be a problem. As I
> understand it, in the proposed workflow the history of the official repo
> will be entirely linear(fast forward commits into "master"). I've mostly
> used Git with single-user(i.e. me) repositories so far, but AFAIK it only
> becomes problematic if you rewrite the history by rebasing or amending
> commits in a branch that is jointly worked on with others. And even if that
> happens it should be easy to recover, because with Git everyone has the
> full project history on his desktop.
>
> It's not very clear to me if this transition period of few months is
> needed and how it would be helpful if people are still using only SVN? I'd
> say if everyone agrees that Git is the way to go, the switch can be made as
> soon as the process of switching and the new workflow are documented. From
> the user's viewpoint it probably won't matter much if they type "svn up" or
> "git pull origin master". Also, I agree with Kirill that keeping SVN in
> sync after the switch could be probably an unnecessary hassle. Having a
> backup is obviously not a problem, because in couple of days of migration
> to Git, there will be enough full copies of the history distributed around
> the world to survive a dinosaur extinction- causing event (even though the
> usefulness of Kaldi in the aftermath is uncertain...).
>
> I haven't been involved with such migrations though, so maybe I'm
> overlooking something...
>
> Vassil
>
>
> On Tue, Apr 7, 2015 at 9:12 AM, Alexander Solovets <aso...@gm...>
> wrote:
>
>> Jan, I don't know how Nikolay does it, but I simply download and
>> apply the diff, and then run `git svn dcommit`. I know that you can
>> also accept the pull request into a custom branch (i.e. non-master).
>> In this case you can do it via GitHub's UI, and then rebase that
>> branch on top of master and push changes.
>>
>> Kirill, by "branching hell" I meant non-linear history with a lot of
>> "merge" points. You not only loose the linearity, but also the merge
>> conflict resolution becomes way more complex, the chance to
>> accidentally drop someone else's changes increases. I do know about
>> variety of git workflows and different techniques of accepting new
>> commits, and I'm not going to argue, I just wanted to share my
>> experience.
>>
>> On Tue, Apr 7, 2015 at 1:47 AM, Kirill Katsnelson
>> <kir...@sm...> wrote:
>> >> -----Original Message-----
>> >> From: Daniel Povey [mailto:dp...@gm...]
>> >> Perhaps you could explain how this is done?
>> >> It might help us make an informed decision if you could give us some
>> >> details on the workflow that you use, at the level of specific
>> >> commands.
>> >
>> > Let me set up a pair of cloned demo repositories to show different
>> practices, as time permits. Give me a day or two.
>> >
>> >  -kkm
>> >
>> ------------------------------------------------------------------------------
>> > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>> > Develop your own process in accordance with the BPMN 2 standard
>> > Learn Process modeling best practices with Bonita BPM through live
>> exercises
>> > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>> event?utm_
>> > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>> > _______________________________________________
>> > Kaldi-developers mailing list
>> > Kal...@li...
>> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>>
>>
>>
>> --
>> Sincerely, Alexander
>>
>>
>> ------------------------------------------------------------------------------
>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>> Develop your own process in accordance with the BPMN 2 standard
>> Learn Process modeling best practices with Bonita BPM through live
>> exercises
>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>> event?utm_
>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>> _______________________________________________
>> Kaldi-developers mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>>
>
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>
>

Re: [Kaldi-developers] Testing Github repository

[Ondrej P. <ond...@gm...>]

Hi,

I just want to mentioned that I like git rebase for cleaning of my messy
LOCAL history into much cleaner one.
On the other hand, AFAIK github does not support rebasing as accepting
somebodies work very well,
and generally rebasing is sometimes discourage for public repositories.

There is quite nice comparison at
https://www.atlassian.com/git/articles/git-team-workflows-merge-or-rebase/

Personally, I see the problem elsewhere with merging a PR.
Let say that the PR branch history is clean at the beginning and a user
submitted PR.
The problem with merging and PR is when you decide not to merge the PR
immediately and substantial work is needed.
Probably multiple people will work on the PR branch and multiple branches
will be created.
Rebasing is now dangerous since it is already published, and the history
gets messy.

Still I find it far more trivial than svn branching.

Ondra




On Tue, Apr 7, 2015 at 11:24 AM, Vassil Panayotov <
vas...@gm...> wrote:

> FWIW +1 from me to the migration to Git. Don't know if that will really
> encourage people to contribute more, but some say the ease of creating pull
> requests is helpful... Having a "Kaldi" GitHub organization may also
> increase the visibility of projects, built on top of Kaldi(e.g. Tanel
> Alumae's ASR server), or related to Kaldi(like PDNN or the Python wrappers
> that were discussed here not long ago). GitHub's support for organizations
> is really basic (AFAIK they don't even support following an organization),
> but hopefully they will (finally) improve that.
>
> IMHO the workflow suggested by Kirill, where everyone submits pull
> requests to the "official" repository, that are merged (possibly after a
> review) seems good. I don't think the branching will be a problem. As I
> understand it, in the proposed workflow the history of the official repo
> will be entirely linear(fast forward commits into "master"). I've mostly
> used Git with single-user(i.e. me) repositories so far, but AFAIK it only
> becomes problematic if you rewrite the history by rebasing or amending
> commits in a branch that is jointly worked on with others. And even if that
> happens it should be easy to recover, because with Git everyone has the
> full project history on his desktop.
>
> It's not very clear to me if this transition period of few months is
> needed and how it would be helpful if people are still using only SVN? I'd
> say if everyone agrees that Git is the way to go, the switch can be made as
> soon as the process of switching and the new workflow are documented. From
> the user's viewpoint it probably won't matter much if they type "svn up" or
> "git pull origin master". Also, I agree with Kirill that keeping SVN in
> sync after the switch could be probably an unnecessary hassle. Having a
> backup is obviously not a problem, because in couple of days of migration
> to Git, there will be enough full copies of the history distributed around
> the world to survive a dinosaur extinction- causing event (even though the
> usefulness of Kaldi in the aftermath is uncertain...).
>
> I haven't been involved with such migrations though, so maybe I'm
> overlooking something...
>
> Vassil
>
>
> On Tue, Apr 7, 2015 at 9:12 AM, Alexander Solovets <aso...@gm...>
> wrote:
>
>> Jan, I don't know how Nikolay does it, but I simply download and
>> apply the diff, and then run `git svn dcommit`. I know that you can
>> also accept the pull request into a custom branch (i.e. non-master).
>> In this case you can do it via GitHub's UI, and then rebase that
>> branch on top of master and push changes.
>>
>> Kirill, by "branching hell" I meant non-linear history with a lot of
>> "merge" points. You not only loose the linearity, but also the merge
>> conflict resolution becomes way more complex, the chance to
>> accidentally drop someone else's changes increases. I do know about
>> variety of git workflows and different techniques of accepting new
>> commits, and I'm not going to argue, I just wanted to share my
>> experience.
>>
>> On Tue, Apr 7, 2015 at 1:47 AM, Kirill Katsnelson
>> <kir...@sm...> wrote:
>> >> -----Original Message-----
>> >> From: Daniel Povey [mailto:dp...@gm...]
>> >> Perhaps you could explain how this is done?
>> >> It might help us make an informed decision if you could give us some
>> >> details on the workflow that you use, at the level of specific
>> >> commands.
>> >
>> > Let me set up a pair of cloned demo repositories to show different
>> practices, as time permits. Give me a day or two.
>> >
>> >  -kkm
>> >
>> ------------------------------------------------------------------------------
>> > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>> > Develop your own process in accordance with the BPMN 2 standard
>> > Learn Process modeling best practices with Bonita BPM through live
>> exercises
>> > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>> event?utm_
>> > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>> > _______________________________________________
>> > Kaldi-developers mailing list
>> > Kal...@li...
>> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>>
>>
>>
>> --
>> Sincerely, Alexander
>>
>>
>> ------------------------------------------------------------------------------
>> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>> Develop your own process in accordance with the BPMN 2 standard
>> Learn Process modeling best practices with Bonita BPM through live
>> exercises
>> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>> event?utm_
>> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>> _______________________________________________
>> Kaldi-developers mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>>
>
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>
>


-- 
Ondřej Plátek, +420 737 758 650, skype:ondrejplatek, ond...@gm...

Re: [Kaldi-developers] Testing Github repository

[Vassil P. <vas...@gm...>]

FWIW +1 from me to the migration to Git. Don't know if that will really
encourage people to contribute more, but some say the ease of creating pull
requests is helpful... Having a "Kaldi" GitHub organization may also
increase the visibility of projects, built on top of Kaldi(e.g. Tanel
Alumae's ASR server), or related to Kaldi(like PDNN or the Python wrappers
that were discussed here not long ago). GitHub's support for organizations
is really basic (AFAIK they don't even support following an organization),
but hopefully they will (finally) improve that.

IMHO the workflow suggested by Kirill, where everyone submits pull requests
to the "official" repository, that are merged (possibly after a review)
seems good. I don't think the branching will be a problem. As I understand
it, in the proposed workflow the history of the official repo will be
entirely linear(fast forward commits into "master"). I've mostly used Git
with single-user(i.e. me) repositories so far, but AFAIK it only becomes
problematic if you rewrite the history by rebasing or amending commits in a
branch that is jointly worked on with others. And even if that happens it
should be easy to recover, because with Git everyone has the full project
history on his desktop.

It's not very clear to me if this transition period of few months is needed
and how it would be helpful if people are still using only SVN? I'd say if
everyone agrees that Git is the way to go, the switch can be made as soon
as the process of switching and the new workflow are documented. From the
user's viewpoint it probably won't matter much if they type "svn up" or
"git pull origin master". Also, I agree with Kirill that keeping SVN in
sync after the switch could be probably an unnecessary hassle. Having a
backup is obviously not a problem, because in couple of days of migration
to Git, there will be enough full copies of the history distributed around
the world to survive a dinosaur extinction- causing event (even though the
usefulness of Kaldi in the aftermath is uncertain...).

I haven't been involved with such migrations though, so maybe I'm
overlooking something...

Vassil


On Tue, Apr 7, 2015 at 9:12 AM, Alexander Solovets <aso...@gm...>
wrote:

> Jan, I don't know how Nikolay does it, but I simply download and
> apply the diff, and then run `git svn dcommit`. I know that you can
> also accept the pull request into a custom branch (i.e. non-master).
> In this case you can do it via GitHub's UI, and then rebase that
> branch on top of master and push changes.
>
> Kirill, by "branching hell" I meant non-linear history with a lot of
> "merge" points. You not only loose the linearity, but also the merge
> conflict resolution becomes way more complex, the chance to
> accidentally drop someone else's changes increases. I do know about
> variety of git workflows and different techniques of accepting new
> commits, and I'm not going to argue, I just wanted to share my
> experience.
>
> On Tue, Apr 7, 2015 at 1:47 AM, Kirill Katsnelson
> <kir...@sm...> wrote:
> >> -----Original Message-----
> >> From: Daniel Povey [mailto:dp...@gm...]
> >> Perhaps you could explain how this is done?
> >> It might help us make an informed decision if you could give us some
> >> details on the workflow that you use, at the level of specific
> >> commands.
> >
> > Let me set up a pair of cloned demo repositories to show different
> practices, as time permits. Give me a day or two.
> >
> >  -kkm
> >
> ------------------------------------------------------------------------------
> > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> > Develop your own process in accordance with the BPMN 2 standard
> > Learn Process modeling best practices with Bonita BPM through live
> exercises
> > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> > _______________________________________________
> > Kaldi-developers mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>
>
>
> --
> Sincerely, Alexander
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>

Re: [Kaldi-developers] Testing Github repository

[Alexander S. <aso...@gm...>]

Jan, I don't know how Nikolay does it, but I simply download and
apply the diff, and then run `git svn dcommit`. I know that you can
also accept the pull request into a custom branch (i.e. non-master).
In this case you can do it via GitHub's UI, and then rebase that
branch on top of master and push changes.

Kirill, by "branching hell" I meant non-linear history with a lot of
"merge" points. You not only loose the linearity, but also the merge
conflict resolution becomes way more complex, the chance to
accidentally drop someone else's changes increases. I do know about
variety of git workflows and different techniques of accepting new
commits, and I'm not going to argue, I just wanted to share my
experience.

On Tue, Apr 7, 2015 at 1:47 AM, Kirill Katsnelson
<kir...@sm...> wrote:
>> -----Original Message-----
>> From: Daniel Povey [mailto:dp...@gm...]
>> Perhaps you could explain how this is done?
>> It might help us make an informed decision if you could give us some
>> details on the workflow that you use, at the level of specific
>> commands.
>
> Let me set up a pair of cloned demo repositories to show different practices, as time permits. Give me a day or two.
>
>  -kkm
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Kaldi-developers mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-developers



-- 
Sincerely, Alexander

Re: [Kaldi-developers] Testing Github repository

[Kirill K. <kir...@sm...>]

> -----Original Message-----
> From: Daniel Povey [mailto:dp...@gm...]
> Perhaps you could explain how this is done?
> It might help us make an informed decision if you could give us some
> details on the workflow that you use, at the level of specific
> commands.

Let me set up a pair of cloned demo repositories to show different practices, as time permits. Give me a day or two.

 -kkm

Re: [Kaldi-developers] Testing Github repository

[Kirill K. <kir...@sm...>]

> -----Original Message-----
> From: jt...@gm... [mailto:jt...@gm...] On Behalf Of Jan Trmal
> 
> The reason was that  if you select the project's licence during
> repository creation on github, github will commit some files into the
> repository (LICENCE file at least) and that messed up the commit
> history so that git svn rebase didn't work -- I had to do recursive
> merge every time before I called rebase and everything was getting out
> my control.

I think starting with an empty repo can be a better way to go. You can just git init a bare repo locally, do the svn fetch and then just git push the repo to github.

> * after that, we can make the git repository primary and instead of
> mirroring SVN->GIT we will mirror GIT->SVN 

You may also consider dropping SVN at that time (or returning to it and dropping git, if it goes horribly wrong). Maintaining both in synch will be troublesome, especially in cases where they lose synchronization for any reason.

 -kkm

Re: [Kaldi-developers] Testing Github repository

[Kirill K. <kir...@sm...>]

It is not immediately clear what do you call the "branching hell" (this emphatic name has been applied to a variety of conditions), but, looking at the way Kaldi is being developed, there seems no place for big branching screw-ups. All work basically happens on a single branch, and there is a multitude of private sandboxes, sending patches once in a while. This is already one step closer to a "branching hell" in one possible sense than the development workflow when these sandboxes are separated into users' private clones. This workflow is even more linear than the current Kaldi svn flow.

I would argue that it all comes to good housekeeping practices rather than to a stipulated inherent proneness of git to compounded branching conditions.

Handling 2 different SCS is indeed a problem. I really doubt that maintaining the content in 2 repositories is worth the effort in the long term. (And remember, this is the time that will not be spent on the main speech recognition work.) FWIW, the cmusphinx/sphinxtrain has seen 3 pull requests in the 5 months since the first one, of which 2 were approved and applied. Kaldi's rate of changes is two order of magnitude higher.

 -kkm

> -----Original Message-----
> From: Alexander Solovets [mailto:aso...@gm...]
> Sent: 2015-04-06 1300
> To: Jan Trmal
> Cc: Kirill Katsnelson; kal...@li...
> Subject: Re: [Kaldi-developers] Testing Github repository
> 
> I would recommend to stick to SVN as long as possible just because when
> the history is linear you can easily convert repository from one VCS to
> another. When the git branching hell comes you'll have to stay with it
> forever. On CMU Sphinx when a pull request is received, it actually
> goes to SVN first and then is closed with the revision number in the
> comment, like this https://github.com/cmusphinx/sphinxtrain/pull/4.
> 
> On Mon, Apr 6, 2015 at 8:04 PM, Jan Trmal <af...@ce...> wrote:
> > Kirill and all,
> > first, thanks for your comments and encouragement As it looks right
> > now, we will have only single, trunk, branch in the repository (we
> > might even delete the stable branch).
> > During a few past days, I have imported the kaldi repository once
> > more. The reason was that  if you select the project's licence during
> > repository creation on github, github will commit some files into the
> > repository (LICENCE file at least) and that messed up the commit
> > history so that git svn rebase didn't work -- I had to do recursive
> > merge every time before I called rebase and everything was getting
> out
> > my control. There might be easy fix for this, but the most
> > straightforward way for me was to run the conversion again. After
> > that, the svn rebase work smoothly (I have a cron job taking care of
> it now).
> >
> > During this week, I intend to do the conversion once more again and
> > hopefully for the last time (i.e. after that, we will have an
> official
> > kaldi git repository mirroring changes from SVN) The current
> tentative
> > plans for kaldi transition is:
> > * first, for a few months, operate the github kaldi repo as an
> > official mirror of svn. That means all commits will go to SVN and the
> > less adventurous users (and possibly companies with more inflexible
> > workflow) can keep using SVN. This is to give us time to sort out the
> > issues, write up the docs and get familiar with git
> > * after that, we can make the git repository primary and instead of
> > mirroring SVN->GIT we will mirror GIT->SVN (note: I still have to
> > figure out if this is feasible -- but I think it is). This should
> > -force- motivate all users to switch to GIT
> > * after that (or right after skipping the previous stage completely)
> > keep only git and stop using and supporting SVN.
> >
> > y.
> >
> >
> >
> >
> > On Mon, Apr 6, 2015 at 2:36 PM, Kirill Katsnelson
> > <kir...@sm...> wrote:
> >>
> >> Totally true. Depending on the type of workflow that Kaldi
> developers
> >> see, there are up to 3 cloned repositories involved. This is what
> >> happens in the project I maintain
> >> (https://github.com/fsprojects/FsLexYacc). We do not accept any
> other
> >> way to submit changes to the main repository (which is a bare repo
> in
> >> github) than using github's pull requests. Even people with write
> >> access to the repo must follow this path. Since the pull request may
> >> only be generated in github between 2 branches on repos necessarily
> >> stored on gihub (although the branches may be in different repos,
> >> having a common history fork in the past), the patch submitter must
> have cloned the main repo at some point. This is the clone #2.
> >>
> >> Since the submitter obviously has to do some hacking first, he
> clones
> >> his github clone #2 to a desktop, locally, and this is the clone #3.
> >> He normally pushes into a branch or the master "trunk," and, come
> >> time to send the changes upstream, issues a pull request for a diff
> >> between clones #1 and #2 (or 2 points in history in #2).
> >>
> >> The only tricky part here is merging upstream changes into this
> train
> >> of clones (analogous to "svn update") in such a way that the
> upstream
> >> changes would not show up in the resulting pull request.
> >>
> >> Another workflow would be allowing changes into the main trunk
> branch
> >> "the svn way." To me, the main benefit of having the pending commits
> >> (as pull
> >> requests) inspected by the peers is lost in this workflow. But the
> >> decision is on Kaldi dev's I any case.
> >>
> >> It perhaps makes sense to write up a walk-through for first time git
> >> (or
> >> github) users. I'd take on the task, but I need to play back the
> >> step-by-step as I write, so I need time for the task. A workflow
> >> decision must be done first, however. There might also be an
> existing
> >> guide on the 'Net for those migrating from svn into the selected git
> workflow.
> >>
> >>  -kkm
> >>
> >> > -----Original Message-----
> >> > From: Phil Garner [mailto:Phi...@id...]
> >> > Sent: 2015-04-05 0622
> >> > To: kal...@li...
> >> > Subject: Re: [Kaldi-developers] Testing Github repository
> >> >
> >> > > So far, I pushed only the trunk and stable branch -- I'm not
> sure
> >> > > if import of all sandboxes makes sense.
> >> >
> >> > If I understand the concept of sandboxes well, then in git they
> may
> >> > better correspond to branches in user clones of the repository.
> >> > That is, the person responsible for the sandbox clones the trunk
> >> > repo and commits to a branch as they wish.  Github makes this very
> easy.
> >> >
> >> > --
> >> > Phil Garner
> >> > http://www.idiap.ch/~pgarner
> >> >
> >> >
> >> > ------------------------------------------------------------------
> -
> >> > ----
> >> > -------
> >> > Dive into the World of Parallel Programming The Go Parallel
> >> > Website, sponsored by Intel and developed in partnership with
> >> > Slashdot Media, is your hub for all things parallel software
> >> > development, from weekly thought leadership blogs to news, videos,
> >> > case studies, tutorials and more. Take a look and join the
> >> > conversation now. http://goparallel.sourceforge.net/
> >> > _______________________________________________
> >> > Kaldi-developers mailing list
> >> > Kal...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >>
> >> --------------------------------------------------------------------
> -
> >> --------- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM
> >> EDT Develop your own process in accordance with the BPMN 2 standard
> >> Learn Process modeling best practices with Bonita BPM through live
> >> exercises
> >> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> >> event?utm_
> >>
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_S
> >> F _______________________________________________
> >> Kaldi-developers mailing list
> >> Kal...@li...
> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >
> >
> >
> > ---------------------------------------------------------------------
> -
> > -------- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> > Develop your own process in accordance with the BPMN 2 standard Learn
> > Process modeling best practices with Bonita BPM through live
> exercises
> > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> > event?utm_
> >
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> > _______________________________________________
> > Kaldi-developers mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >
> 
> 
> 
> --
> Sincerely, Alexander

Re: [Kaldi-developers] Testing Github repository

[Jan T. <af...@ce...>]

Or maybe let me phrase the question differently: how much automatic can it
be done  (and I realize that we might not want to do it fully automatic)
and how is it done :)
y.

On Mon, Apr 6, 2015 at 4:16 PM, Jan Trmal <af...@ce...> wrote:

> Alexander, can you send show us (i.e. how this is done on command-line
> level) how this is done?
> y.
>
> On Mon, Apr 6, 2015 at 3:59 PM, Alexander Solovets <aso...@gm...>
> wrote:
>
>> I would recommend to stick to SVN as long as possible just because
>> when the history is linear you can easily convert repository from one
>> VCS to another. When the git branching hell comes you'll have to stay
>> with it forever. On CMU Sphinx when a pull request is received, it
>> actually goes to SVN first and then is closed with the revision number
>> in the comment, like this
>> https://github.com/cmusphinx/sphinxtrain/pull/4.
>>
>> On Mon, Apr 6, 2015 at 8:04 PM, Jan Trmal <af...@ce...> wrote:
>> > Kirill and all,
>> > first, thanks for your comments and encouragement
>> > As it looks right now, we will have only single, trunk, branch in the
>> > repository (we might even delete the stable branch).
>> > During a few past days, I have imported the kaldi repository once more.
>> The
>> > reason was that  if you select the project's licence during repository
>> > creation on github, github will commit some files into the repository
>> > (LICENCE file at least) and that messed up the commit history so that
>> git
>> > svn rebase didn't work -- I had to do recursive merge every time before
>> I
>> > called rebase and everything was getting out my control. There might be
>> easy
>> > fix for this, but the most straightforward way for me was to run the
>> > conversion again. After that, the svn rebase work smoothly (I have a
>> cron
>> > job taking care of it now).
>> >
>> > During this week, I intend to do the conversion once more again and
>> > hopefully for the last time (i.e. after that, we will have an official
>> kaldi
>> > git repository mirroring changes from SVN)
>> > The current tentative plans for kaldi transition is:
>> > * first, for a few months, operate the github kaldi repo as an official
>> > mirror of svn. That means all commits will go to SVN and the less
>> > adventurous users (and possibly companies with more inflexible
>> workflow) can
>> > keep using SVN. This is to give us time to sort out the issues, write
>> up the
>> > docs and get familiar with git
>> > * after that, we can make the git repository primary and instead of
>> > mirroring SVN->GIT we will mirror GIT->SVN (note: I still have to
>> figure out
>> > if this is feasible -- but I think it is). This should -force- motivate
>> all
>> > users to switch to GIT
>> > * after that (or right after skipping the previous stage completely)
>> keep
>> > only git and stop using and supporting SVN.
>> >
>> > y.
>> >
>> >
>> >
>> >
>> > On Mon, Apr 6, 2015 at 2:36 PM, Kirill Katsnelson
>> > <kir...@sm...> wrote:
>> >>
>> >> Totally true. Depending on the type of workflow that Kaldi developers
>> see,
>> >> there are up to 3 cloned repositories involved. This is what happens
>> in the
>> >> project I maintain (https://github.com/fsprojects/FsLexYacc). We do
>> not
>> >> accept any other way to submit changes to the main repository (which
>> is a
>> >> bare repo in github) than using github's pull requests. Even people
>> with
>> >> write access to the repo must follow this path. Since the pull request
>> may
>> >> only be generated in github between 2 branches on repos necessarily
>> stored
>> >> on gihub (although the branches may be in different repos, having a
>> common
>> >> history fork in the past), the patch submitter must have cloned the
>> main
>> >> repo at some point. This is the clone #2.
>> >>
>> >> Since the submitter obviously has to do some hacking first, he clones
>> his
>> >> github clone #2 to a desktop, locally, and this is the clone #3. He
>> normally
>> >> pushes into a branch or the master "trunk," and, come time to send the
>> >> changes upstream, issues a pull request for a diff between clones #1
>> and #2
>> >> (or 2 points in history in #2).
>> >>
>> >> The only tricky part here is merging upstream changes into this train
>> of
>> >> clones (analogous to "svn update") in such a way that the upstream
>> changes
>> >> would not show up in the resulting pull request.
>> >>
>> >> Another workflow would be allowing changes into the main trunk branch
>> "the
>> >> svn way." To me, the main benefit of having the pending commits (as
>> pull
>> >> requests) inspected by the peers is lost in this workflow. But the
>> decision
>> >> is on Kaldi dev's I any case.
>> >>
>> >> It perhaps makes sense to write up a walk-through for first time git
>> (or
>> >> github) users. I'd take on the task, but I need to play back the
>> >> step-by-step as I write, so I need time for the task. A workflow
>> decision
>> >> must be done first, however. There might also be an existing guide on
>> the
>> >> 'Net for those migrating from svn into the selected git workflow.
>> >>
>> >>  -kkm
>> >>
>> >> > -----Original Message-----
>> >> > From: Phil Garner [mailto:Phi...@id...]
>> >> > Sent: 2015-04-05 0622
>> >> > To: kal...@li...
>> >> > Subject: Re: [Kaldi-developers] Testing Github repository
>> >> >
>> >> > > So far, I pushed only the trunk and stable branch -- I'm not sure
>> if
>> >> > > import of all sandboxes makes sense.
>> >> >
>> >> > If I understand the concept of sandboxes well, then in git they may
>> >> > better correspond to branches in user clones of the repository.  That
>> >> > is, the person responsible for the sandbox clones the trunk repo and
>> >> > commits to a branch as they wish.  Github makes this very easy.
>> >> >
>> >> > --
>> >> > Phil Garner
>> >> > http://www.idiap.ch/~pgarner
>> >> >
>> >> >
>> >> >
>> -----------------------------------------------------------------------
>> >> > -------
>> >> > Dive into the World of Parallel Programming The Go Parallel Website,
>> >> > sponsored
>> >> > by Intel and developed in partnership with Slashdot Media, is your
>> hub
>> >> > for all
>> >> > things parallel software development, from weekly thought leadership
>> >> > blogs to
>> >> > news, videos, case studies, tutorials and more. Take a look and join
>> >> > the
>> >> > conversation now. http://goparallel.sourceforge.net/
>> >> > _______________________________________________
>> >> > Kaldi-developers mailing list
>> >> > Kal...@li...
>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>> >>
>> >>
>> ------------------------------------------------------------------------------
>> >> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>> >> Develop your own process in accordance with the BPMN 2 standard
>> >> Learn Process modeling best practices with Bonita BPM through live
>> >> exercises
>> >> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>> >> event?utm_
>> >> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>> >> _______________________________________________
>> >> Kaldi-developers mailing list
>> >> Kal...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>> >
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>> > Develop your own process in accordance with the BPMN 2 standard
>> > Learn Process modeling best practices with Bonita BPM through live
>> exercises
>> > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
>> event?utm_
>> > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>> > _______________________________________________
>> > Kaldi-developers mailing list
>> > Kal...@li...
>> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
>> >
>>
>>
>>
>> --
>> Sincerely, Alexander
>>
>
>

Re: [Kaldi-developers] Testing Github repository

[Jan T. <af...@ce...>]

Alexander, can you send show us (i.e. how this is done on command-line
level) how this is done?
y.

On Mon, Apr 6, 2015 at 3:59 PM, Alexander Solovets <aso...@gm...>
wrote:

> I would recommend to stick to SVN as long as possible just because
> when the history is linear you can easily convert repository from one
> VCS to another. When the git branching hell comes you'll have to stay
> with it forever. On CMU Sphinx when a pull request is received, it
> actually goes to SVN first and then is closed with the revision number
> in the comment, like this
> https://github.com/cmusphinx/sphinxtrain/pull/4.
>
> On Mon, Apr 6, 2015 at 8:04 PM, Jan Trmal <af...@ce...> wrote:
> > Kirill and all,
> > first, thanks for your comments and encouragement
> > As it looks right now, we will have only single, trunk, branch in the
> > repository (we might even delete the stable branch).
> > During a few past days, I have imported the kaldi repository once more.
> The
> > reason was that  if you select the project's licence during repository
> > creation on github, github will commit some files into the repository
> > (LICENCE file at least) and that messed up the commit history so that git
> > svn rebase didn't work -- I had to do recursive merge every time before I
> > called rebase and everything was getting out my control. There might be
> easy
> > fix for this, but the most straightforward way for me was to run the
> > conversion again. After that, the svn rebase work smoothly (I have a cron
> > job taking care of it now).
> >
> > During this week, I intend to do the conversion once more again and
> > hopefully for the last time (i.e. after that, we will have an official
> kaldi
> > git repository mirroring changes from SVN)
> > The current tentative plans for kaldi transition is:
> > * first, for a few months, operate the github kaldi repo as an official
> > mirror of svn. That means all commits will go to SVN and the less
> > adventurous users (and possibly companies with more inflexible workflow)
> can
> > keep using SVN. This is to give us time to sort out the issues, write up
> the
> > docs and get familiar with git
> > * after that, we can make the git repository primary and instead of
> > mirroring SVN->GIT we will mirror GIT->SVN (note: I still have to figure
> out
> > if this is feasible -- but I think it is). This should -force- motivate
> all
> > users to switch to GIT
> > * after that (or right after skipping the previous stage completely) keep
> > only git and stop using and supporting SVN.
> >
> > y.
> >
> >
> >
> >
> > On Mon, Apr 6, 2015 at 2:36 PM, Kirill Katsnelson
> > <kir...@sm...> wrote:
> >>
> >> Totally true. Depending on the type of workflow that Kaldi developers
> see,
> >> there are up to 3 cloned repositories involved. This is what happens in
> the
> >> project I maintain (https://github.com/fsprojects/FsLexYacc). We do not
> >> accept any other way to submit changes to the main repository (which is
> a
> >> bare repo in github) than using github's pull requests. Even people with
> >> write access to the repo must follow this path. Since the pull request
> may
> >> only be generated in github between 2 branches on repos necessarily
> stored
> >> on gihub (although the branches may be in different repos, having a
> common
> >> history fork in the past), the patch submitter must have cloned the main
> >> repo at some point. This is the clone #2.
> >>
> >> Since the submitter obviously has to do some hacking first, he clones
> his
> >> github clone #2 to a desktop, locally, and this is the clone #3. He
> normally
> >> pushes into a branch or the master "trunk," and, come time to send the
> >> changes upstream, issues a pull request for a diff between clones #1
> and #2
> >> (or 2 points in history in #2).
> >>
> >> The only tricky part here is merging upstream changes into this train of
> >> clones (analogous to "svn update") in such a way that the upstream
> changes
> >> would not show up in the resulting pull request.
> >>
> >> Another workflow would be allowing changes into the main trunk branch
> "the
> >> svn way." To me, the main benefit of having the pending commits (as pull
> >> requests) inspected by the peers is lost in this workflow. But the
> decision
> >> is on Kaldi dev's I any case.
> >>
> >> It perhaps makes sense to write up a walk-through for first time git (or
> >> github) users. I'd take on the task, but I need to play back the
> >> step-by-step as I write, so I need time for the task. A workflow
> decision
> >> must be done first, however. There might also be an existing guide on
> the
> >> 'Net for those migrating from svn into the selected git workflow.
> >>
> >>  -kkm
> >>
> >> > -----Original Message-----
> >> > From: Phil Garner [mailto:Phi...@id...]
> >> > Sent: 2015-04-05 0622
> >> > To: kal...@li...
> >> > Subject: Re: [Kaldi-developers] Testing Github repository
> >> >
> >> > > So far, I pushed only the trunk and stable branch -- I'm not sure if
> >> > > import of all sandboxes makes sense.
> >> >
> >> > If I understand the concept of sandboxes well, then in git they may
> >> > better correspond to branches in user clones of the repository.  That
> >> > is, the person responsible for the sandbox clones the trunk repo and
> >> > commits to a branch as they wish.  Github makes this very easy.
> >> >
> >> > --
> >> > Phil Garner
> >> > http://www.idiap.ch/~pgarner
> >> >
> >> >
> >> >
> -----------------------------------------------------------------------
> >> > -------
> >> > Dive into the World of Parallel Programming The Go Parallel Website,
> >> > sponsored
> >> > by Intel and developed in partnership with Slashdot Media, is your hub
> >> > for all
> >> > things parallel software development, from weekly thought leadership
> >> > blogs to
> >> > news, videos, case studies, tutorials and more. Take a look and join
> >> > the
> >> > conversation now. http://goparallel.sourceforge.net/
> >> > _______________________________________________
> >> > Kaldi-developers mailing list
> >> > Kal...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >>
> >>
> ------------------------------------------------------------------------------
> >> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> >> Develop your own process in accordance with the BPMN 2 standard
> >> Learn Process modeling best practices with Bonita BPM through live
> >> exercises
> >> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> >> event?utm_
> >> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> >> _______________________________________________
> >> Kaldi-developers mailing list
> >> Kal...@li...
> >> https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> > Develop your own process in accordance with the BPMN 2 standard
> > Learn Process modeling best practices with Bonita BPM through live
> exercises
> > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> > _______________________________________________
> > Kaldi-developers mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-developers
> >
>
>
>
> --
> Sincerely, Alexander
>