Re: [Kaldi-users] FST minimization of LMs

[Lahiru S. <lah...@gm...>]

Please help me with this one last query.
If the kaldi program "fstisstochastic"  gives significantly different
values for two fsts, is it safe to say those are not equivalent?

For example,
fstisstochastic A.fst
*0.693747*      -0.877091

fstisstochastic B.fst
*24.1318*      -0.877091

Thanks in advance,
Lahiru


On Fri, Dec 20, 2013 at 1:08 PM, Lahiru Samarakoon <lah...@gm...>wrote:

> Thanks a lot for the clarification Dan.
>
>
> On Fri, Dec 20, 2013 at 1:04 PM, Daniel Povey <dp...@gm...> wrote:
>
>> No, because for each path there will be multiple alternatives in B and
>> the sum of the path weights will be more than one.  I recommend to address
>> further questions on this topic to www.openfst.org.
>> Dan
>>
>>
>>
>> On Thu, Dec 19, 2013 at 8:53 PM, Lahiru Samarakoon <lah...@gm...>wrote:
>>
>>> Is this also applies to the log semiring?
>>> Thanks
>>> Lahiru
>>>
>>>
>>> On Fri, Dec 20, 2013 at 12:46 PM, Daniel Povey <dp...@gm...> wrote:
>>>
>>>>
>>>> I am trying to decompose weighted acceptors (Language models) . I
>>>>> observe this behavior when I compose them to build a new fst. I think I am
>>>>> doing something wrong here.
>>>>>
>>>>> Please be kind enough to clarify me the following.
>>>>>
>>>>> Given a *weighted acceptor *"*A*",  lets say acceptor "*B*" is
>>>>> created by setting all the weights of "A" to *zero *including the
>>>>> final states weights.
>>>>> is
>>>>> *"A compose B"  equivalent to "A"?*
>>>>>
>>>>
>>>> If you are in the tropical semiring, then yes, because any path in A is
>>>> also in B (with zero cost).  But "A compose B" is a wasteful way of
>>>> representing the acceptor because there are multiple alternative paths
>>>> through B that will end up multiplying the size, potentially by the n-gram
>>>> order minus one.  So it's equivalent in the FST sense, but it's much
>>>> larger.
>>>>
>>>> Dan
>>>>
>>>>
>>>>
>>>>>
>>>>> if the answer is not  equivalent, could you state the reason briefly
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Best Regards,
>>>>> Lahiru
>>>>>
>>>>>
>>>>> On Fri, Dec 20, 2013 at 3:41 AM, Al Zatv <al...@gm...> wrote:
>>>>>
>>>>>> Hello! Actually, I don't have such a problem. I think Lahiru
>>>>>> Samarakoon, the topic starter, have it:
>>>>>> Quote:
>>>>>> =====
>>>>>>
>>>>>> I have my own language model. When I minimize it, actually it
>>>>>> increases the number of states and arcs.
>>>>>>
>>>>>> How can this be possible?
>>>>>> I have created the LM in arpa format and used scripts from Kaldi to
>>>>>> create the G.fst
>>>>>> =====
>>>>>>
>>>>>> Usually ARPA LMs are acceptors and they are determinized by default,
>>>>>> so if Lahiru really observe that min(G.fst) is bigger than G.fst, it is
>>>>>> suspicious.
>>>>>> 19.12.2013 20:09 пользователь "Michael Riley" <ri...@go...>
>>>>>> написал:
>>>>>>
>>>>>> Not sure what exactly you are doing or what code you are calling but
>>>>>>> if you are using the algorithms in OpenFst then
>>>>>>> |det(A)| can be > that |A| but |min(det(A))| <= |det(A)| if A is an
>>>>>>> acceptor.
>>>>>>>
>>>>>>> I'd be surprised if there is a bug here (given this condition I
>>>>>>> believe is tested on millions of randomly generated machines) but If you
>>>>>>> have a counter-example when using unmodified OpenFst as above, then send it
>>>>>>> to us along with the precise commands you ran that demonstrate the problem.
>>>>>>>
>>>>>>> -m
>>>>>>>
>>>>>>> -m
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Dec 19, 2013 at 2:55 AM, Al Zatv <al...@gm...> wrote:
>>>>>>>
>>>>>>>> According to this document, size of transducer can be larger than
>>>>>>>> original. But ARPA LM is an acceptor, so it must have optimal number of
>>>>>>>> states, or I don't misunderstood something?
>>>>>>>> 19.12.2013 10:23 пользователь "Michael Riley" <ri...@go...>
>>>>>>>> написал:
>>>>>>>>
>>>>>>>> http://www.openfst.org/twiki/bin/view/FST/MinimizeDoc
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Dec 19, 2013 at 1:13 AM, Lahiru Samarakoon <
>>>>>>>>> lah...@gm...> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> HI All,
>>>>>>>>>>
>>>>>>>>>> I have my own language model. When I minimize it, actually it
>>>>>>>>>> increases the number of states and arcs.
>>>>>>>>>>
>>>>>>>>>> How can this be possible?
>>>>>>>>>> I have created the LM in arpa format and used scripts from Kaldi
>>>>>>>>>> to create the G.fst.
>>>>>>>>>>
>>>>>>>>>> Thank you,
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Lahiru
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>> Rapidly troubleshoot problems before they affect your business.
>>>>>>>>>> Most IT
>>>>>>>>>> organizations don't have a clear picture of how application
>>>>>>>>>> performance
>>>>>>>>>> affects their revenue. With AppDynamics, you get 100% visibility
>>>>>>>>>> into your
>>>>>>>>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>>>>>>>>> AppDynamics Pro!
>>>>>>>>>>
>>>>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Kaldi-users mailing list
>>>>>>>>>> Kal...@li...
>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>> Rapidly troubleshoot problems before they affect your business.
>>>>>>>>> Most IT
>>>>>>>>> organizations don't have a clear picture of how application
>>>>>>>>> performance
>>>>>>>>> affects their revenue. With AppDynamics, you get 100% visibility
>>>>>>>>> into your
>>>>>>>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>>>>>>>> AppDynamics Pro!
>>>>>>>>>
>>>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>>>>>>>>> _______________________________________________
>>>>>>>>> Kaldi-users mailing list
>>>>>>>>> Kal...@li...
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Rapidly troubleshoot problems before they affect your business. Most IT
>>>>> organizations don't have a clear picture of how application performance
>>>>> affects their revenue. With AppDynamics, you get 100% visibility into
>>>>> your
>>>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>>>> AppDynamics Pro!
>>>>>
>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>>>>> _______________________________________________
>>>>> Kaldi-users mailing list
>>>>> Kal...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>>
>>>>>
>>>>
>>>
>>
>