Re: [Kaldi-users] FST minimization of LMs

[Lahiru S. <lah...@gm...>]

Thanks a lot for the clarification Dan.


On Fri, Dec 20, 2013 at 1:04 PM, Daniel Povey <dp...@gm...> wrote:

> No, because for each path there will be multiple alternatives in B and the
> sum of the path weights will be more than one.  I recommend to address
> further questions on this topic to www.openfst.org.
> Dan
>
>
>
> On Thu, Dec 19, 2013 at 8:53 PM, Lahiru Samarakoon <lah...@gm...>wrote:
>
>> Is this also applies to the log semiring?
>> Thanks
>> Lahiru
>>
>>
>> On Fri, Dec 20, 2013 at 12:46 PM, Daniel Povey <dp...@gm...> wrote:
>>
>>>
>>> I am trying to decompose weighted acceptors (Language models) . I
>>>> observe this behavior when I compose them to build a new fst. I think I am
>>>> doing something wrong here.
>>>>
>>>> Please be kind enough to clarify me the following.
>>>>
>>>> Given a *weighted acceptor *"*A*",  lets say acceptor "*B*" is created
>>>> by setting all the weights of "A" to *zero *including the final states
>>>> weights.
>>>> is
>>>> *"A compose B"  equivalent to "A"?*
>>>>
>>>
>>> If you are in the tropical semiring, then yes, because any path in A is
>>> also in B (with zero cost).  But "A compose B" is a wasteful way of
>>> representing the acceptor because there are multiple alternative paths
>>> through B that will end up multiplying the size, potentially by the n-gram
>>> order minus one.  So it's equivalent in the FST sense, but it's much
>>> larger.
>>>
>>> Dan
>>>
>>>
>>>
>>>>
>>>> if the answer is not  equivalent, could you state the reason briefly
>>>>
>>>> Thank you,
>>>>
>>>> Best Regards,
>>>> Lahiru
>>>>
>>>>
>>>> On Fri, Dec 20, 2013 at 3:41 AM, Al Zatv <al...@gm...> wrote:
>>>>
>>>>> Hello! Actually, I don't have such a problem. I think Lahiru
>>>>> Samarakoon, the topic starter, have it:
>>>>> Quote:
>>>>> =====
>>>>>
>>>>> I have my own language model. When I minimize it, actually it
>>>>> increases the number of states and arcs.
>>>>>
>>>>> How can this be possible?
>>>>> I have created the LM in arpa format and used scripts from Kaldi to
>>>>> create the G.fst
>>>>> =====
>>>>>
>>>>> Usually ARPA LMs are acceptors and they are determinized by default,
>>>>> so if Lahiru really observe that min(G.fst) is bigger than G.fst, it is
>>>>> suspicious.
>>>>> 19.12.2013 20:09 пользователь "Michael Riley" <ri...@go...>
>>>>> написал:
>>>>>
>>>>> Not sure what exactly you are doing or what code you are calling but
>>>>>> if you are using the algorithms in OpenFst then
>>>>>> |det(A)| can be > that |A| but |min(det(A))| <= |det(A)| if A is an
>>>>>> acceptor.
>>>>>>
>>>>>> I'd be surprised if there is a bug here (given this condition I
>>>>>> believe is tested on millions of randomly generated machines) but If you
>>>>>> have a counter-example when using unmodified OpenFst as above, then send it
>>>>>> to us along with the precise commands you ran that demonstrate the problem.
>>>>>>
>>>>>> -m
>>>>>>
>>>>>> -m
>>>>>>
>>>>>>
>>>>>> On Thu, Dec 19, 2013 at 2:55 AM, Al Zatv <al...@gm...> wrote:
>>>>>>
>>>>>>> According to this document, size of transducer can be larger than
>>>>>>> original. But ARPA LM is an acceptor, so it must have optimal number of
>>>>>>> states, or I don't misunderstood something?
>>>>>>> 19.12.2013 10:23 пользователь "Michael Riley" <ri...@go...>
>>>>>>> написал:
>>>>>>>
>>>>>>> http://www.openfst.org/twiki/bin/view/FST/MinimizeDoc
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Dec 19, 2013 at 1:13 AM, Lahiru Samarakoon <
>>>>>>>> lah...@gm...> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> HI All,
>>>>>>>>>
>>>>>>>>> I have my own language model. When I minimize it, actually it
>>>>>>>>> increases the number of states and arcs.
>>>>>>>>>
>>>>>>>>> How can this be possible?
>>>>>>>>> I have created the LM in arpa format and used scripts from Kaldi
>>>>>>>>> to create the G.fst.
>>>>>>>>>
>>>>>>>>> Thank you,
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Lahiru
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>> Rapidly troubleshoot problems before they affect your business.
>>>>>>>>> Most IT
>>>>>>>>> organizations don't have a clear picture of how application
>>>>>>>>> performance
>>>>>>>>> affects their revenue. With AppDynamics, you get 100% visibility
>>>>>>>>> into your
>>>>>>>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>>>>>>>> AppDynamics Pro!
>>>>>>>>>
>>>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>>>>>>>>> _______________________________________________
>>>>>>>>> Kaldi-users mailing list
>>>>>>>>> Kal...@li...
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Rapidly troubleshoot problems before they affect your business.
>>>>>>>> Most IT
>>>>>>>> organizations don't have a clear picture of how application
>>>>>>>> performance
>>>>>>>> affects their revenue. With AppDynamics, you get 100% visibility
>>>>>>>> into your
>>>>>>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>>>>>>> AppDynamics Pro!
>>>>>>>>
>>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>>>>>>>> _______________________________________________
>>>>>>>> Kaldi-users mailing list
>>>>>>>> Kal...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Rapidly troubleshoot problems before they affect your business. Most IT
>>>> organizations don't have a clear picture of how application performance
>>>> affects their revenue. With AppDynamics, you get 100% visibility into
>>>> your
>>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>>> AppDynamics Pro!
>>>>
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>>>> _______________________________________________
>>>> Kaldi-users mailing list
>>>> Kal...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>
>>>>
>>>
>>
>