Re: [Kaldi-users] FST minimization of LMs

[Lahiru S. <lah...@gm...>]

Is this also applies to the log semiring?
Thanks
Lahiru


On Fri, Dec 20, 2013 at 12:46 PM, Daniel Povey <dp...@gm...> wrote:

>
> I am trying to decompose weighted acceptors (Language models) . I observe
>> this behavior when I compose them to build a new fst. I think I am doing
>> something wrong here.
>>
>> Please be kind enough to clarify me the following.
>>
>> Given a *weighted acceptor *"*A*",  lets say acceptor "*B*" is created
>> by setting all the weights of "A" to *zero *including the final states
>> weights.
>> is
>> *"A compose B"  equivalent to "A"?*
>>
>
> If you are in the tropical semiring, then yes, because any path in A is
> also in B (with zero cost).  But "A compose B" is a wasteful way of
> representing the acceptor because there are multiple alternative paths
> through B that will end up multiplying the size, potentially by the n-gram
> order minus one.  So it's equivalent in the FST sense, but it's much
> larger.
>
> Dan
>
>
>
>>
>> if the answer is not  equivalent, could you state the reason briefly
>>
>> Thank you,
>>
>> Best Regards,
>> Lahiru
>>
>>
>> On Fri, Dec 20, 2013 at 3:41 AM, Al Zatv <al...@gm...> wrote:
>>
>>> Hello! Actually, I don't have such a problem. I think Lahiru Samarakoon,
>>> the topic starter, have it:
>>> Quote:
>>> =====
>>>
>>> I have my own language model. When I minimize it, actually it increases
>>> the number of states and arcs.
>>>
>>> How can this be possible?
>>> I have created the LM in arpa format and used scripts from Kaldi to
>>> create the G.fst
>>> =====
>>>
>>> Usually ARPA LMs are acceptors and they are determinized by default, so
>>> if Lahiru really observe that min(G.fst) is bigger than G.fst, it is
>>> suspicious.
>>> 19.12.2013 20:09 пользователь "Michael Riley" <ri...@go...>
>>> написал:
>>>
>>> Not sure what exactly you are doing or what code you are calling but if
>>>> you are using the algorithms in OpenFst then
>>>> |det(A)| can be > that |A| but |min(det(A))| <= |det(A)| if A is an
>>>> acceptor.
>>>>
>>>> I'd be surprised if there is a bug here (given this condition I believe
>>>> is tested on millions of randomly generated machines) but If you have a
>>>> counter-example when using unmodified OpenFst as above, then send it to us
>>>> along with the precise commands you ran that demonstrate the problem.
>>>>
>>>> -m
>>>>
>>>> -m
>>>>
>>>>
>>>> On Thu, Dec 19, 2013 at 2:55 AM, Al Zatv <al...@gm...> wrote:
>>>>
>>>>> According to this document, size of transducer can be larger than
>>>>> original. But ARPA LM is an acceptor, so it must have optimal number of
>>>>> states, or I don't misunderstood something?
>>>>> 19.12.2013 10:23 пользователь "Michael Riley" <ri...@go...>
>>>>> написал:
>>>>>
>>>>> http://www.openfst.org/twiki/bin/view/FST/MinimizeDoc
>>>>>>
>>>>>>
>>>>>> On Thu, Dec 19, 2013 at 1:13 AM, Lahiru Samarakoon <
>>>>>> lah...@gm...> wrote:
>>>>>>
>>>>>>>
>>>>>>> HI All,
>>>>>>>
>>>>>>> I have my own language model. When I minimize it, actually it
>>>>>>> increases the number of states and arcs.
>>>>>>>
>>>>>>> How can this be possible?
>>>>>>> I have created the LM in arpa format and used scripts from Kaldi to
>>>>>>> create the G.fst.
>>>>>>>
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Lahiru
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Rapidly troubleshoot problems before they affect your business. Most
>>>>>>> IT
>>>>>>> organizations don't have a clear picture of how application
>>>>>>> performance
>>>>>>> affects their revenue. With AppDynamics, you get 100% visibility
>>>>>>> into your
>>>>>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>>>>>> AppDynamics Pro!
>>>>>>>
>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>>>>>>> _______________________________________________
>>>>>>> Kaldi-users mailing list
>>>>>>> Kal...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Rapidly troubleshoot problems before they affect your business. Most
>>>>>> IT
>>>>>> organizations don't have a clear picture of how application
>>>>>> performance
>>>>>> affects their revenue. With AppDynamics, you get 100% visibility into
>>>>>> your
>>>>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>>>>> AppDynamics Pro!
>>>>>>
>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>>>>>> _______________________________________________
>>>>>> Kaldi-users mailing list
>>>>>> Kal...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>>>
>>>>>>
>>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business. Most IT
>> organizations don't have a clear picture of how application performance
>> affects their revenue. With AppDynamics, you get 100% visibility into your
>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
>> Pro!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>
>>
>