Sphinx3 and n-best lists

[Tanel Alumäe]

I have some questions about n-best lists generated with sphinx 3.x (from CVS).

Each N-best hypothesis begins with a prefix like:

T -50957380 A -39061703 L -1206956 ...

I assume the number following T is the total score (in log_1.0003). The
problem is that often, some hypothesis down in the N-best list have better
scores than the 1st hypothesis. E.g., the 5th hypothesis of the same list:

T -50903709 A -39008032 L -1206956...

Is it normal?

Second, when I calculate A + lmw * L, I don't get T (the total score). What
else is in the total score? Is it the word insertion penalty? It could be,
since the sum of A + lmw * L + N_words * log_1.0003(wip) is almost the total
score, althoug not exactly. Does the sentence ending </s> also trigger
the word insertion penalty?

[Nickolay V. Shmyrev]

> What else is in the total score? Is it the word insertion penalty? It
could be, since the sum of

According to code it should be something like
A + lmw * (L + N_words * log_wip)

But I'm not sure. It's easier to look into C sources and check. The code is in
astar.c and in lm.c

> The problem is that often, some hypothesis down in the N-best list have
better scores than the 1st hypothesis.

During astart search the list is sorted by total score which is the sum of
path score and some "heuristic" score guessed as the score till the
end of the utterance. This pp->tscr is different from pp->pscr and this
may cause sorting issues I think. But I also wonder if there is a more precise
documentation on that.