performance of current models

[georgi georgiev]

Dear,

I was unable to find any information on what is the performance of the model: located at the model directory of open nlp svn. Can someone point me to evaluation analysis etc.?

Thanks

[Thomas Morton]

Hi,
  I have a white paper that has some of this information, but I haven't gotten it close to done yet.  Can you tell me specifically what models you are looking for information on? 

I have MUC 6 and 7 named-entity and coreference numbers, and parse-eval numbers for the parser. 

Thanks...Tom

[georgi georgiev]

Hi Tom,

In fact I was wondering is there a resource where I can check the evaluation, numbers with some discussion etc. My original idea was to compare ME modeling to others.
I am interested in NE, pos, co-ref., chunking and parsing. In case having the numbers for tokenizer and sentence split It would be great to share.

Thanks a lot...

[Thomas Morton]

This is all for English data.  these are pretty standard evaluation sets in the academic literature, but let me know if you need clarification.

POS Tagger
00       44851/46451 (96.56%)
Brown 19058/19377 (98.35%)
23       54880/56684 (96.82%)
Total    118789/122512 (96.96%)

Named Entity
muc6 train 76.82% 81/73
muc6 test 78.73% 83/75
muc 7 train 88.61% 87/90
muc 7 test 83.48% 94/75

Parsing: parse eval P/R <40
00       88.58/88.66
23       87.56/88.01
Brown 85.94/86.6

Coref
muc6 dev     59.7 77.2/48.7
muc7 dev     56.1 76.5/44.3
muc6 formal 64.0 76.7/54.9
muc7 formal 58.4 79.4/46.4

Hope this helps...Tom