Re: [Mat-annotation-users] 27.3994, Software: Computational Linguistics; Text/Corpus Linguistics: MITRE Annotation Toolkit 3.1

[Samuel B. <sa...@mi...>]

It's been a long time since someone begged me to use MAT :-).

There's no problem with the expressiveness of MAT's document model or 
annotation scheme - with the exception of discontinuous spans, which MAT 
doesn't do, it's at least as expressive as brat in every way. There's 
also no problem with viewing dependencies in the annotation table. My 
concerns are several:

- First, it's pretty clumsy to associate the document sentences with the 
relevant annotations in the table. I suppose you could get around this 
by having each sentence be a separate document.

- Second, I'm surprised that your annotators prefer to review 
dependencies in a table - when I've reviewed dependency annotations, 
I've found the arcs in the palette to be crucial to my figuring out what 
was going on. But to each his or her own, I suppose.

- Third, I think you'd find that editing the annotations would be 
extremely clumsy. If you don't care about doing that, this isn't an issue.

There's no on-line demo. But I can promise you that installing MAT is 
trivial, and if you want to check it out, by all means do so. However, 
you will need to spend some time to design your annotation task and 
convert your data into something MAT can read.

Cheers,
Sam

Alexandre Rademaker wrote:
> Hi Sam,
>
>
> I am curious to understand why you described the annotation of dependencies in MAT as a likely ugly experience. If I understood right, MAT was designed for annotation of NE and relations between them. But we can take the POS as entity types and the relationships between then as the dependencies, right? The Brat tool takes this approach in making any span of text a possible entity that can receive key/values annotations and can be linked to other entities via relations; that is why Brat can be used for NE and relations but also for dependencies and POS annotations.
>
> Reading the documentation (http://mat-annotation.sourceforge.net/current_docs/html/index.html) I saw that you could import/export in a JSON format. Moreover, the relations are not drawn in the sentence with arrows, but displayed as a table in the bottom of the screen; the funny part is that our team of linguists actually prefer to read the dependencies in the table format. Moreover, we would not annotate the documents from scratch, our use case is to review and correct dependencies.
>
> Is there any online demo? I would like to give MAT a chance?! ;-)
>
> Cheers,
>
> --
> Alexandre Rademaker
> http://arademaker.github.io <http://arademaker.github.io/>
> http://researcher.ibm.com/person/br-alexrad <http://researcher.ibm.com/person/br-alexrad>
>
>
>
>> On 5 Dec 2016, at 20:39, Livy Real <liv...@gm...> wrote:
>>
>> Hi Livy -
>>
>> Thanks for writing, and thanks for your interest in the toolkit. Alas, doing dependency annotation in MAT would be an ugly, ugly thing. I've wanted for quite a while to have a custom annotation palette that would support this, but I've had neither the opportunity or the resources to do it.
>>
>> If you do find a tool that suits your needs, I'd love to hear about it. One tool you might look into is brat (http://brat.nlplab.org <http://brat.nlplab.org/>). It's more appropriate than MAT because it supports drawing arcs directly on its annotation palette.
>>
>> Cordially,
>> Sam Bayer
>> The MITRE Corporation
>>
>> On Sun, Oct 9, 2016 at 10:25 AM, Alexandre Rademaker <ara...@gm... <mailto:ara...@gm...>> wrote:
>>
>>
>> Sent from my iPhone
>>
>> Begin forwarded message:
>>
>>> *****************    LINGUIST List Support    *****************
>>>                        Fund Drive 2016
>>>                    25 years of LINGUIST List!
>>> Please support the LL editors and operation with a donation at:
>>>            http://funddrive.linguistlist.org/donate/ <http://funddrive.linguistlist.org/donate/>
>>>
>>> Editor for this issue: Amanda Foster <am...@li... <mailto:am...@li...>>
>>> ================================================================
>>>
>>>
>>> Date: Fri, 07 Oct 2016 11:05:06
>>> From: Samuel Bayer [sa...@mi... <mailto:sa...@mi...>]
>>> Subject: Computational Linguistics; Text/Corpus Linguistics: MITRE Annotation Toolkit 3.1
>>>
>>>
>>> All -
>>>
>>> My colleagues and I are pleased to announce the availability of version 3.1 of
>>> the MITRE Annotation Toolkit.
>>>
>>> - What is it?
>>>
>>> MAT is a suite of cooperating, loosely-coupled tools for supporting the human
>>> annotator in constructing models and corpora for computational linguistics and
>>> analysis, especially using the iterative 'tag a little, learn a little'
>>> workflow. MAT includes MITRE's open source Java Carafe CRF trainer/tagger; a
>>> Web-based annotation and process control UI; a flexible and powerful scorer;
>>> an XML-configurable experiment harness; a workflow manager; and more.
>>>
>>> MAT is platform-independent, open source, and compatible (with some effort)
>>> with multiple trainer/tagger engines and hand annotation tools.
>>>
>>> You can learn about MAT 3.1, and download it, at
>>>
>>> http://mat-annotation.sf.net <http://mat-annotation.sf.net/>
>>>
>>> - What's new in version 3.1?
>>>
>>> This release introduces crossvalidation in the experiment engine; a much more
>>> streamlined way of declaring annotation sets; the ability to infer tasks in
>>> the UI from annotated documents; and significant UI enhancements, including a
>>> guided mode for filling in annotation popups and much better management of
>>> overlapping annotations. For additional details, visit the documentation at
>>>
>>> http://mat-annotation.sf.net/current_docs/html/index.html <http://mat-annotation.sf.net/current_docs/html/index.html>
>>>
>>> and select ''Upgrade and release notes'' from the documentation sidebar.
>>>
>>> If you have MAT 3.0, MAT 3.1 is completely backward compatible. If you have
>>> MAT 2.0, there are some conversions you may have to perform. See the release
>>> notes for details.
>>>
>>> - How stable is it?
>>>
>>> For many years, MAT has served as the underpinning of MITRE's Identification
>>> Scrubber Toolkit (MIST), an open-source package for deidentification of
>>> free-text medical documents. As MIST, it has been used by a wide range of
>>> researchers in this area. MAT itself has been used extensively by MITRE in its
>>> internal research and development projects, and by a variety of MITRE's
>>> sponsors.
>>>
>>> - Is it supported?
>>>
>>> Unfortunately, no. MITRE doesn't (currently) have the resources to provide
>>> public open-source support. There is a public mailing list on Sourceforge,
>>> which the MITRE developers subscribe to, but we likely won't have the
>>> resources to answer.
>>>
>>> On the other hand, MAT has been under development for almost ten years, and
>>> has seen a good deal of use, and comes with a ton of documentation.
>>>
>>> If you download MAT, and you like it, please let us know.
>>>
>>> Cordially,
>>> Sam Bayer
>>> The MITRE Corporation
>>> sa...@mi... <mailto:sa...@mi...>
>>>
>
>