Labeling Data

Help
2012-03-02
2013-04-26
  • catrambonep

    catrambonep - 2012-03-02

    Hello.  I am a newbie to NLP, and to Minor Third.  In reading through the LabelingAndLoadingDataTutorial, I cam across this statement:

    "Labels files are not meant to be created by hand. There are a few GUI tools in MinorThird that allow users to graphically see documents, add and/or edit labels, and save their work in a labels file. "

    I assume the TextBaseLabeler is used to only label "whole documents" (as stated in the tutorial).  So, what are the "few GUI tools in MinorThird that allow users to graphically see documents, add and/or edit labels, and save their work in a labels file. "?  Thanks in advance for any guidance you can offer.

    All I'm wanting to to at this point is label a simple text file that I am using to test with and to learn how MinorThird works.

    -Paul

     
  • catrambonep

    catrambonep - 2012-03-02

    I may have answered my own question……I found the TextBaseEditor and ran it.  It seems I can select the text file in the upper panel of the display and get it to show up in the lower panel.  From there, I select a subset of text and click "Add".  The text is highlighted in yellow.  I click "Props" and enter a property name called "date" and leave the property value blank (as I'm unsure what that field is for at the moment).  Then, I perform the same operation on another subset of text in the file, but when I click the "Props" button again (for the newly highlighted text), I'm presented with the list of properties I've defined (just one at this point), but I can't get that property to be linked to the second span of text I've highlighted.  The "save file" simply has this:

    setSpanProp SampleData.csv  0 23 date

    when it should really be:

    setSpanProp SampleData.csv  0 23 date
    setSpanProp SampleData.csv  25 50 date

    I'm sure I'm just missing something very basic here, but I just don't see it.

    Any tips you can offer would be greatly appreciated.

    Thanks!

    -Paul

     
  • Frank Lin

    Frank Lin - 2012-03-09

    Hi Paul,

    Sorry the TextBaseEditor gui is not very user-friendly and is a little buggy right now; generally we don't recommend people using it for labeling a large amount of data.

    With SpanProps if you must use the gui you can:
    1) highlight the span
    2) click "Add"
    3) click "Props"
    4) enter something in both fields at the bottom. for example, you can enter "POS" for property and "noun" or "verb" for property value.
    5) click "Insert"
    6) repeat the above for another label
    7) click "Save"  - now you should be able to see all the labels in the file.

    As you can see it's not as convenient as it should be - it was mostly used to examine and correct already labeled data. Hopefully we'll get the chance to fix this soon.

     
  • catrambonep

    catrambonep - 2012-03-13

    Thanks very much for the reply!  I have been making progress with the utility on a variety of fronts.  I'm currently knee (maybe neck…) deep in Mixup.  I think I have a question regarding the extraction, but I think I need to organize my thoughts a bit before posting.  Should be this week though.

    Thanks again. 
    -Paul

     
  • catrambonep

    catrambonep - 2012-03-15

    Ok, here is another question….Hopefully you'll see this since it is related to my original question, so  I'm simply adding new information to this thread instead of starting another one…..Here goes….

    It seems the only luck I'm having with visually showing (via any of the GUI tools) how a sample document has been marked up with annotations is if I wrap the data of interest in XML tags.  I cannot get any of the annotations that have been saved off to a .labels file to show up in any GUI presentation.  The reason I'm trying to use the GUI is because I need to show the results of a sample annotation exercise in a demo to other team members (some of which aren't very technical…..so pretty colors work best for those folks!).

    Am I chasing my own tail in trying to get the annotations in my .labels file to mix with my "manually-created" XML-tagged data?

    Thanks in advance for any suggestions you can offer.

    -Paul

     
  • Frank Lin

    Frank Lin - 2012-03-16

    Hi Paul,

    The GUI is not the greatest tool for adding annotations to text, but it should be able to show existing annotations in a .labels file. If it's not working for you, can you post or send me a sample of the .labels file that's not working?

     
  • catrambonep

    catrambonep - 2012-03-16

    Sure thing.  I couldn't see an easy way to email you the file, so here is the raw text:

    addToType SampleArcsiteLogDataTagged.csv 103 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 399 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 690 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 1025 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 1360 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 1695 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 2035 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 2318 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 2596 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 2844 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 3179 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 3467 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 3733 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 4068 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 4403 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 4738 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 5073 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 5413 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 5692 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 5971 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 6254 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 6715 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 6970 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 7246 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 7501 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 7836 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 8171 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 8506 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 8841 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 9114 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 9387 3 protocol
    addToType SampleArcsiteLogDataTagged.csv 9722 3 protocol
    closeType SampleArcsiteLogDataTagged.csv protocol
    addToType SampleArcsiteLogDataTagged.csv 48 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 142 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 154 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 166 12 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 179 12 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 205 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 227 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 344 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 438 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 450 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 462 12 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 475 12 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 501 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 523 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 640 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 734 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 747 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 772 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 794 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 975 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1069 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1082 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1107 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1129 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1310 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1404 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1417 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1442 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1464 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1645 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1739 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1752 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1777 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1799 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 1980 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2074 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2085 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2096 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2108 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2133 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2155 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2263 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2357 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2368 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2379 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2391 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2416 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2438 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2546 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2640 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2652 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2681 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2703 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2720 9 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2794 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2888 11 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2901 10 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2926 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 2948 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 3129 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 3223 12 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 3237 12 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 3264 14 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 14768 8 _mixupIP
    addToType SampleArcsiteLogDataTagged.csv 107 7 status
    addToType SampleArcsiteLogDataTagged.csv 403 7 status
    addToType SampleArcsiteLogDataTagged.csv 694 7 status
    addToType SampleArcsiteLogDataTagged.csv 1029 7 status
    addToType SampleArcsiteLogDataTagged.csv 1364 7 status
    addToType SampleArcsiteLogDataTagged.csv 1699 7 status
    addToType SampleArcsiteLogDataTagged.csv 2039 7 status
    addToType SampleArcsiteLogDataTagged.csv 2322 7 status
    addToType SampleArcsiteLogDataTagged.csv 2600 7 status
    addToType SampleArcsiteLogDataTagged.csv 2848 7 status
    addToType SampleArcsiteLogDataTagged.csv 3183 7 status
    addToType SampleArcsiteLogDataTagged.csv 3471 7 status
    addToType SampleArcsiteLogDataTagged.csv 3737 7 status
    addToType SampleArcsiteLogDataTagged.csv 4072 7 status
    addToType SampleArcsiteLogDataTagged.csv 4407 7 status
    addToType SampleArcsiteLogDataTagged.csv 4742 7 status
    addToType SampleArcsiteLogDataTagged.csv 5077 7 status
    addToType SampleArcsiteLogDataTagged.csv 5417 7 status
    addToType SampleArcsiteLogDataTagged.csv 5696 7 status
    addToType SampleArcsiteLogDataTagged.csv 5975 7 status
    addToType SampleArcsiteLogDataTagged.csv 6258 7 status
    addToType SampleArcsiteLogDataTagged.csv 6719 7 status
    addToType SampleArcsiteLogDataTagged.csv 6974 7 status
    closeType SampleArcsiteLogDataTagged.csv status
    addToType SampleArcsiteLogDataTagged.csv 74 28 description
    addToType SampleArcsiteLogDataTagged.csv 666 23 description
    addToType SampleArcsiteLogDataTagged.csv 1001 23 description
    addToType SampleArcsiteLogDataTagged.csv 1336 23 description
    addToType SampleArcsiteLogDataTagged.csv 1671 23 description
    addToType SampleArcsiteLogDataTagged.csv 2572 23 description
    addToType SampleArcsiteLogDataTagged.csv 2820 23 description
    addToType SampleArcsiteLogDataTagged.csv 3155 23 description
    addToType SampleArcsiteLogDataTagged.csv 3709 23 description
    addToType SampleArcsiteLogDataTagged.csv 4044 23 description
    addToType SampleArcsiteLogDataTagged.csv 4379 23 description
    addToType SampleArcsiteLogDataTagged.csv 4714 23 description
    addToType SampleArcsiteLogDataTagged.csv 5049 23 description
    addToType SampleArcsiteLogDataTagged.csv 7477 23 description
    addToType SampleArcsiteLogDataTagged.csv 7812 23 description
    addToType SampleArcsiteLogDataTagged.csv 8147 23 description
    addToType SampleArcsiteLogDataTagged.csv 8482 23 description
    addToType SampleArcsiteLogDataTagged.csv 9363 23 description
    addToType SampleArcsiteLogDataTagged.csv 9698 23 description
    addToType SampleArcsiteLogDataTagged.csv 10033 23 description
    closeType SampleArcsiteLogDataTagged.csv description
    addToType SampleArcsiteLogDataTagged.csv 63 5 device
    addToType SampleArcsiteLogDataTagged.csv 359 5 device
    addToType SampleArcsiteLogDataTagged.csv 655 5 device
    addToType SampleArcsiteLogDataTagged.csv 990 5 device
    addToType SampleArcsiteLogDataTagged.csv 1325 5 device
    addToType SampleArcsiteLogDataTagged.csv 1660 5 device
    addToType SampleArcsiteLogDataTagged.csv 1995 5 device
    addToType SampleArcsiteLogDataTagged.csv 2278 5 device
    addToType SampleArcsiteLogDataTagged.csv 2561 5 device
    addToType SampleArcsiteLogDataTagged.csv 2809 5 device
    addToType SampleArcsiteLogDataTagged.csv 3144 5 device
    addToType SampleArcsiteLogDataTagged.csv 3418 5 device
    addToType SampleArcsiteLogDataTagged.csv 3698 5 device
    addToType SampleArcsiteLogDataTagged.csv 4033 5 device
    addToType SampleArcsiteLogDataTagged.csv 4368 5 device
    addToType SampleArcsiteLogDataTagged.csv 4703 5 device
    addToType SampleArcsiteLogDataTagged.csv 5038 5 device
    closeType SampleArcsiteLogDataTagged.csv device
    addToType SampleArcsiteLogDataTagged.csv 69 4 method
    addToType SampleArcsiteLogDataTagged.csv 365 4 method
    addToType SampleArcsiteLogDataTagged.csv 661 4 method
    addToType SampleArcsiteLogDataTagged.csv 996 4 method
    addToType SampleArcsiteLogDataTagged.csv 1331 4 method
    addToType SampleArcsiteLogDataTagged.csv 1666 4 method
    addToType SampleArcsiteLogDataTagged.csv 2001 4 method
    addToType SampleArcsiteLogDataTagged.csv 2284 4 method
    addToType SampleArcsiteLogDataTagged.csv 2567 4 method
    addToType SampleArcsiteLogDataTagged.csv 2815 4 method
    addToType SampleArcsiteLogDataTagged.csv 3150 4 method
    addToType SampleArcsiteLogDataTagged.csv 3424 4 method
    addToType SampleArcsiteLogDataTagged.csv 3704 4 method
    addToType SampleArcsiteLogDataTagged.csv 4039 4 method
    addToType SampleArcsiteLogDataTagged.csv 4374 4 method
    addToType SampleArcsiteLogDataTagged.csv 4709 4 method
    addToType SampleArcsiteLogDataTagged.csv 5044 4 method
    addToType SampleArcsiteLogDataTagged.csv 5379 4 method
    addToType SampleArcsiteLogDataTagged.csv 5658 4 method
    addToType SampleArcsiteLogDataTagged.csv 5937 3 method
    addToType SampleArcsiteLogDataTagged.csv 6219 4 method
    addToType SampleArcsiteLogDataTagged.csv 6498 4 method
    addToType SampleArcsiteLogDataTagged.csv 6652 3 method
    addToType SampleArcsiteLogDataTagged.csv 6936 3 method
    addToType SampleArcsiteLogDataTagged.csv 7218 3 method
    addToType SampleArcsiteLogDataTagged.csv 7472 4 method
    addToType SampleArcsiteLogDataTagged.csv 7807 4 method
    addToType SampleArcsiteLogDataTagged.csv 8142 4 method
    closeType SampleArcsiteLogDataTagged.csv method
    addToType SampleArcsiteLogDataTagged.csv 0 23 date
    addToType SampleArcsiteLogDataTagged.csv 24 23 date
    addToType SampleArcsiteLogDataTagged.csv 296 23 date
    addToType SampleArcsiteLogDataTagged.csv 320 23 date
    addToType SampleArcsiteLogDataTagged.csv 592 23 date
    addToType SampleArcsiteLogDataTagged.csv 616 23 date
    addToType SampleArcsiteLogDataTagged.csv 927 23 date
    addToType SampleArcsiteLogDataTagged.csv 951 23 date
    addToType SampleArcsiteLogDataTagged.csv 1262 23 date
    addToType SampleArcsiteLogDataTagged.csv 1286 23 date
    addToType SampleArcsiteLogDataTagged.csv 1597 23 date
    addToType SampleArcsiteLogDataTagged.csv 1621 23 date
    addToType SampleArcsiteLogDataTagged.csv 1932 23 date
    addToType SampleArcsiteLogDataTagged.csv 1956 23 date
    addToType SampleArcsiteLogDataTagged.csv 2215 23 date
    addToType SampleArcsiteLogDataTagged.csv 2239 23 date
    closeType SampleArcsiteLogDataTagged.csv date
    addToType SampleArcsiteLogDataTagged.csv 48 14 ip
    addToType SampleArcsiteLogDataTagged.csv 344 14 ip
    addToType SampleArcsiteLogDataTagged.csv 640 14 ip
    addToType SampleArcsiteLogDataTagged.csv 734 11 ip
    addToType SampleArcsiteLogDataTagged.csv 975 14 ip
    addToType SampleArcsiteLogDataTagged.csv 1310 14 ip
    addToType SampleArcsiteLogDataTagged.csv 1645 14 ip
    addToType SampleArcsiteLogDataTagged.csv 1980 14 ip
    addToType SampleArcsiteLogDataTagged.csv 2263 14 ip
    addToType SampleArcsiteLogDataTagged.csv 2546 14 ip
    addToType SampleArcsiteLogDataTagged.csv 2794 14 ip
    addToType SampleArcsiteLogDataTagged.csv 3129 14 ip
    addToType SampleArcsiteLogDataTagged.csv 3403 14 ip
    addToType SampleArcsiteLogDataTagged.csv 3683 14 ip
    addToType SampleArcsiteLogDataTagged.csv 4018 14 ip
    addToType SampleArcsiteLogDataTagged.csv 4353 14 ip
    addToType SampleArcsiteLogDataTagged.csv 4688 14 ip
    addToType SampleArcsiteLogDataTagged.csv 5023 14 ip
    addToType SampleArcsiteLogDataTagged.csv 5358 14 ip
    addToType SampleArcsiteLogDataTagged.csv 5637 14 ip
    addToType SampleArcsiteLogDataTagged.csv 5916 14 ip
    addToType SampleArcsiteLogDataTagged.csv 6198 14 ip
    addToType SampleArcsiteLogDataTagged.csv 6477 14 ip
    addToType SampleArcsiteLogDataTagged.csv 6631 14 ip
    addToType SampleArcsiteLogDataTagged.csv 6915 14 ip
    addToType SampleArcsiteLogDataTagged.csv 7197 14 ip
    addToType SampleArcsiteLogDataTagged.csv 7451 14 ip
    addToType SampleArcsiteLogDataTagged.csv 7786 14 ip
    addToType SampleArcsiteLogDataTagged.csv 8121 14 ip
    addToType SampleArcsiteLogDataTagged.csv 8456 14 ip
    addToType SampleArcsiteLogDataTagged.csv 8791 14 ip
    addToType SampleArcsiteLogDataTagged.csv 9064 14 ip
    addToType SampleArcsiteLogDataTagged.csv 9337 14 ip
    addToType SampleArcsiteLogDataTagged.csv 9672 14 ip
    closeType SampleArcsiteLogDataTagged.csv ip

    You will notice a tag in there called "_mixupIP".  This is the tag I created by running Mixup on the data.  The other tags are XML tags I embedded manually in the .csv file.  When I use the TextBaseEditor to view the .csv file, only the XML tags are available in the drop-down boxes for highlight/compare.  The "_mixupIP" tag is not offered for selection, so that has me thinking that the TextBaseEditor is only looking at the embedded XML tags and not the .labels file for its choices.  I'm running this via Eclipse with the following arguments:

    Main class:
    edu.cmu.minorthird.text.gui.TextBaseEditor

    Arguments:
    C:/MinorThird/SampleData ArcsiteMixupOutput.labels

    Where, ../SampleData is holding the .csv file that's been manually tagged with XML, as well as having Mixup run against it.

    Thanks very much for any tips you can offer.

    -Paul

     
  • Frank Lin

    Frank Lin - 2012-03-19

    Hi Paul,

    Currently minorthird can only read either text with xml tags or text with a labels file, but not both at the same time - this is to avoid confusion in lining up the offsets found in the .labels file with xml-tagged text.

     
  • catrambonep

    catrambonep - 2012-03-20

    Thanks for the reply.  I had traced through the code late last week to find just as you posted regarding the xml and labels file.  I'm moving on to the training and testing of extractors.  Will post any questions I have.  Thanks again.

     

Log in to post a comment.