#378 Encoding of Standoff annotations
The annotation of documents using standoff annotations is a very useful and flexible methodology. Nevertheless, TEI does not have any specific elements for encoding this information.
In most of cases, the standoff annotations are stored as external TEI files linked to the text being annotated. Nevertheless, this way of storing the standoff annotations is very rigid and presents numerous problems, for example, for indexing or searching the corpus of documents using the information of the annotations. In these cases, it would be very useful to have the standoff annotations INSIDE the TEI documents being annotated (!!!).
Therefore, it is suggested to include define a new set of TEI elements specifically dedicated to the encoding of the standoff annotations.
The idea would be to store the standoff annotations between the <teiHeader> and the <text>, following the same philosophy as used for the <facsimile> and for <sourceDoc> (in some way these two elements could also be considered as a "type" of annotation).
For the standoff annotation, the structure could be:
<TEI>
<teiHeader>
...
</texHeader>
<standoff>
[information of the annotations]
</standoff>
<text>
...
</text>
</TEI>
This structure would provide the extra advantage of allowing to annotate the information at different TEI levels in a natural manner. So for more complicated TEI documents having different hierarchical levels, the standoff annotations could be encoded as follows:
<teiCorpus>
<teiHeader>
...
</teiHeader>
<TEI>
<teiHeader>
...
</texHeader>
<standoff>
...
</standoff>
<text>
...
</text>
</TEI>
<TEI>
<teiHeader>
...
</texHeader>
<standoff>
...
</standoff>
<text>
...
</text>
</TEI>
</teiCorpus>
This structure would also provide the extra advantage of allowing to annotate, not only the text of the document, but also the metadata of the different hierarchical levels of the TEI document.
The specific encoding of the annotations inside <standoff> could be as follows:
<standoff>
<annotation type="..." subtype="...">
<author>...</author>
<date>...</date>
<ptr>...</ptr>
[other data needed]
</annotation>
</standoff>
As a last remark it is also suggested to allow inside the <annotation> the TEI element <figure> in order to facilitate the annotation not only of textual information, but also of images and formulas.
Conclusion: the proposed structure for the encoding of standoff annotations in TEI provides the following advantages:
- allows to encode standoff annotations under TEI in a natural manner, which is not the case at the moment
- allows to store the standoff annotations INSIDE the TEI document being annotated in a specific location, facilitating the process of indexing and
searching said documents
- it is naturally integrated in the hierarchical structure of the TEI (<teiCorpus>, <TEI>)
- allows to annotate both, the textual and not textual information, and also the metadata of the <teiHeader>
- facilitates the exchange of annotations because they are already stored in the original TEI document being annotated
Remark: this idea has been already suggested by Piotr Bański in his article "Why TEI stand-off annotation doesn't quite work and why you might want to use it nevertheless", in http://www.balisage.net/Proceedings/vol5/html/Banski01/BalisageVol5-Banski01.html
I entirely agree that this new element doesn't belong inside
<text>. However, I am less convinced that it doesn't belong inside<teiHeader>. Javier says above "The stand-off annotations comprise extra information, ... that are not part of the text being described or metadata associated to it" But surely that extra information is precisely "metadata" -- information about the text? And we already have a lot of "standoffish" elements in the header, which are nothing to do with the "titlepage" aspects e.g.<particDesc><listPlace> etc. I think there's a big difference between things like<facsimile> or<sourceDoc> which are different views of the same textual object, and the header which is at a different "meta" level.Last edit: Lou Burnard 2013-06-18
I don't think ancillary textual content is the same thing as metadata at all. Editorial annotations, prosopographical information etc. is not part of the core source document, but it is part of the text in another sense. It would not normally be put in a library catalogue, and it would traditionally be printed either on the page (as footnotes etc.) or in appendices. So I don't think it belongs in teiHeader.
As I already indicated, I think that the annotation shouln't be part of the teiHeader. It is important to stablish a conceptual differentiation between the metadata of the text (i.e. information about the core nature of the source text) and the annotations (i.e. added information NOT "naturally" linked to the original text and which has been created, probably latter on, with some specific meaning not directly related to the specific nature of the text). In some way the annotations could be seen as "postit" markers that provide information about the source text or parts of said source text. It would be some how estrange to put the information of these postits as part of the teiHeader of the source document (!!!). Following this simile, one could think the annotations as very small kind-of-documents linked to the source text. Now, one interesting question would be, when an "annotation" has enough "entity" to be considered as an independent document? This issue could be clarified latter on. For the moment, and in view of the previous reasons, I would think that the annotations shouldn't be part of the teiHeader
A related question in my mind is that of other kinds of descriptive data
or annotation, such as RDF triples defining properties of the text
content in some ontology. Would those be included within the proposed
element?
On 18/06/13 13:56, Javier Pose wrote:
Related
Feature Requests: #378
Hello,
I have been thinking these days about a possible structure for the "standoff" element and a general framework for encoding standoff annotations in TEI.
In order to have a more clear proposal, I wrote a working document explaining in detail a proposed structure for encoding standoff annotations in <standoff>. I attach the document to this message
I hope this can help for further discussions.
Regards,
Can I suggest you publish this more widely, Javier, and tell people on TEI-L about it? this really deserves careful reading by many people.
Got a notification of Sebastian's post, but not of Javier's. Earlier today, I received Javer's document and I find it impressive. We'll try to open it for discussion from the SIG pages, if that is OK with everyone.
Thanks Javier for your very detailed description of the proposal. I have a couple of immediate responses to it:
I think it should be called
<standoff>. There's no reason to abbreviate, because this is not an element that's going to crop up dozens of times in a document, and it's really not clear what<stf>might mean if you don't already know.I think it should also be available as a sibling of
<teiHeader>inside<teiCorpus>. I would imagine that a lot of the kind of data appearing in this element would be applicable to all the documents in a corpus.Hi Martin, regarding your comments:
I agree, the main standoff annotations block should be called "standoff" (the document I released also agrees with this, see section 5). The elements "stf" and "stfGrp" are child of this main element for implementing the hierarchical standoff annotations and allowing the grouping of them.
I am still not sure whether the metadata of the standoff annotations should appear in the teiHeader. In principle I like the idea of considering the standoff anotations as independent entities, so I would like to have the annotations as "atomic" as possible, i.e. self contained. In this approach also helps the fact of having "fsdDecl" as an element that can also be independent of teiHeader, so if further metadata about the annotations is needed (in the case of the feature structures) it can be defined outside teiHeader. In any case, I am still not 100% convince of this, so it could also be an idea to have part of the standoff metadata in the teiHeader (in the document this is also disclosed as an Open Issue, see section 8)
(Resetting the priority back to "5", with apologies for the incident, while pointing my finger at the SF maintainers.)
There is going to be a meeting devoted to this issue, in Jan/Feb 2014 in Berlin, with, minimally, Javier, Laurent and Piotr, hopefully also Andreas Witt, hopefully a Council representative, and surely several other colleagues interested in the topic from various angles.
The Council is obviously going to be informed about the results of that meeting, either via its representative, or with a report from us. We hope that these results will be taken into consideration when the content of <standOff> (or <standoff>, or whatever it ends up called) is decided on.
Last edit: Piotr Banski 2013-11-09
Some quick comments …
I very much like the idea of a new linked-data-block kind of container element, although I'm not sure that <standoff> is sufficiently generic. One can imagine that a lot of useful stuff would get tucked into this space. The already mentioned <spanGrp>, <interpGrp>, <linkGrp>, of course; but also contextual information (<listPerson>, <listPlace>, etc.), specialized annotations (<listChange>, <witDetail>), and “phantom” text (e.g., <castItem>s that do not appear in the source text, the words spelled out by an acrostic).
I mildly prefer making this new container an optional child of <text> before <front>, but could easily be talked into the “between <teiHeader> and <text>” idea. I shy away from putting it in the <teiHeader>.
I have not figured out what the difference is between the suggested <annotation> element and the TEI <note> element. At first blush, they look the same.
I have not paid attention to the OAC for some time now, but we should take a peek at what they're doing and also brush up on XStandoff before settling on anything.
This is just to note that we have just finished a very successful (imnsho) meeting devoted to these issues, hosted and chaired by Laurent Romary at HUB, with life-maintaining infrastructural support from Carolin Odebrecht. The results will be reported on to the Council by Peter Stadler, who has been of great help, by keeping the minutes and editing the ODD, and raising valid issues all at the same time. Members of the LingSIG, which was happy (well, eager) to provide the small-scale "institutional" umbrella for the relevant post-meeting activities, will be notified and queried as well. The same goes for the TEI community at large, in due time (which the Council will probably determine for us).
The ball is rolling and we will be hoping for the Council's comments and hints after the upcoming F2F meeting, if not earlier.
From Nov. 2014 F2F:
The last information we have is the minutes of the WG meeting in Berlin from PWS: https://docs.google.com/document/d/1QqJK08sff4ral0tadmNXs0j0VN5Yp-O7_mIsv2f-WjQ/edit#heading=h.odkayr64o4dg
Council is waiting for a proposal - pls link information so we can review it.
Dear all, I'm very interested in using TEI-standoff for linguistic annotation for historic data (manuscripts, 15th–17th century). I'd like to ask what's the current status on that proposal?
\<standoff> in my opinion is neither \<header> nor \<text> content. I would differentiate between meta elements like a manuscript description and actual metadata on the text. The element \<standoff> could be used for segmentation information like word tokenization and statements regarding those segments like POS-Tagging – at least to my understanding.
Thank you!
I put together a Github project with ODD spec and examples for review and discussion: https://github.com/laurentromary/stdfSpec
It would be good to move a head slowly with this.
@PeterHinkelmanns: please provide possible examples that could be used as possible application of the proposed element.
Assigning to Peter to get this moving again.
Standoff has moved to a proposed implementation now available at https://github.com/laurentromary/stdfSpec
Council working group (PFS, LB, MH, FC, SM, PWS) created an alternative proposal as the "Ann Arbor" branch at https://github.com/laurentromary/stdfSpec/tree/AnnArbor