1832

Journal of Cultural Analytics

2371-4549

Center for Digital Humanities, Princeton University

Website: Journal of Cultural Analytics

11193

10.22148/16.049

Article

A Shared Task for the Digital Humanities Chapter 2: Evaluating Annotation Guidelines

Gius

Evelyn

Reiter

Nils

Wielland

Marcus

2 12 2019

2019

4 3

A Shared Task for the Digital Humanities: Annotating Narrative Levels

11193

http://creativecommons.org/licenses/by/4.0

This is an open access article distributed under the terms of the Creative Commons Attribution License (4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

In this section, we will discuss our idea of guideline evaluation and the underlying considerations. Evaluating annotation guidelines in this way is a fairly new endeavor, and we have developed the evaluation setup from the ground up. Although we do not claim our choices to be universally valid or applicable, we believe that this approach to guideline evaluation is relevant for similar settings and can be adapted to projects that might have other preferences and priorities.

shared task narratology