ISA-16 (@LREC 2020)
ISA-15 (Gothenburg, 2019)
ISA-14 (Santa Fe (NM), 2018)
ISA-13 (Montpellier, 2017)
ISA-12 (Portoroz, 2016)
ISA-11 (London, 2015)
ISA-10 (Reykjavik, 2014)
ISA-9 (Potsdam, 2013)
ISA-8 (Pisa, 2012)
ISA-7 (Istanbul, 2012)
ISA-6 (Oxford, 2011)
ISA-5 (Hong Kong, 2010)
ISA-4 (Tilburg, 2009)
ISA-3 (Marina del Rey, 2008)
ISA-2 (Tilburg, 2007)
ISA-1 (Tilburg, 2005)
Welcome to ISA-17, the Seventeenth Workshop on Interoperable Semantic Annotation. This is the 2021 edition of a series of joint workshops of the ACL Special Interest Group on Computational Semantics (SIGSEM) and ISO TC 37/SC 4 (Language Resources) WG 2: Semantic annotation.
These workshops bring together experts in the annotation of semantic information as expressed in text, speech, gestures, graphics, video, images, and in multiple modalities combined. Examples of semantic annotation include the markup of events, time, space, motion, dialogue acts, discourse relations, semantic roles, coreference, and quantification phenomena, for which the ISO organization pursues the establishment of standardised annotation methods and representation schemes, to support the creation of interoperable semantic resources.
|13:00||Opening||Session 1: Paper presentations (1)|
|13:05||Purificação Silvano, António Leal, Fátima Silva, Inês Cantante, Fatima Oliveira and Alípio Mario Jorge||Developing a multilayer semantic annotation scheme based on ISO standards for the visualization of a newswire corpus|
|13:35||Volha Petukhova and Hafiza Erum Manzoor||Towards an ISO 24617-2-compliant typology of metacognitive events, presentation slides here.|
|14:05||Tillmann Dönicke, Luisa Gödeke and Hanna Varachkina||Annotating Quantified Phenomena in Complex Sentence Structures Using the Example of Generalising Statements in Literary Texts, presentation slides here.|
|14:30||Session 2: Quantification Track|
|Harry Bunt||The ISA-17 Quantification Challenge: Background and introduction|
|14:40 - 16:00 flash presentations and discussion|
|Maxime Amblard, Karen Fort, Bruno Guillaume, Philippe de Groote, Chuyuan Li, Pierre Ludmann, Michel Musiol, Siyana Pavlova, Guy Perrier, and Sylvain Pogodalla||The Annotators Did Not Agree on Some of the Guidelines Examples, with accompanying annotations and specification of their authors.|
|Johan Bos||Quantification Annotation in Discourse Representation Theory (annotations included)|
|Harry Bunt||Observations on the use of QuantML, with accompanying annotations.|
|12:00||Session 3: Paper presentations (2)|
|12:00||Rodolfo Delmonte, Serena Trolvi and Francesco Stiffoni||Converting Multilayer Glosses into Semantic and Pragmatic forms with GENLIS, presentation slides here.|
|12:30||Ekaterina Saveleva, Volha Petukhova, Marius Mosbach and Dietrich Klakow||Discourse-based Argument Segmentation and Annotation, presentation slides here.|
|13:00||Giuseppe Abrami, Alexander Henlein, Andy Lücking, Attila Kett, Pascal Adeberg and Alexander Mehler||Unleashing annotations with TextAnnotator: Multimedia, multi-perspective document views for ubiquitous annotation|
|13:30||Session 4: Visual Representation Track||James Pustejovsky and Nikhil Krishnaswamy||VoxML-based shared task: Background and introduction|
|13:40 - 15:00 flash presentations and discussion|
|Mark Klement, Alexander Henlein, and Alexander Mehler||VoxML Annotation Tool Review and Suggestions for Improvement|
|Manolito Octaviano||VoxML Standardization|
Besides a main track, ISA-17 features two specialised tracks, focused on:
These topics are inspired by a recently started project on developing an ISO standard for the annotation of quantification (ISO/WD 24617-12, 'QuantML'), and a proposal for developing an ISO standard for the representation of visual information ('VoxML').
Both specialised tracks consist of a pre-conference on-line portion and an on-site portion during the IWCS 2021 conference in Groningen. For the pre-conference on-line portion participants are invited to take part in a shared task, submitting their commented annotations and observations for a batch of example items that are provided with documentation and guidelines of the envisaged annotation and representation schemes.
For both tracks, annotators should attempt to follow the guidelines and stay within the provided vocabulary where possible, but are permitted to go outside if required. In this case, we ask that they summarily justify their additions and alternatives in their project note. The track organisers will provide feedback on submitted annotations, observations and analyses during the on-site portion of the workshop. This will be followed by a discussion of the entire annotation effort, lessons learned (problems, gaps, suitability of results for ML, automated generations, etc.), and the adequacy and expressiveness of the representation language.
The general topics of the two tracks (quantification and vizualization) are also topics for submissions to the main track of the workshop, which furthermore include, but are not limited to:
For the pre-conference part we make the current version available of the QuantML annotation scheme and markup language, as developed as a basis for the ISO project, and documented in the technical report Semantic Annotation of Quantification in Natural Language (TiCC Technical Report 2020-2, Tilburg University). Participants in the shared task, the 'Quantification Annotation Challenge', are asked to annotate as many sentences as possible of the Quantification Challenge test suite: (.docx file), using QuantML, and to submit a short project note (up t0 4 pages) with their observations on the adequacy of the QuantML scheme and representations, the annotation guidelines, problems encountered in annotating the test sentences or related examples, other properties that should be included, and relations to recently proposed representations of quantification, such as those of the Groningen MeaningBank or AMR annnotations or frame representations. The aim of the shared task is to identify strenghts and weaknesses of the QuantML scheme, to signal problematic issues, to get an impression of annotator agareement, and to make comparisons to other recently proposed representations of quantification phenomena. Questions to consider include the following:
Please submit your observations and annotations (in a separate file) before May 1st (extended deadline) on the ISA-17 submission page. The project notes will be reviewed for presentation during the on-site part of the workshop.
Relevant background publications are:
For the pre-conference on-line work, we supply a new VoxML annotation interface and batch of images of objects and people participating in activities and events. Follow this link to the page for the VoxML/Voxicon simulation and reasoning platform, where you find the images to be annotated, using a vocabulary of objects, attributes, actions, and relations. Accompanying the collection of images you also find guidelines for how annotation should be performed, using an annotation template. Participants in the shared task of this track are asked to annotate as many of the images as they can using the provided tool, and to submit a short project note covering the adequacy of the annotation guidelines, of the functionality of the annotation tool, and any properties in the raw image data while annotating, and optional deeper investigations into other things that would be interesting to annotate (other data, properties that should be included, problems pertaining to standardisation, ects.). These project notes will be reviewed for presentations at the on-side portion of the workshop. For any questions please contact the track coordinator James Pustejovsky (email@example.com).
Background document: the LREC 2016 paper VoxML: A Visualization Modeling Language by James Pustejovsky and Nikhil Krishnaswamy.
For both tracks, annotators should attempt to follow the guidelines and stay within the provided vocabulary where possible, but are permitted to go outside if required. In this case, we ask that they summarily justify their additions and alternatives in their project note. The track organisers will provide feedback on submitted annotations, observations and analyses during the on-site portion of the workshop. This will be followed by a discussion of the entire annotation effort, lessons learned (problems, gaps, suitability of results for ML, automated generations, etc.), and the adequacy and expressiveness of the representation language.Deadline for submission of project note and accompanying annotations is April 21.
|Submission Link||The ISA-17 submission site|
|Submission Format||PDF formatted with the IWCS LaTeX style files|
The ISA-17 workshop is hosted by the IWCS 2021 conference in Groningen.
|5 April||Submission of papers and project notes for the main track|
|18 April||Notification of Acceptance|
|1 May||Submissions to Quantification and Visualization tracks|
|7 May||Submission of camera-ready material|
|16 June, 13:00 - 16:00 and 17 June, 12:00 - 15:00||On-line ISA-17 Workshop|
For any questions contact the workshop chair, at the address firstname.lastname@example.org