isa -16:
Sixteenth Joint ACL - ISO Workshop on Interoperable Semantic Annotation

Marseille, 11 - 16 May 2020, as part of LREC 2020

isa-16, the Sixteenth Workshop on Interoperable Semantic Annotation, is the most recent edition of a series of joint workshops of the ACL Special Interest Group on Computational Semantics (SIGSEM) and ISO TC 37/SC 4 (Language Resources) WG 2: Semantic annotation.

These workshops bring together experts in the annotation of semantic information as expressed in text, speech, gestures, graphics, video, images, and in multiple modalities combined. Examples of semantic annotation include the markup of events, time, space, dialogue acts, discourse relations,semantic roles, reference relations, and quantification for which the ISO organization pursues the establishment of annotation standards, in order to support the creation of interoperable semantic resources.

See also the most recent previous ISA workshops:

ISA-15 in Gothenburg, Sweden, May 2019 -- See the program and the ISA-15 proceedings.
ISA-14 in Santa Fe, New Mexico, USA, August 2018 -- See the program and the ISA-14 proceedings.
ISA-13 in Montpellier, France, September 2017 -- program and proceedings.
ISA-12 in Portoroz, Slovenia, June 2016
ISA-11 in London, UK, 2015
ISA-10 in Reykjavik, Iceland, 2014

ISA-9 in Potsdam, Germany, 2013

ISA-8 in Pisa, Italy, October 2012

Location: Le Palais du Pharo, Marseille

Besides a main track, ISA-16 will feature two specialised tracks, focused on (a) the annotation of quantification (and quantified modification) in natural language and (b) the design and representation of data structures for generating visualisations of linguistically represented objects, properties, and events. These topics relate to a recently started project on developing an ISO annotation standard for quantification (ISO/WD 24617-12) and a proposal for developing an ISO standard for the representation of visual information ('VoxML').

Both specialised tracks will consist of a pre-conference on-line portion and an on-site portion during the ISA-16 workshop. For the pre-conference on-line portion participants will be invited to submit their commented annotations and representations for a batch of example items that will be provided with documentation and guidelines of the envisaged annotation and representation schemes.

Concerning the quantification track: comparisons will be made to recent representations for quantification and scope in the Groningen MeaningBank, as well as to efforts to represent quantifier scope with AMR annotation. Material from diverse sources will be supplied for annotation, analysis and identification of problematic issues. The results, specific issues that emerge in the process, and comparisons will be discussed in the on-site part of the workshop. 

Concerning the 'VoxML'/visual information track: for the pre-conference on-line work, we will supply a batch of images of objects and people participating in activities and events. There will be no captions provided with the images. There will be a vocabulary of objects, attributes, actions, and relations, with which annotators need to create an annotation of the image, as exhaustively as possible. We provide guidelines for how annotation should be performed.  

For both tracks, annotators should attempt to follow the guidelines and stay within the provided vocabulary where possible, but are permitted to go outside if required.  In this case, we ask that they submit a brief summary of and justification for their additions.

We will provide feedback on submitted annotations. We will select certain annotation submissions for spotlight presentations. These are to be given during the on-site portion of the workshop and will focus on the results of the annotation exercise. This will be followed by a discussion of the entire annotation effort, lessons learned (problems, gaps, suitability of results for ML, automated generation, etc), and the adequacy and expressiveness of the representation language. 

Call for Papers
Topics for submissions to the main track of the workshop include, but are not limited to:
  • methodological aspects of semantic annotation
  • experiments in semantic annotation
  • context in semantic annotation
  • applications of semantic annotation
  • generating scenes from annotated text
  • events and participants in annotation and visualisation
  • uncertainty and ambiguity in annotations
  • combining annotations from different schemes
  • semantic annotation, interpretation, and inference
  • granularity in annotation schemes
  • evaluation and comparison of semantic annotations
  • semantic annotation in specialised domains
  • issues in the modelling of semantic information in specific areas, such as:
    • events, states, processes, circumstances, facts
    • space, motion events and 3D objects as participants
    • modality, factuality, polarity and negation
    • quantification and modification
    • relations in discourse and dialogue
    • referential relations
    • attribution, sentiment, attitudes, and emotions
Three types of submission are invited:
  • Research papers, describing original research in the area of semantic annotation; these can be either long (6-8 pages, with additional pages for references references) or short (3-5 pages, plus references);
  • Project notes, describing recent, ongoing or planned projects involving semantic annotation (2-4 pages including references);
  • Commented annotations/representations for the specialised tracks.
Submission of papers is in PDF form through the ISA-16 submission site. All submitted papers should be formatted in the same way as submissions to the LREC 2020 conference, see the LREC Author's Kit.

About the LRE Map, the sharing of language resources, and the use of ISLRN numbers:

Describing your LRs in the LRE Map is now a normal practice in the submission procedure of LREC (introduced in 2010 and adopted by other conferences). To continue the efforts initiated at LREC 2014 about “Sharing LRs” (data, tools, web-services, etc.), authors will have the possibility, when submitting a paper, to upload LRs in a special LREC repository. This effort of sharing LRs, linked to the LRE Map for their description, may become a new “regular” feature for conferences in our field, thus contributing to creating a common repository where everyone can deposit and share data.

As scientific work requires accurate citations of referenced work so as to allow the community to understand the whole context and also replicate the experiments conducted by other researchers, LREC 2020 endorses the need to uniquely identify LRs through the use of the International Standard Language Resource Number (ISLRN,, a Persistent Unique Identifier to be assigned to each Language Resource. The assignment of ISLRNs to LRs cited in LREC papers will be offered at submission time.

The ISA-16 workshop is hosted by the LREC 2020 conference and is held at the conference center Le Palais du Pharo in Marseille.

Important dates
25 February 2020: Deadline for submitting papers
16 March 2020: Notification of acceptance
1 April 2020 Submission of camera-ready papers
12 May 2020:
isa-15 Workshop

The two specialised tracks will have their own detailed time schedule, in particular for their on-line pre-conference part (t.b.a.).

Organizing committee

  • Harry Bunt (chair)
  • Nancy Ide
  • Kiyong Lee
  • Volha Petukhova
  • James Pustejovsky
  • Laurent Romary

    Program committee

  • Lasha Abzianidze
  • Jan Alexandersson
  • James Allen
  • Ron Artstein
  • Johan Bos
  • Harry Bunt (chair)
  • Nicoletta Calzolari
  • Jae-Woong Choe
  • Robin Cooper
  • David DeVault
  • Simon Dobnik
  • Jens Edlund
  • Alex Fang
  • Robert Gaizauskas
  • Kallirroi Georgila
  • Jan Hajic
  • Koiti Hasida
  • Nancy Ide
  • Elisabetta Jezek
  • Nikhil Krishnaswamy
  • Kiyong Lee
  • Paul Mc Kevitt
  • Adam Meyers
  • Roser Morante
  • Philippe Muller
  • Rainer Osswald
  • Volha Petukhova
  • Massimo Poesio
  • Eric Postma
  • Laurent Prevot
  • James Pustejovsky
  • Livio Robaldo
  • Laurent Romary
  • Ielka van der Sluis
  • Manfred Stede
  • Matthew Stone
  • Thora Tenbrink
  • Sara Tonelli
  • Thorsten Trippel
  • Carl Vogel
  • Menno van Zaanen
  • Annie Zaenen
  • Heike Zinsmeister

    More information
    For any questions contact the workshop chair,