Quantification Track

Issues in the annotation of aspects and forms of quantification in natural language are t be discussed, with the aim to establish a good basis for developing an interoperable annotation scheme that would ideally fit into the ISO Semantic Annotation Framework (ISO 24617). The document Semantic Annotation of Quantification in Natural Language (TiCC Technical Report 2020-2) describes the current state of developing such an annotation scheme, including the markup language QuantML. Previous publications from preliminary studies are:

• Bunt, Harry, James Pustejovsky, and Kiyong Lee. "Towards an ISO Standard for the Annotation of Quantification." Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). 2018.
• Bunt, Harry. "An Annotation Scheme for Quantification."Proceedings of the Thirteenth International Conference on Computational Semantics (IWCS 2019).

In the pre-conference, on-line part, track participants are invited to test, comment on, ciriticise, praise, compare, improve, the annotations that can be made with the current version of the QuantML scheme, making use of the short list of test sentences made available here, and consider how to annotate them. The aim is to identify strenghts and weaknesses of the current scheme, to signal problematic issues, and to make comparisons to recent representations of quantification and scope. Note any specific issues that emerge in the annotation process and in comparisons, that would be good to be discussed in the on-site part of the workshop. Questions to consider include at least the following:

• Can you annotate the sentence in a satisfactory manner using the QuantML annotation scheme? What problems do you encounter in trying to do so?
• Do you think the sentence should rather be annotated in a different way? How? Why?
• How does the annotation compare to other recent representations for quantification and scope for instance in the Groningen MeaningBank or in efforts to represent quantifier scope with AMR annotation?
• Are the current annotation guidelines adequate for indicating how to deal with this sentence? How could they be improved?
• What implications do you see from this example for the design of an annotation tool for quantification?

Please send your commented annotations and observations and any other relevant considerations that occur to you (in any readable form) to the track coordinator Harry Bunt (harry.bunt@uvt.nl) please before March 20. We will take it interactively from there in preparation of the on-site part of the workshop.

Visualization/VoxML Track

For the pre-conference on-line work, we supply a batch of images of objects and people participating in activities and events follow this link to the homepage for the VoxML/Voxicon simulation and reasoning platform. Here you find a collection of images to be annotated, using a vocabulary of objects, attributes, actions, and relations. Annotators are asked to use these to create an annotation of the image, as exhaustively as possible. Accompanying the collection of images you also find guidelines for how annotation should be performed, using an annotation template, and submitted. For any questions please contact the track coordinator James Pustejovsky (jamesp@brandeis.edu).

Background document: the LREC 2016 paper VoxML: A Visualization Modeling Language by James Pustejovsky and Nikhil Krishnaswamy.

For both tracks, annotators should attempt to follow the guidelines and stay within the provided vocabulary where possible, but are permitted to go outside if required. In this case, we ask that they submit a brief summary of and justification for their additions.

We will provide feedback on submitted annotations. Deadline for submissions is March 20. We will select certain annotation submissions for spotlight presentations. These are to be given during the on-site portion of the workshop and will focus on the results of the annotation exercise. This will be followed by a discussion of the entire annotation effort, lessons learned (problems, gaps, suitability of results for ML, automated generation, etc), and the adequacy and expressiveness of the representation language.