Elsnet
 
   


ELSNET-list archive

Category:   E-Announce
Subject:   Preliminary announcement of the 1st Anaphora Resolution Exercise (ARE)
From:   Constantin Orasan
Email:   C.Orasan_(on)_wlv.ac.uk
Date received:   02 Nov 2006
Start date:   28 Mar 2007

[Apologies for multiple postings] ------------------------------------------------------- Preliminary announcement of the 1st Anaphora Resolution Exercise (ARE) http://clg.wlv.ac.uk/events/ARE/ ------------------------------------------------------- We are pleased to announce the first Anaphora Resolution Exercise (ARE) organised in conjunction with the 6th Discourse Anaphora and Anaphor Resolution Colloquium (DAARC2007). The general objective of ACE is to develop discourse anaphora resolution methods and to evaluate them in a common and consistent manner. The exercise proposes four tasks: 1. Pronominal resolution on pre-annotated texts: in this task, participants will be provided with documents in which the NPs are annotated. Among these NPs, some of them will have an attribute which indicates that they are referential and have to be resolved. In this task only referential pronouns will have this attribute (i.e. pleonastic pronouns will not be annotated). The participants will have to identify for each pronominal referential expression any correct non-pronominal antecedent from the list of annotated NPs. 2. Coreferential chains resolution on pre-annotated texts: in this task participants will be provided with documents in which the NPs are annotated. The participants will have to identify full coreferential chains which contain the annotated NPs. 3. Pronominal resolution on raw texts: in this task, participants will have to resolve referential pronouns in unannotated texts (i.e. the referential expressions and antecedent candidates are not marked as in task 1) 4. Coreferential chains resolution on raw texts: in this task, participants will have to identify full coreferential chains in unannotated texts (i.e. the NPs are not pre-marked) In this first edition, the evaluation is organised only for English and tackles pronominal anaphora and NP coreference. If successful, in future editions it can be extended to include other types of referential expressions and anaphoric relations (e.g. bridging references) and other languages. The participants will be given a set of guidelines which will explain which referential expressions are considered, when two elements are coreferential and the evaluation procedures. In addition, we foresee the development of a web interface where the participants will be able to evaluate their results on the training data using the evaluation metrics to be used in the formal evaluation. Given that in normal circumstances the tasks of anaphora and coreference resolution are not very computational intensive the participants will have only 24h between the moment they download the test data and the time they have to submit the results. Tentative timeline: 2nd Nov 2006: the first announcement of ARE 2nd Nov 2006: the registration opens 15th Nov 2006: the training data available 15th Nov 2006: release of the guidelines 12th March 2007: the test data available 16th March 2007: the competition finishes 28th - 29th March 2007: DAARC2007 The results of this evaluation will be disseminated at DAARC where we plan to organise a round table and/or a poster session on this theme. If enough interest is shown by the participants, we also consider publishing a collection of paper afterwords. Participation is open to all. To indicate your intention to participate or for any other issues please contact Constantin Orasan (C.Orasan_(on)_wlv.ac.uk) Web page: http://clg.wlv.ac.uk/events/ARE/ Organising committee: Antonio Branco, University of Lisbon, Portugal Dan Cristea, "Alexandru Ioan Cuza" University, Iasi, Romania Ruslan Miktov, University of Wolverhampton, UK Constantin Orasan, University of Wolverhampton, UK -- Constantin Orasan <C.Orasan_(on)_wlv.ac.uk> Lecturer in Computational Linguistics Research Group in Computational Linguistics http://www.wlv.ac.uk/%7Ein6093/ University of Wolverhampton _______________________________________________ Elsnet-list mailing list Elsnet-list_(on)_elsnet.org http://mailman.elsnet.org/mailman/listinfo/elsnet-list
 

[print/pda] [no frame] [navigation table] [navigation frame]     Page generated 14-02-2008 by Steven Krauwer Disclaimer / Contact ELSNET