||[ E-Announce ] Call for Participation: Semantic Textual Similarity (STS) - SemEval task 6
||26 Nov 2012
||15 Feb 2013
Call for participation
Semantic Textual Similarity (STS) SemEval 2013 - Task 6
Following the success of the 2012 task on STS (Agirre et al.
2012), with 35 teams and 88 runs, we solicit participation in the
second STS task, which will held at SemEval 2013, co-located with
the *SEM and NAACL-2013 conferences.
STS in 2013
Participants will submit systems that examine the degree of
semantic equivalence between two sentences. The goal of the STS
task is to create a unified framework for the evaluation of
semantic textual similarity modules and to characterize their
impact on NLP applications. We particularly encourage submissions
from the lexical semantics, summarization, machine translation
evaluation metric, and textual entailment communities.
The task will follow a similar design as last year, but instead
of providing train/test data from the same datasets, we will
provide all the 2012 data as training data, and the test data
will be drawn from related but different datasets. This setting
is more realistic, and the teams which prefer to train/test on
the same datasets can perfomr experiments using the 2012 data.
The task will thus comprise all the 2012 data plus the following
New test data similar to the datasets used last year
(Paraphrase, SMT evaluation, gloss pairs). The SMT evaluation
data will include data from HyTER graphs. A dataset on
typed-similarity between semi-structured items, comprising
both train and test data. An open source baseline, comprising
a pipeline to combine STS systems and components. An new
measure for evaluation.
Please check http://www.cs.york.ac.uk/semeval-2013/task6/ for a
detailed task description.
Initial training dataset - 11 November Full training dataset +
test scripts ready - 31 December Registration deadline - 15
February Start of Evaluation period - 1 March End of Evaluation
period - 15 March 23:59 (UTC-11) Paper due - 9 April [TBC]
Reviews due - 23 April [TBC] Camera Ready - 4 May [TBC]
Use the following mailing list: sts-semeval googlegroups com.
If interested in the task please join the mailing list for
Eneko Agirre, University of the Basque Country, Basque Country
Daniel Cer, Stanford University, USA Mona Diab, Columbia
University, USA Aitor Gonzalez-Agirre, University of the Basque
Country, Basque Country
- ELSNET mailing list Elsnet-list_(at)_elsnet.org
- To manage your subscription go to:
[ Search | Events calendar | Deadline calendar