ELSNET-List Message

ELSNET-List Message

Subject: [ E-CFP ] Semeval-2013 Task #11: Word Sense Induction & Disambiguation within an End-User Application
From: <navigli_(on)_di.uniroma1.it>
Date received: 17 Jan 2013
Deadline: 15 Feb 2013
Start date: -

                              **** CALL FOR PARTICIPATION ****

        Word Sense Induction & Disambiguation within an End-User
        Application SemEval 2013 - Task #11


The aim of this task is to provide a framework for the objective
evaluation and comparison of Word Sense Disambiguation and
Induction algorithms in an end-user application, namely Web
Search Result Clustering.

The proposed application is Web Search Result Clustering, a task
consisting of grouping into clusters the snippet results returned
by a search engine for an input query. Results in a given cluster
are assumed to be semantically related to each other and each
cluster is expected to represent a specific meaning of the input

A Word Sense Induction (WSI) system will be asked to identify the
meaning of the input query and cluster the snippets into
semantically-related groups according to their meanings. Instead,
a Word Sense Disambiguation (WSD) system will be requested to
sense-tag the above snippets with the appropriate senses of the
input query and this, again, will implicitly result in a
clustering of snippets (i.e., one cluster per sense). WSD and WSI
systems will then be evaluated in an end-user application, i.e.,
according to their ability to diversify the search results for
the input query. This evaluation scheme, previously proposed for
WSI by Navigli and Crisafulli (2010) and Di Marco and Navigli
(2013), is extended here to WSD and WSI systems and is aimed at
overcoming the limitations of in vitro evaluations. In fact, the
quality of the output clusters will be assessed in terms of their
ability to diversify the snippets across the query meanings.

No training data will be provided.

We will release new test data for this task. The test data will
be created by:

- Manually selecting ambiguous queries of different lengths;
- Querying Google;
- Retrieving the top 64 results for each query;
- Associating each resulting snippet with the most appropriate
  Wikipedia sense (i.e., page) for that query. The annotations
  will be obtained by crowdsourcing+further checks by the

*** An evaluation tool is now available from the task website!

February 15, 2013 - Registration Deadline [for Task Participants]
March 1, 2013 onwards - Start of evaluation period [Task
Dependent] March 15, 2013 - End of evaluation period April 9,
2013 - Paper submission deadline [TBC] April 23, 2013 - Reviews
Due [TBC] May 4, 2013 - Camera ready Due [TBC]

The Semeval-2013 Task #11 website, for signup and details, is:


If interested in the task please join our mailing list for


Roberto Navigli (lastname_(at)_di.uniroma1.it), Sapienza University of
Rome, Italy Daniele Vannella (lastname_(at)_di.uniroma1.it), Sapienza
University of Rome, Italy

R. Navigli, G. Crisafulli. Inducing Word Senses to Improve Web
   Search Result Clustering. Proc. of EMNLP 2010, Massachusets,
   USA, pp. 116-126, 2010.
A. Di Marco, R. Navigli. Clustering and Diversifying Web Search
   Results with Graph-Based Word Sense Induction. Computational
   Linguistics, 39(4), MIT Press, 2013.


- ELSNET mailing list Elsnet-list_(at)_elsnet.org
- To manage your subscription go to:


[ Search | Events calendar | Deadline calendar ]


Page generated 22-06-2018 by Steven Krauwer Disclaimer / Contact ELSNET